Systematic Review: Factors Associated With Risk for and Possible Prevention of Cognitive Decline in Later Life FREE

Persons with cognitive decline are at increased risk for progressing to mild cognitive impairment (MCI) and dementia. Such intellectual deterioration is feared and often broadly affects quality of life because of limitations on functioning in daily life and increased disability. The term cognitive decline actually reflects a continuum of cognitive changes; some are considered to be within the spectrum of normal aging, whereas others exceed expected decline and are categorized as mild impairment. Typically, performance in 1 or more cognitive domains, such as memory, orientation, language, executive function, and praxis, is assessed to determine decline, but the diagnostic threshold between normal and pathologic cognitive changes is imprecise. Pathologic cognitive decline is often referred to as MCI or cognitive impairment without dementia. The diagnostic criteria for MCI and cognitive impairment without dementia are still evolving, but guidelines generally include greater-than-expected cognitive decline for the person's age and education level and no more than mild functional impairment that is insufficient to meet the threshold for a diagnosis of dementia. Each term has multiple subtypes, which reflect multiple causes and outcomes.

Findings from numerous epidemiologic and clinical studies suggest that multiple biological, behavioral, social, and environmental factors may contribute to the risk for cognitive decline. However, few systematic reviews have examined the breadth of evidence on the wide range of factors that are potentially associated with cognitive decline or the evidence about interventions that may slow decline. Our review addresses this gap in the literature to determine whether evidence is sufficient to warrant specific recommendations for behavioral, lifestyle, or pharmaceutical interventions targeted at maintaining cognitive ability in later life. We present evidence on the association between putative risk and protective factors for cognitive decline and the therapeutic or adverse effects of interventions for preserving cognition.

We developed and followed a standard protocol for all steps of this review. A technical report that details our methods, including the literature search strategies, and presents results for all 6 of the original key research questions is available at www.ahrq.gov. The Appendix, provides further details of our methods.

This evidence review addresses 2 of the 6 original key research questions developed by a National Institutes of Health (NIH) Office of Medical Applications of Research (OMAR) State-of-the-Science Conference planning committee. We refined the specific factors and interventions to be reviewed with input from members of the project's technical expert panel, representatives of NIH OMAR, and staff at the Agency for Healthcare Research and Quality (AHRQ).

First question (original key question 2): What factors are associated with the reduction of risk of cognitive decline in older adults?

Second question (original key question 4): What are the therapeutic and adverse effects of interventions to improve or maintain cognitive ability or function? Are there different outcomes in identifiable subgroups?

We searched for English-language publications in MEDLINE and the Cochrane Database of Systematic Reviews from 1984 through 27 October 2009. For topics with a recent, good-quality systematic review, we updated the review with primary literature published from 1 year before the search end date through 27 October 2009. We supplemented electronic searching by examining the bibliographies of reviews and primary studies. We developed a separate search strategy for the genetics literature by examining the HuGEpedia (1) and AlzGene (2) databases to identify relevant systematic reviews for genes identified as being of special interest, in consultation with the technical expert panel. We included studies of adults 50 years or older who were drawn from general populations in economically developed countries. We required a sample size of at least 300 for cohort studies and 50 for randomized, controlled trials (RCTs). We also required at least 1 year between exposure and final outcomes assessment. For the question on the factors associated with reducing the risk for cognitive decline, we restricted the study design to prospective cohort studies, except for traumatic brain injury, toxic environmental exposures, and sleep apnea, for which we also searched case–control designs because of the absence or limited number of cohort studies. For the question on the therapeutic effects of interventions to improve or maintain cognition, we restricted the study design to RCTs.

Two reviewers independently screened the title and abstract of each citation by using standardized guidelines based on the prespecified inclusion and exclusion criteria. Articles that either reviewer deemed potentially eligible underwent full-text screening. At the full-text screening stage, 2 independent reviewers read each article to determine whether it met eligibility criteria. We resolved disagreements by consensus.

We extracted information on applicability, quality elements, intervention and exposure details, and outcomes. At least 1 reviewer confirmed all data extraction. We developed separate criteria for assessing the methodological quality of included systematic reviews, RCTs, and observational studies (Appendix). In brief, we used the guidelines described in the User's Guides to the Medical Literature on systematic reviews (3) for systematic reviews. The Appendix provides additional details. For RCTs, we used the key criteria described in the AHRQ methods manual for comparative effectiveness reviews (4), adapted to this specific topic. To assess individual observational studies, we adapted a basic set of quality criteria used in previous AHRQ evidence reports (5 - 6). These criteria concern the methods used to select the cohort, the adequacy of the sample size, the methods used to ascertain exposure status and outcomes, the adequacy and completeness of follow-up, and the appropriateness of the analytic methods used (Appendix).

We used principles from the GRADE (Grading of Recommendations Assessment, Development, and Evaluation) working group to summarize the quality of the evidence overall for each factor as low, moderate, or high, reflecting the confidence that the estimate of effect is correct. The GRADE approach considers the body of evidence for each outcome and assigns initial ratings of low to observational studies and high to RCTs. These ratings may be modified by detailed study design, consistency, strength of association, dose–response effect, directness, precision, and whether all plausible confounding would reduce a demonstrated effect. One investigator, who is also a member of the GRADE working group, initially assigned the ratings. Another investigator reviewed them, and we discussed disagreements until consensus was reached.

Synthesizing studies for these key questions was particularly challenging, because cognitive decline can be classified categorically by meeting specific criteria, such as those for MCI, or by exceeding a threshold for decline on 1 or more cognitive measures, typically a global measure, such as the Mini-Mental State Examination (MMSE). Cognitive decline may also be examined by using continuous measures of global cognition, domain-specific measures (such as memory or processing speed), or composites of multiple measures. Because of the heterogeneity of the continuous measures reported and the large scope of the review, we evaluated the relevance of studies that reported continuous measures for each exposure. When more than one half of the studies used categorical outcomes to address the question and the conclusions from the studies that used categorical and continuous outcomes were generally consistent, we did not provide detailed summaries of the studies that reported continuous outcomes. We did not quantitatively summarize the data because of the marked heterogeneity of the categorical outcomes.

An NIH OMAR State-of-the Science Conference planning committee developed the key questions. We refined our list of specific factors and interventions to review with input from members of the project's technical expert panel, representatives of NIH OMAR, and staff at the AHRQ. The AHRQ provided copyright release for this manuscript. Both AHRQ and NIH OMAR representatives were informed of key methodological decisions but did not participate in the literature search, data analysis, or interpretation.

We screened 6907 titles and abstracts and evaluated 1626 full-text articles for our overall review. Of these, 250 primary publications met our eligibility criteria, of which 127 studies addressed the key question specific to factors associated with reducing the risk for cognitive decline and 22 focused on the therapeutic and adverse effects of interventions to improve or maintain cognitive ability or function (Figure). In addition, 25 systematic reviews met the eligibility criteria for the larger report (7); of these, 16 were included for the questions addressed here. We grouped the factors identified by the OMAR planning committee into 5 major categories: nutritional; medical (including medical conditions and prescription and nonprescription medications); social, economic, or behavioral; toxic environmental exposures; and genetic. Table 1 (8 - 114) summarizes the details of the evidence of the association between cognitive decline and each factor that had only observational data. Table 2 (115 - 174) provides the same details for factors that had both observational and RCT data.

The studies that we reviewed defined cognitive decline in different ways. About 10% defined cognitive decline as incident MCI or cognitive impairment without dementia. An estimated 40% of the studies defined it on the basis of change in performance on a single cognitive measure, typically a brief general cognitive function measure. About one half of these studies defined cognitive change as a continuous outcome, whereas the other half defined it as a categorical variable. Another 40% of the included studies assessed cognitive decline on the basis of multiple neuropsychological measures. Most of these studies reported decline as a continuous outcome. Some of these studies reported results for both the individual cognitive measures and a global composite measure that combined some or all of the tests. The specific cognitive tests varied across studies but typically included tests in several of the following domains: global cognitive function, verbal memory, visual memory, verbal fluency, naming, speed of processing, attention, executive function, working memory, and reasoning. Finally, about 10% of the studies defined decline on the basis of a composite global index of performance on all tests combined.

Our systematic review of factors associated with cognitive decline highlighted several issues that made drawing firm conclusions from the available evidence difficult. The definitions and assessments of cognitive decline were not standardized across studies, which limited quantitative synthesis of the data. Also, validation of the type and degree of exposure was limited or absent for many of the factors. When significant differences were noted, the magnitude of the actual differences in outcomes between the exposed and unexposed groups were often small and were evident for only a small portion of the many cognitive measures. In addition, the cognitive measures associated with decline differed across studies.

We found few consistent risk or protective associations from the observational studies (Tables 1 and 2). Summary estimates of the strength of the association were available only for tobacco use and diabetes. A systematic review on tobacco use (175) included a meta-analysis on 3 studies that used a predetermined threshold to define cognitive decline on the MMSE. Current smokers were more likely to exhibit cognitive decline than former smokers or nonsmokers (relative risk, 1.41 [95% CI, 1.16 to 1.71]). A systematic review on diabetes (17) included a meta-analysis on 6 studies that used either the MMSE or the Modified MMSE, with follow-up that ranged from 2 to 6 years. The 6 studies used different predetermined definitions of cognitive decline on these tests. Diabetic persons were more likely than nondiabetic persons to show cognitive decline on the MMSE (odds ratio, 1.2 [CI, 1.05 to 1.4]). Among the other studies on diabetes that we identified, findings on specific cognitive functions differed. For example, one study (20) reported a diabetes-associated decline on tests of verbal memory and processing speed, whereas another study (21) reported no association between diabetes and verbal memory or processing speed but did find that verbal fluency decreased more among diabetic persons than nondiabetic persons. Other factors that showed a likely association with increased risk for cognitive decline were the apolipoprotein E ε4 allele, depressive symptoms, and the metabolic syndrome. Factors probably associated with a lower risk for cognitive decline were involvement in cognitive, physical, or other leisure activities and dietary factors, such as a Mediterranean diet or vegetable or ω-3 fatty acid intake. For most factors associated with cognitive decline, the strength of the association represented by relative risks or hazard ratios were generally in the range of 0.6 to 1.6, often with CIs that approached 1.0.

Table 2 shows that few RCTs assessed the effects of the various factors on cognitive decline. Some of the RCTs for vitamins B, C, and E and folate; antihypertensive drugs; statins; and nonsteroidal anti-inflammatory drugs added cognitive measures as secondary outcomes, often after the start of the trial. Although one of the aims with this key question was to determine whether interventions had a differential effect among subgroups, the few RCTs we identified reported no information on differential outcomes by subgroups.

Trial data showed an effect on cognitive decline from only 2 factors. Of these factors, the trial on cognitive training (176) concluded that domain-specific training was beneficial for maintaining cognition for the targeted domain. At 2 years after the intervention, the effect sizes for cognitive training, calculated as the difference in composite domain-specific mean performance of the groups divided by the intraparticipant SD, were 0.17 for memory (P < 0.001), 0.26 for reasoning (P < 0.001), and 0.87 for speed of processing (P < 0.001). These modest benefits remained at the 5-year follow-up. A single RCT (173) with 170 participants examined an intervention of physical exercise and reported that at 18 months (12 months after treatment), the treatment group showed a decrease of 0.73 point (CI, −1.27 to 0.03 point) on the 70-point Alzheimer's Disease Assessment Scale–Cognitive Subscale (lower scores indicate better performance). The control group showed a decrease of 0.04 point (CI, −0.46 to 0.88 point). The repeated measures analysis of covariance across the 6-, 12-, and 18-month follow-ups showed statistically significant improvement in the exercise group (P = 0.04). Observational studies for some factors, such as ω-3 fatty acids, the Mediterranean diet, vegetable intake, and some leisure activities, suggested a decreased risk for cognitive decline. However, RCT data on these factors are limited to absent and have no GRADE elements (such as a dose–response relationship) to strengthen the confidence in the association, so the quality of evidence for these factors is low.

The results from observational studies and RCTs were discrepant for some factors. For example, the observational studies on ω-3 fatty acids suggested a decreased risk for cognitive decline with higher levels of ω-3 fatty acids, but the single short-term RCT for this factor showed no effect of ω-3 fatty acids on cognitive decline.

To synthesize all evidence for a factor, we reviewed the combined observational and RCT evidence and assigned an overall GRADE quality rating for each factor (Table 3). We rated the evidence for most factors as low-quality, but we note that the quality of evidence may vary substantially even within a given rating level; for example, it varies considerably within the low-quality level. We also note that low-quality evidence indicates that additional research is likely to alter the point estimate or direction of association of a factor with an outcome and is not a statement about how an individual study was designed or performed.

We found high-quality evidence for only 1 factor's effect on cognitive decline. Cognitive training on processing speed or reasoning showed a decreased risk for cognitive decline in the specific targeted cognitive abilities over a 2-year follow-up, and the evidence was rated as high-quality primarily on the basis of 1 RCT (176). Observational studies of self-reported engagement in cognitively stimulating activities suggest an association with less cognitive decline, but this exposure is probably different from the cognitive training used in the RCT. Several factors showed no association with cognitive decline, based on evidence that ranged from low- to high-quality. Table 3 also lists several factors for which the evidence was inadequate to assess the presence of an association. We reached this conclusion if no studies or too few studies had been done on the factor or if the exposure or outcome measures for the studies were too discrepant to synthesize the results.

We conducted this systematic review to summarize the evidence on factors associated with cognitive decline and the therapeutic and adverse effects of interventions to maintain cognitive function. Among the extensive list of putative risk or protective factors we reviewed, few had sufficient evidence from which to draw firm conclusions about their association with cognitive decline. This does not mean that the remaining factors do not play a role in cognitive function in later life, but only that evidence was insufficient to draw such a conclusion.

Cognitive decline in later life has numerous causes, and each may be associated with different risk or protective factors. In addition, some persons initially identified as mildly impaired revert to normal cognition on follow-up assessment. Grouping together all persons with cognitive decline may dilute associations linked with specific subtypes of cognitive decline. In the studies we reviewed, different definitions of cognitive decline made comparisons across studies difficult; some studies used incident MCI or cognitive impairment without dementia as the outcome, whereas others used arbitrary categories of decline on cognitive measures or continuous measures of cognitive decline. Because of the lack of standardization across outcome measures, we focused on studies with categorical outcome measures when the studies with continuous outcome measures showed results consistent with those that used categorical measures. Quantitative synthesis of studies that used continuous outcome measures may provide greater sensitivity to detect group differences than categorical outcomes, but the clinical importance of these differences may be unclear. Future studies need more standardization at various steps of the research process before all available data can be synthesized. However, standardization has its weaknesses and may limit innovations that advance science. The key is to strike a balance between methods that are sufficiently uniform to maximize the use of study results and those that are novel enough to advance the field to the next level.

Issues related to age are central to the interpretation of these results. The incidence of Alzheimer disease increases markedly with age, doubling in rate approximately every 5 years. Cognitive decline is often a prodromal symptom of Alzheimer disease. The older the sample, the more likely that cognitive decline represents prodromal Alzheimer disease, and thus such factors as depression, physical exercise, and cognitive activities that were associated with cognitive decline may actually be a symptom or correlate of Alzheimer disease. Age is also likely to be a key issue with regard to the timing of the exposure. We examined exposures that spanned from childhood to late life. Exposures may influence risk for cognitive decline during certain times. Ideally, the exposure should be measured in different age groups within the same study to control for interstudy variability in measurement, but this may not be realistic because of the long follow-up necessary when studying midlife exposures.

Interventions may also have different effects at different points throughout life or the cognitive decline process. Using an example from the Alzheimer disease intervention literature, hormone replacement therapy has not been shown to lower risk for Alzheimer disease, but the large RCTs on hormone replacement therapy have been conducted in women 65 years or older. It is unknown whether hormone replacement therapy may have cognitive benefits in younger women who are closer to menopause. Although interventions or lifestyle modifications should presumably be done as early as possible, a given intervention may exert its effect during other times in a person's life. Careful consideration of the complex relationships among exposure, age, and disease will be fundamental to understanding the factors that alter risk for cognitive decline.

Our review of the research on this topic indicated several limitations in the extant literature. We have mentioned some of these issues in the Results section, but additional points deserve mention. Epidemiologic studies of complex diseases that use observational data often simultaneously evaluate the association between a range of exposures and the outcome of interest. These studies do not typically make their analyses specific to 1 or 2 factors of interest. How this approach may influence the association between the factor of interest and the outcome is unknown, but we note the issue as one to be considered when interpreting results. Another area of uncertainty relates to how exposure is defined. Studies did not define whether exposure levels were determined a priori, linked to biological rationale or clinical relevance, or informed by previous studies. In addition, the relatively modest effect reported in many studies raised questions about the robustness of the results, concerns about statistical versus clinical significance, and the possibility of residual confounding. The failure of some studies to overtly exclude persons with cognitive impairment or dementia at baseline also complicated our interpretation of the results. Given the small difference in the outcome measures between the exposed and unexposed groups, persons with prodromal or mild dementia may account for some of a factor's association with cognitive decline if that factor is also an early symptom of dementia (for example, depressive symptoms or decreased participation in cognitively engaging activities). Finally, information about the statistical power of samples was typically lacking, so when null findings were reported, questions remained about whether the study was adequately powered to detect a difference.

The preferred research design for investigating potential interventions is an RCT. However, because many potential interventions will be difficult to evaluate in trials, efforts to obtain quality evidence on these factors require well-designed observational studies. In addition, many of the exposures we reviewed probably affect the risk for cognitive decline in combination with other factors, not in isolation. For example, it is not realistic to isolate one nutritional component from others; one factor, such as physical exercise or cognitively stimulating leisure activities, from others that may also be part of a healthier lifestyle; or medical conditions from their treatments. Thus, ideal future interventions would be multidimensional, combine interventions for multiple risk factors, and control for many other factors. Many of the factors that are not amenable to interventions are exposures that one would want to avoid because of their negative effect on outcomes other than cognition, such as smoking, diabetes, hypertension, and few years of education. These and other factors may be most appropriately addressed through public policy interventions (such as educational campaigns or community designs that promote physical activity) and public health interventions (such as educational campaigns that promote a healthy diet).

Our review has limitations. By excluding RCTs with sample sizes of fewer than 50 participants and observational studies with fewer than 300 participants, we may have missed some important evidence, particularly for factors with scant data. By excluding RCTs that lasted for less than 1 year, we may have missed some studies that showed promising short-term results. These studies would not have been adequate to conclude that the intervention was useful for preventing cognitive decline, but they may have provided the impetus to conduct longer trials. The scarcity of articles that used similar cognitive measures precluded any efforts to pool findings statistically. Finally, although our search strategies were similar to those used in other systematic reviews, this is a difficult body of literature to search, and we may have overlooked relevant studies.

In summary, critical improvements in research methods are needed, such as precise, better validated, and more standard exposure measures; more standardized cognitive assessment measures across studies; and studies of longer duration. Studies should also take into account the intensity, duration, and timing of the exposure, because exposures may be more influential and interventions more effective during critical or sensitive periods throughout life. The current literature does not provide adequate evidence to make recommendations for interventions. Previous work has provided a few potential leads, such as cognitive training, physical exercise, and some nutritional patterns. These should now be pursued with potentially novel approaches and increasingly rigorous scientific methods to identify a real signal among the numerous factors throughout the life course that may contribute to complex, late-life cognitive disorders.

This appendix describes further details of the methods we used for this systematic review. We addressed 6 questions in the original review:

Key question 1: What factors are associated with the reduction of risk of Alzheimer disease?

Key question 2: What factors are associated with the reduction of risk of cognitive decline in older adults?

Key question 3: What are the therapeutic and adverse effects of interventions to delay the onset of Alzheimer disease? Are there differences in outcomes among identifiable subgroups?

Key question 4: What are the therapeutic and adverse effects of interventions to improve or maintain cognitive ability or function? Are there different outcomes in identifiable subgroups?

Key question 5: What are the relationships between the factors that affect Alzheimer disease and the factors that affect cognitive decline?

Key question 6: If recommendations for interventions cannot be made currently, what studies need to be done to provide the quality and strength of evidence necessary to make such recommendations to individuals?

This systematic review focuses on key questions 2 and 4.

For all questions, we were interested in adults 50 years or older who were drawn from general populations. For key questions 1 and 2, we focused on factors that are not amenable to randomization (such as hypertension) and limited our review to observational studies. For key questions 3 and 4, we prioritized RCTs, but because the evidence was often sparse or limited to selected samples, we supplemented trial data with evidence from observational studies when necessary. For key question 5, we were interested in the consistency of findings for each exposure or intervention on risk for Alzheimer disease and cognitive decline.

Appendix Table 1 lists the exposures and interventions that we evaluated.

Our analytic framework (Appendix Figure) describes the progression from normal cognition, to the initiation of subclinical pathophysiologic processes, to cognitive decline that is beyond what is expected for normal aging, and finally to Alzheimer disease. Although primary prevention may be possible before the initiation of pathophysiologic processes that lead to cognitive decline, no well-validated methods exist to measure these processes in the absence of cognitive decline. Therefore, we did not consider effects of the candidate factors on changes in pathophysiologic processes. The Appendix Figure shows the potential for treatment interventions to affect cognitive decline and the risk for Alzheimer disease.

After discussion with the technical expert panel, we generated article inclusion and exclusion criteria for the key questions (Appendix Table 2). Because of the many factors and interventions to review, we searched initially for good-quality systematic reviews. We included primary literature to update eligible reviews or when good-quality reviews were unavailable. We limited the primary literature to English-language comparative studies that enrolled adults 50 years or older who were drawn from general populations in economically developed countries. We defined general populations as those drawn primarily from noninstitutionalized community settings or general medical populations. With 3 exceptions, we limited observational studies to longitudinal designs in which the risk factor or intervention was measured before the outcome. We made exceptions for traumatic brain injury and toxic environmental exposures, because of the difficulty of studying these factors longitudinally, and for sleep apnea, because of the absence of cohort studies. Because Alzheimer disease is a relatively uncommon event, we required a sample size greater than 300 to focus on studies with higher statistical power. Because of the long subclinical prodromal phase of Alzheimer disease, we required at least 2 years between exposure and outcomes assessments in Alzheimer disease studies. To provide sufficient time for meaningful change to occur, we required at least 1 year for studies of cognitive decline.

On the basis of our inclusion and exclusion criteria, we generated a list of Medical Subject Heading (MeSH) search terms, supplemented by keyword searches, which we used to search MEDLINE. We developed search terms and strategies in consultation with a medical librarian. Our original report (7) details the exact search strategies we used. In addition to MEDLINE, we searched the Cochrane Database of Systematic Reviews to identify relevant systematic reviews. For topics with a recent good-quality systematic review, we updated the search by identifying relevant primary literature published from 1 year before the search end date through 27 October 2009. (The 1-year overlap is necessary because of delays between publication in journals and availability for searching in MEDLINE.) We included relevant older literature that was missed or excluded in previous reviews if it met our eligibility criteria.

When we did not identify a relevant good-quality review, we searched the primary literature to include studies from 1984 through 27 October 2009. We supplemented our electronic searching by examining the bibliographies of reviews and primary studies. Because of the large volume of literature and the availability of specialized registries for genetic studies, we developed a separate search strategy for this literature. We examined the HuGEpedia and AlzGene databases to identify relevant systematic reviews for genes identified as being of special interest, in consultation with the technical expert panel.

Using the prespecified inclusion and exclusion criteria, 2 reviewers independently examined titles and abstracts for potential relevance to the key questions. Articles included by either reviewer underwent full-text screening. At the full-text screening stage, 2 independent reviewers read each article to determine whether it met eligibility criteria. We resolved disagreements by consensus. At the full-text review stage, simple agreement was 84% and median chance corrected agreement had a κ value of 0.63 (range, 0.40 to 1.0). We included articles that met our eligibility criteria for data abstraction.

One reviewer abstracted data from published reports into evidence tables and another overread them. The data elements we abstracted included descriptors to assess applicability, quality elements, intervention and exposure details, and outcomes. We resolved disagreements by consensus or by obtaining a third reviewer's opinion when consensus could not be reached. The final evidence tables are intended to provide sufficient information so that readers can understand the study and determine its quality and are available as part of the original report (7).

We developed separate criteria for assessing the methodological quality of included systematic reviews, RCTs, and observational studies. In brief, for systematic reviews, we assessed the comprehensiveness of the search strategy, the description and appropriateness of inclusion criteria, whether primary studies were assessed for quality and the adequacy of the quality measure, the reproducibility of the methods to assess studies, whether the results of relevant studies were combined appropriately, whether heterogeneity and publication bias were assessed, and whether the conclusions were supported by the data presented.

For RCTs, we used the key criteria described in the AHRQ methods manual for comparative effectiveness reviews (4), adapted to this specific topic. These criteria are adequacy of randomization and allocation concealment, comparability of groups at baseline, adequacy of blinding, completeness of follow-up and differential loss to follow-up, appropriate addressing of incomplete data, validity of outcome measures, and degree of conflict of interest.

To assess individual observational studies, we adapted a basic set of quality criteria used in previous AHRQ evidence reports (5 - 6). These criteria concern the methods used to select the cohort, the adequacy of the sample size, the methods used to ascertain exposure status and outcomes, the adequacy and completeness of follow-up, and the appropriateness of the analytic methods used. Abstractors assigned a rating of “yes,” “partially,” “no,” or “can't tell” to each item and provided a brief rationale for their decisions. We did not attempt to assign a summary quality score (such as “A, B, C” or “good, fair, poor”) to individual RCTs or observational studies, because no evidence indicates that the use of any particular quality scoring system has a substantial effect on the results of systematic reviews (177). In addition, our experience has been that it is more helpful to identify consistent and specific quality issues that affect most of the included studies (such as those that concern sample size, analytic methods, or ascertainment bias) to guide future research, rather than relying on a global quality score.

We used principles from the GRADE working group to summarize the quality of evidence for each factor as low, moderate, or high. The GRADE approach considers the body of evidence for each outcome and assigns an initial rating of low quality to observational studies and high quality to RCTs. These initial ratings may be modified by detailed study design, consistency, strength of association, dose–response effect, directness, precision, and whether all plausible confounding would reduce a demonstrated effect. A high rating reflects high confidence that the evidence reflects the true effect and that further research is very unlikely to change our confidence in the estimate of effect. A moderate rating reflects moderate confidence that the evidence reflects the true effect but a recognition that further research may change both our confidence in the estimate of effect and the estimate. A low rating reflects low confidence that the evidence reflects the true effect and indicates that further research is likely to change both our confidence in the estimate of effect and the estimate. A rating of “insufficient evidence” means that evidence either is unavailable or does not permit estimation of an effect. At least 2 reviewers judged the strength of evidence; we reached our final ratings by consensus.

When we identified good-quality systematic reviews, we summarized their findings in narrative form. We descriptively summarized any new studies identified since the systematic review was published in a table that included the study sample, exposure classification, duration of follow-up, adjustment for confounding, and primary outcomes. We evaluated whether the new evidence was likely to change estimates from the previous review by considering the precision and stability of estimates from the original review, the number and size of the new studies relative to those in the original review, the quality of the new studies, and the consistency in estimates and conclusions between the new evidence and the original review. After considering these issues, we updated previous meta-analyses when substantial new evidence was available and a new summary estimate was likely to lead to different conclusions. We performed primary meta-analysis when the studies were conceptually homogeneous and the needed data were available for the summary estimate. Because meta-analysis of observational studies may give spurious precision (178), we applied meta-analysis to observational data only when studies were of high quality and conceptually homogeneous (for example, containing similar participants, exposure, or outcomes).

Synthesizing studies that evaluated cognitive decline was particularly challenging. Cognitive decline can be classified categorically by meeting proposed criteria, such as those for MCI, or by exceeding a threshold on a global cognitive measure, such as the MMSE. We prioritized these categorical outcomes because they are often more clinically meaningful. Cognitive decline may also be examined by using continuous measures of global, isolated, domain-specific measures (such as memory or processing speed) or by composites of multiple measures for a domain. Many of these measures have not been demonstrated to be responsive to change, and any changes observed may be of uncertain clinical significance. Because of the heterogeneity of the continuous measures reported and the large scope of our review, we evaluated the relevance of studies that reported continuous measures for each exposure. When an adequate number of studies that used categorical outcomes addressed the question and when the results were similar from the categorical and continuous outcome studies, we did not provide detailed summaries of the studies that reported continuous outcomes.