Safi U. Khan, MD; Muhammad U. Khan, MD; Haris Riaz, MD; Shahul Valavoor, MD; Di Zhao, PhD; Lauren Vaughan, MD; Victor Okunrintemi, MD, MPH; Irbaz Bin Riaz, MD, MS; Muhammad Shahzeb Khan, MD; Edo Kaluski, MD; M. Hassan Murad, MD; Michael J. Blaha, MD, MPH; Eliseo Guallar, MD, DrPH; Erin D. Michos, MD, MHS
Financial Support: Drs. Zhao, Guallar, and Michos are funded by the Blumenthal Scholars Fund in Preventive Cardiology at Johns Hopkins University.
Disclosures: Dr. Blaha reports grants from the National Heart, Lung, and Blood Institute, the Food and Drug Administration, the American Heart Association, Amgen, and the Aetna Foundation and personal fees from the Food and Drug Administration, Amgen, Sanofi, Novartis, Novo Nordisk, and Bayer outside the submitted work. Authors not named here have disclosed no conflicts of interest. Disclosures can also be viewed at www.acponline.org/authors/icmje/ConflictOfInterestForms.do?msNum=M19-0341.
Editors' Disclosures: Christine Laine, MD, MPH, Editor in Chief, reports that her spouse has stock options/holdings with Targeted Diagnostics and Therapeutics. Darren B. Taichman, MD, PhD, Executive Editor, reports that he has no financial relationships or interests to disclose. Cynthia D. Mulrow, MD, MSc, Senior Deputy Editor, reports that she has no relationships or interests to disclose. Jaya K. Rao, MD, MHS, Deputy Editor, reports that she has stock holdings/options in Eli Lilly and Pfizer. Catharine B. Stack, PhD, MS, Deputy Editor, Statistics, reports that she has stock holdings in Pfizer, Johnson & Johnson, and Colgate-Palmolive. Christina C. Wee, MD, MPH, Deputy Editor, reports employment with Beth Israel Deaconess Medical Center. Sankey V. Williams, MD, Deputy Editor, reports that he has no financial relationships or interests to disclose. Yu-Xiao Yang, MD, MSCE, Deputy Editor, reports that he has no financial relationships or interest to disclose.
Corresponding Author: Safi U. Khan, MD, West Virginia University, 1 Medical Center Drive, Morgantown, WV 26505; e-mail, email@example.com.
Current Author Addresses: Drs. S.U. Khan, M.U. Khan, and Valavoor: West Virginia University, One Medical Center Drive, Morgantown, WV 26508.
Dr. H. Riaz: Cleveland Clinic, 9500 Euclid Avenue, Cleveland, OH 44195.
Drs. Zhao and Guallar: Johns Hopkins Bloomberg School of Public Health, Welch Center, 2024 East Monument Street, Suite 2600, Baltimore, MD 21205.
Drs. Vaughan and Okunrintemi: East Carolina University, 2100 Stantonsburg Road, Greenville, NC 27834.
Drs. I.B. Riaz and Murad: Mayo Clinic, 200 1st Street Southwest, Rochester, MN 55901.
Dr. M.S. Khan: 903 South Ashland Avenue, Apt 1206, Chicago, IL 60607.
Dr. Kaluski: Guthrie Robert Packer Hospital, One Guthrie Square, Sayre, PA 18840.
Dr. Blaha: Johns Hopkins School of Medicine, 600 North Wolfe Street, Blalock 524-D1, Baltimore, MD 21287.
Dr. Michos: Johns Hopkins School of Medicine, 600 North Wolfe Street, Blalock 524-B, Baltimore, MD 21287.
Author Contributions: Conception and design: S.U. Khan.
Analysis and interpretation of the data: S.U. Khan, D. Zhao, E. Guallar, E.D. Michos.
Drafting of the article: S.U. Khan, H. Riaz, L. Vaughan, V. Okunrintemi, E. Guallar, E.D. Michos.
Critical revision of the article for important intellectual content: S.U. Khan, I.B. Riaz, M.S. Khan, M.H. Murad, M.J. Blaha, E. Guallar, E.D. Michos.
Final approval of the article: S.U. Khan, M.U. Khan, H. Riaz, S. Valavoor, D. Zhao, L. Vaughan, V. Okunrintemi, I.B. Riaz, M.S. Khan, E. Kaluski, M.H. Murad, M.J. Blaha, E. Guallar, E.D. Michos.
Provision of study materials or patients: S.U. Khan, M.U. Khan, S. Valavoor.
Statistical expertise: S.U. Khan, M.H. Murad, D. Zhao, E. Guallar, E.D. Michos.
Administrative, technical, or logistic support: E.D. Michos.
Collection and assembly of data: S.U. Khan, M.U. Khan, S. Valavoor.
The role of nutritional supplements and dietary interventions in preventing mortality and cardiovascular disease (CVD) outcomes is unclear.
To examine evidence about the effects of nutritional supplements and dietary interventions on mortality and cardiovascular outcomes in adults.
PubMed, CINAHL, and the Cochrane Library from inception until March 2019; ClinicalTrials.gov (10 March 2019); journal Web sites; and reference lists.
English-language, randomized controlled trials (RCTs) and meta-analyses of RCTs that assessed the effects of nutritional supplements or dietary interventions on all-cause mortality or cardiovascular outcomes, such as death, myocardial infarction, stroke, and coronary heart disease.
Two independent investigators abstracted data, assessed the quality of evidence, and rated the certainty of evidence.
Nine systematic reviews and 4 new RCTs were selected that encompassed a total of 277 trials, 24 interventions, and 992 129 participants. A total of 105 meta-analyses were generated. There was moderate-certainty evidence that reduced salt intake decreased the risk for all-cause mortality in normotensive participants (risk ratio [RR], 0.90 [95% CI, 0.85 to 0.95]) and cardiovascular mortality in hypertensive participants (RR, 0.67 [CI, 0.46 to 0.99]). Low-certainty evidence showed that omega-3 long-chain polyunsaturated fatty acid (LC-PUFA) was associated with reduced risk for myocardial infarction (RR, 0.92 [CI, 0.85 to 0.99]) and coronary heart disease (RR, 0.93 [CI, 0.89 to 0.98]). Folic acid was associated with lower risk for stroke (RR, 0.80 [CI, 0.67 to 0.96]; low certainty), whereas calcium plus vitamin D increased the risk for stroke (RR, 1.17 [CI, 1.05 to 1.30]; moderate certainty). Other nutritional supplements, such as vitamin B6, vitamin A, multivitamins, antioxidants, and iron and dietary interventions, such as reduced fat intake, had no significant effect on mortality or cardiovascular disease outcomes (very low– to moderate-certainty evidence).
Suboptimal quality and certainty of evidence.
Reduced salt intake, omega-3 LC-PUFA use, and folate supplementation could reduce risk for some cardiovascular outcomes in adults. Combined calcium plus vitamin D might increase risk for stroke.
Evidence search and selection.
RCT = randomized controlled trial.
Effects of nutritional supplements and dietary interventions on all-cause mortality.
ALA = α-linolenic acid; LC-PUFA = long-chain polyunsaturated fatty acid.
* Updated meta-analysis after inclusion of new clinical trials.
Effects of nutritional supplements and dietary interventions on cardiovascular mortality.
Evidence map of availability and appraisal of certainty of the evidence.
The In the Clinic® slide sets are owned and copyrighted by the American College of Physicians (ACP). All text, graphics, trademarks, and other intellectual property incorporated into the slide sets remain the sole and exclusive property of the ACP. The slide sets may be used only by the person who downloads or purchases them and only for the purpose of presenting them during not-for-profit educational activities. Users may incorporate the entire slide set or selected individual slides into their own teaching presentations but may not alter the content of the slides in any way or remove the ACP copyright notice. Users may make print copies for use as hand-outs for the audience the user is personally addressing but may not otherwise reproduce or distribute the slides by any means or media, including but not limited to sending them as e-mail attachments, posting them on Internet or Intranet sites, publishing them in meeting proceedings, or making them available for sale or distribution in any unauthorized form, without the express written permission of the ACP. Unauthorized use of the In the Clinic slide sets will constitute copyright infringement.
In this video, Erin D. Michos, MD, MHS and Safi U. Khan, MD, offer additional insight into the article, "Effects of Nutritional Supplements and Dietary Interventions on Cardiovascular Outcomes-An Umbrella Review and Evidence Map."
A. David Smith ,Professor Emeritus of Pharmacology
Department of Pharmacology, University of Oxford, UK
July 21, 2019
Trials of nutritional supplements – do not ignore the subgroups
It is hardly surprising that the conclusions of the review by Khan et al were so negative because the majority, if not all, the trials analysed were conducted on large groups of people and no account seems to have been taken of subgroups within each population. Many subgroups have been identified that respond differently to nutrients and in a trial these should be identified and a pre-specified subgroup analysis should always be done. The single most important aspect for subgroup analysis is the status of the participants in that particular nutrient. Clearly, the subgroup with a high nutrient status is not likely to respond to further exposure to that nutrient, whereas the subgroup with deficiency, or with low-normal status or intake, may well respond (1). In trials on healthy people in high-income countries it is likely that the majority will have a good status in most common nutrients. Such trials should as a matter of design measure the nutrient status in each participant and carry out subgroup analysis of the outcomes according to the status.
Other subgroups have been identified in trials with nutrients. Nutrients often interact with genes and common genetic polymorphisms, such as ApoE 4, can influence the response to nutrients such as vitamin B12 (2) and omega-3 fatty acids (3). The biological effect of a nutrient may be influenced by common medications, for example, anti-platelet drugs interact with the effect of B vitamins on stroke (4). Different nutrients interact with each other: for example the beneficial effect of B vitamins on the brain requires a good omega-3 fatty acid status (5).
It is important that trials of nutritional supplements should recognise the heterogeneity of the population and stratify for different subgroups in case one or other subgroup responds differently. Until this is done, we should not dismiss the use of supplements for improving health outcomes since we might well be denying benefit to sections of the population.
1. Morris MC, Tangney CC. A potential design flaw of randomized trials of vitamin supplements. JAMA. 2011;305(13):1348-9. doi: 10.1001/jama.2011.383
2. Vogiatzoglou A, Smith AD, Nurk E, Drevon CA, Ueland PM, Vollset SE, et al. Cognitive function in an elderly population: Interaction between vitamin B12 status, depression, and apolipoprotein E E4: The Hordaland Homocysteine Study. Psychosom Med. 2013;75(1):20-9. doi: 10.1097/PSY.0b013e3182761b6c
3. Hennebelle M, Plourde M, Chouinard-Watkins R, Castellano CA, Barberger-Gateau P, Cunnane SC. Ageing and apoE change DHA homeostasis: relevance to age-related cognitive decline. Proc Nutr Soc. 2014;73(1):80-6. doi: 10.1017/s0029665113003625
4. Arshi B, Ovbiagele B, Markovic D, Saposnik G, Towfighi A. Differential effect of B-vitamin therapy by antiplatelet use on risk of recurrent vascular events after stroke. Stroke. 2015;46(3):870-3. doi: 10.1161/strokeaha.114.006927
5. Jernerén F, Elshorbagy AK, Oulhaj A, Smith SM, Refsum H, Smith AD. Brain atrophy in cognitively impaired elderly: the importance of long-chain omega-3 fatty acids and B vitamin status in a randomized controlled trial. Am J Clin Nutr. 2015;102(7):215-21. doi: 10.3945/ajcn.114.103283
Jesper M Kivelä
University of Helsinki, Helsinki, Finland
August 5, 2019
Does statistics in meta-analyses matter?
Khan and coworkers reported that reduced salt intake in normotensive persons reduced the risk of all-cause mortality with risk ratio (RR) (95% CI) of 0.90 (0.85 to 0.95) based on three studies with a small number of events (n=79) (1). This result was based on reanalysis of a Cochrane review in which RR (95% CI) was 0.90 (0.58 to 1.40) for all-cause mortality at longest follow-up (Analysis 1.2) (2). The width of CI is s shorter with random-effects (RE) model used by Khan and coworkers compared to fixed-effect (FE) model used in the Cochrane review (2). Indeed, in some situations the RE model by Hartung-Knapp/Sidik-Jonkman (HKSJ) can result narrower CI compared to the FE model, and an ad hoc modification has been introduced (3, 4). Contrary, with this modification 95% CI would be 0.34 to 2.36 based on reanalysis of three studies [Analysis 1.2 in (2)]. However, confidence intervals can be too wide, especially when there are few studies (4). A between-study heterogeneity was estimated to be zero, which is implausible even for outcomes like all-cause mortality based on what we know from a large sample of meta-analyses conducted within Cochrane reviews (5). Therefore, I conducted a Bayesian RE meta-analysis with R bayesmeta package using a prior distribution for between-study variance of 0.030 [i.e. all-cause mortality as outcome and non-pharma vs. non-pharma as comparison according to (5)]. Further, a uniform prior for mean effect was used. In sensitivity analysis, a normal prior of RR 1 with standard deviation of 0.35 (i.e. 5% probability for RR to be outside 95% interval of 0.5 to 2.0) was used.The Bayesian RE model yielded a posterior mean for RR (95% credible interval) of 0.90 (0.53 to 1.54) based on reanalysis of three studies [Analysis 1.2 in (2)]. There was 65% probability that salt reduction has some benefit but also 35% probability for some harm in the context of premature deaths. In sensitivity analysis, posterior mean for RR was 0.94 (95% credible interval 0.62 to 1.42).In conclusion, robustness of results in meta-analyses should be examined with a more than one statistical method. The certainty of evidence considering salt reduction in normotensive persons and all-cause mortality should be downgraded. References1. Khan SU, Khan MU, Riaz H, et al. Effects of Nutritional Supplements and Dietary Interventions on Cardiovascular Outcomes: An Umbrella Review and Evidence Map. Ann Intern Med. 2019 Jul 9. doi: 10.7326/M19-0341.2. Adler AJ, Taylor F, Martin N, Gottlieb S, Taylor RS, Ebrahim S. Reduced dietary salt for the prevention of cardiovascular disease. Cochrane Database Syst Rev. 2014;12:CD009217.3. Wiksten A, Rücker G, Schwarzer G. Hartung-Knapp method is not always conservative compared with fixed-effect meta-analysis. Stat Med. 2016;35:2503-15.4. Röver C, Knapp G, Friede T. Hartung-Knapp-Sidik-Jonkman approach and its modification for random-effects meta-analysis with few studies. BMC Med Res Methodol. 2015;15:99.5. Turner RM, Jackson D, Wei Y, Thompson SG, Higgins JP. Predictive distributions for between-study heterogeneity and simple methods for their application in Bayesian meta-analysis. Stat Med. 2015;34:984-98.
Safi U. Khan1; Erin D. Michos2
1. Department of Medicine, West Virginia University, USA 2. Division of Cardiology, John Hopkins School of Medicine, USA
August 7, 2019
Subgroup Analyses: Take It with a Pinch of Salt
Randomized controlled trials are generally powered to examine the efficacy or safety of an intervention in heterogenous group of patients and subgroup analyses are usually performed as post hoc analyses1. Multiple subgroup analyses can potentially generate false positive results; which may further complicate the interpretation of the results when analyses are conducted based on dichotomization of variables, such as low nutrient status versus high nutrient status in this case1. As such a process can lead to divergent results, one way to ensure that probability of a false-positive finding is not >5%, correction should be made for multiple testing (e.g. Bonferroni adjustment of P-values)2. While, prespecifying the subgroups in a trial protocol might provide some credibility to the subgroups results, even the prespecified analyses are at a higher risk of false positive results secondary to multiple testing, and should be interpreted cautiously1. For instance, subgroup analyses based on baseline 25-OH vitamin D levels were specified a priori in VITAL trial, but multiple hypothesis testing and formal adjustment of P-values or confidence intervals were not done. Regardless, vitamin D supplementation showed null effect compared with control among participants with baseline serum 25-OH vitamin D (<20 ng/mL and ≥ 20 ng/mL, or <median of 31 ng/mL or ≥ 31 ng/mL; p=interaction 0.75 and 0.42, respectively)3.
In view of these issues, generating subgroup analyses based on trial level information in meta-analyses can be extremely misleading. Therefore, we refrained from generating subgroup analyses and focused on providing a broad-based evidence for nutritional supplements and dietary interventions4. We stated in our discussion that “because the focus of our study was to provide broad-based evidence for various nutritional supplements and dietary interventions using existing meta-analyses and trial-level information, we could not analyze interventions according to important subgroups, such as sex, body mass index, lipid values, blood pressures thresholds, diabetes, and history of CVD.” We also acknowledged the limitations related to health and socioeconomic status, interventions and lack of dose-response analyses and discussed results in the context of these limitations. For instance, in case of folate supplementation, we discussed that it remains unclear that whether the benefit of folate supplementation can be generalized to US population, which has folate fortification. This umbrella review examined the effects of “routine use of supplements and dietary interventions on cardiovascular outcomes” and highlights the issue that the current evidence reporting the effects of these interventions was suboptimal.
1. Wang R, Lagakos SW, Ware JH, Hunter DJ, Drazen JM. Statistics in medicine--reporting of subgroup analyses in clinical trials. The New England journal of medicine. 2007;357(21):2189-2194.
2. Jafari M, Ansari-Pour N. Why, When and How to Adjust Your P Values? Cell J. 2019;20(4):604-607.
3. Manson JE, Cook NR, Lee IM, et al. Vitamin D Supplements and Prevention of Cancer and Cardiovascular Disease. The New England journal of medicine. 2019;380(1):33-44.
4. Khan SU, Khan MU, Riaz H, et al. Effects of Nutritional Supplements and Dietary Interventions on Cardiovascular Outcomes: An Umbrella Review and Evidence MapEffects of Supplements and Dietary Interventions on Cardiovascular Outcomes. Annals of internal medicine. 2019.
Innovations and Evidence-Based Medicine Development, EBSCO Health; Triad Hospitalist Group, Cone Health
August 21, 2019
Deciphering discrepancy in meta-analytic results: The model matters
I applaud Khan and colleagues taking on the substantial task of performing an umbrella review on nutritional supplements and dietary interventions on cardiovascular outcomes.(1) However, their results for salt reduction in patients without hypertension for the outcome of all-cause mortality deserve additional scrutiny. They cite a Cochrane review(2) as the sole source for this outcome, and the number of events and participants are consistent with the Cochrane review's analysis for all-cause mortality at longest follow-up (Analysis 1.2.1). However, although the risk ratio (RR) point estimates are the same in the Cochrane review and Khan and colleagues' review (RR of 0.90), the 95% confidence intervals (CIs) differ substantially (0.58 to 1.40 versus 0.85 to 0.95, respectively). The considerably wider CI in the Cochrane review raises immediate suspicion. The Cochrane review used a fixed-effects model and Khan and colleagues used a random-effects model, and it does not make sense for the fixed-effects model to yield a wider CI than the random-effects model. This oddity is due to Khan and colleagues' use of the Hartung-Knapp adjustment. In many cases, use of the Hartung-Knapp adjustment achieves its intended effect: more conservative estimates (i.e., estimates with wider CIs). However, this is not always the case, which has understandably resulted in some recommending sensitivity analyses if one uses the Hartung-Knapp adjustment, or perhaps even an "adaptive meta-analysis", where the underlying data drive decisions about which analytic model to present as the primary model.(3,4)Repeating the analysis with Khan and colleagues' methods but excluding the Hartung-Knapp adjustment yields results identical to the Cochrane review (analyses carried out with R version 3.6.1 and meta version 4.9-6; data and analytic code available on request). The consistency of the results is expected given no statistical heterogeneity is identified in the analysis. The fact that τ^2 = 0 is also a probable cause for why the Hartung-Knapp adjustment resulted in a narrower CI than the fixed-effects model in the Cochrane review.(3,4) Results without the Hartung-Knapp adjustment should be considered more reliable here.I encourage Khan and colleagues to consider conducting additional analyses for their results where relevant based on the above finding. This might be considered an example of "vibration in effects" due to model specification, and although not severe enough to result in a Janus phenomenon, the inferential difference that might result if considering Khan and colleagues' results versus the Cochrane review’s results feels much more substantial than a "vibration".(5)References1. Khan SU, Khan MU, Riaz H, Valavoor S, Zhao D, Vaughan L, et al. Effects of Nutritional Supplements and Dietary Interventions on Cardiovascular Outcomes: An Umbrella Review and Evidence Map. Ann Intern Med. 2019 Aug 6;171(3):190. 2. Adler AJ, Taylor F, Martin N, Gottlieb S, Taylor RS, Ebrahim S. Reduced dietary salt for the prevention of cardiovascular disease. Cochrane Database Syst Rev. 2014 Dec 18;(12):CD009217. 3. Wiksten A, Rücker G, Schwarzer G. Hartung-Knapp method is not always conservative compared with fixed-effect meta-analysis: Hartung-Knapp method is not always conservative compared with fixed-effect meta-analysis. Statist Med. 2016 Jul 10;35(15):2503–15. 4. Jackson D, Law M, Rücker G, Schwarzer G. The Hartung-Knapp modification for random-effects meta-analysis: A useful refinement but are there any residual concerns? Statist Med. 2017 Nov 10;36(25):3923–34. 5. Patel CJ, Burford B, Ioannidis JPA. Assessment of vibration of effects due to model specification can demonstrate the instability of observational associations. Journal of Clinical Epidemiology. 2015 Sep 1;68(9):1046–58.
Khan SU, Khan MU, Riaz H, Valavoor S, Zhao D, Vaughan L, et al. Effects of Nutritional Supplements and Dietary Interventions on Cardiovascular Outcomes: An Umbrella Review and Evidence Map. Ann Intern Med. [Epub ahead of print 9 July 2019]171:190–198. doi: 10.7326/M19-0341
Download citation file:
Published: Ann Intern Med. 2019;171(3):190-198.
Published at www.annals.org on 9 July 2019
Results provided by:
Copyright © 2019 American College of Physicians. All Rights Reserved.
Print ISSN: 0003-4819 | Online ISSN: 1539-3704
Conditions of Use