Dena Zeraatkar, MSc; Mi Ah Han, MD, PhD; Gordon H. Guyatt, MD, MSc; Robin W.M. Vernooij, PhD; Regina El Dib, PhD; Kevin Cheung, MD, MSc; Kirolos Milio, BSc; Max Zworth, BASc; Jessica J. Bartoszko, HBSc; Claudia Valli, MSc; Montserrat Rabassa, PhD; Yung Lee, BHSc; Joanna Zajac, PhD; Anna Prokop-Dorner, PhD; Calvin Lo, BHSc; Malgorzata M. Bala, PhD; Pablo Alonso-Coello, MD, PhD; Steven E. Hanna, PhD; Bradley C. Johnston, PhD
Acknowledgment: The authors thank Thomasin Adams-Webber (Hospital for Sick Children) for her help in designing the search strategy.
Disclosures: Dr. El Dib received a São Paulo Research Foundation (FAPESP) (2018/11205-6) scholarship and funding from the National Council for Scientific and Technological Development (CNPq) (CNPq 310953/2015-4) and the Faculty of Medicine, Dalhousie University. Dr. Johnston received a grant from Texas A&M AgriLife Research to fund investigator-driven research related to saturated and polyunsaturated fats within the 36-month reporting period required by the International Committee of Medical Journal Editors, as well as funding received from the International Life Science Institute (North America) that ended before the 36-month reporting period. Authors not named here have disclosed no conflicts of interest. Disclosures can also be viewed at www.acponline.org/authors/icmje/ConflictOfInterestForms.do?msNum=M19-0655.
Editors' Disclosures: Christine Laine, MD, MPH, Editor in Chief, reports that her spouse has stock options/holdings with Targeted Diagnostics and Therapeutics. Darren B. Taichman, MD, PhD, Executive Editor, reports that he has no financial relationships or interests to disclose. Cynthia D. Mulrow, MD, MSc, Senior Deputy Editor, reports that she has no relationships or interests to disclose. Jaya K. Rao, MD, MHS, Deputy Editor, reports that she has stock holdings/options in Eli Lilly and Pfizer. Catharine B. Stack, PhD, MS, Deputy Editor, Statistics, reports that she has stock holdings in Pfizer, Johnson & Johnson, and Colgate-Palmolive. Christina C. Wee, MD, MPH, Deputy Editor, reports employment with Beth Israel Deaconess Medical Center. Sankey V. Williams, MD, Deputy Editor, reports that he has no financial relationships or interests to disclose. Yu-Xiao Yang, MD, MSCE, Deputy Editor, reports that he has no financial relationships or interest to disclose.
>Reproducible Research Statement: Study protocol: Registered with PROSPERO (CRD42017074074). Statistical code and data set: Available from Ms. Zeraatkar (e-mail, email@example.com). For sample code, see Supplement 2.
Corresponding Author: Bradley C. Johnston, PhD, Department of Community Health and Epidemiology, Faculty of Medicine, Dalhousie University, Room 404, 5790 University Avenue, Halifax, Nova Scotia B3J 0E4, Canada; e-mail, firstname.lastname@example.org.
Current Author Addresses: Ms. Zeraatkar, Drs. Guyatt and Hanna, and Ms. Bartoszko: Department of Health Research Methods, Evidence, and Impact, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S 4K1, Canada.
Dr. Han: Department of Preventive Medicine, College of Medicine, Chosun University, 309 Philmun-daero, Dong-gu, Gwangju 61452, Korea.
Dr. Vernooij: Department of Research, Netherlands Comprehensive Cancer Organisation, Godebaldkwartier 419, Utrecht 3511 DT, the Netherlands.
Dr. El Dib: Institute of Science and Technology, São José dos Campos, Avenida Engenheiro Francisco José Longo, 777, Jardim São Dimas, São José dos Campos, 12245-000, Spain.
Dr. Cheung: 114 Loganberry Crescent, Toronto, Ontario M2H 3H1, Canada.
Mr. Milio: 592 Regal Place, Waterloo, Ontario N2V 2G3, Canada.
Mr. Zworth: 28 York Downs Drive, Toronto, Ontario M3H 1J1, Canada.
Ms. Valli and Drs. Rabassa and Alonso-Coello: Iberoamerican Cochrane Centre, (IIB Sant Pau-CIBERESP), Carrer de Sant Antoni Maria Claret, 167, Barcelona, 08025, Spain.
Mr. Lee: 30 White Lodge Crescent, Richmond Hill, Ontario L4C 9A1, Canada.
Drs. Zajac, Prokop-Dorner, and Bala: Jagiellonian University Medical College, Department of Hygiene and Dietetics, Kopernika 7 Street, 31-034 Krakow, Poland.
Mr. Lo: 556 Amarone Court, Mississauga, Ontario L5W 0A7, Canada.
Dr. Johnston: Department of Community Health and Epidemiology, Faculty of Medicine, Dalhousie University, Room 404, 5790 University Avenue, Halifax, Nova Scotia B3J 0E4, Canada.
Author Contributions: Conception and design: D. Zeraatkar, G.H. Guyatt, M.M. Bala, P. Alonso-Coello, S.E. Hanna, B.C. Johnston.
Analysis and interpretation of the data: D. Zeraatkar, M.A. Han, R.W.M. Vernooij, K. Milio, M. Rabassa, A. Prokop-Dorner, M.M. Bala, P. Alonso-Coello, S.E. Hanna, B.C. Johnston.
Drafting of the article: D. Zeraatkar, M.A. Han, S.E. Hanna, B.C. Johnston.
Critical revision for important intellectual content: D. Zeraatkar, G.H. Guyatt, R.W.M. Vernooij, M. Rabassa, Y. Lee, A. Prokop-Dorner, C. Lo, M.M. Bala, P. Alonso-Coello, S.E. Hanna, B.C. Johnston.
Final approval of the article: D. Zeraatkar, M.A. Han, G.H. Guyatt, R.W.M. Vernooij, R. El Dib, K. Cheung, K. Milio, M. Zworth, J.J. Bartoszko, C. Valli, M. Rabassa, Y. Lee, J. Zajac, A. Prokop-Dorner, C. Lo, M.M. Bala, P. Alonso-Coello, S.E. Hanna, B.C. Johnston.
Statistical expertise: D. Zeraatkar, S.E. Hanna, B.C. Johnston.
Administrative, technical, or logistic support: D. Zeraatkar, R. El Dib, Y. Lee, S.E. Hanna, B.C. Johnston.
Collection and assembly of data: D. Zeraatkar, M.A. Han, R.W.M. Vernooij, R. El Dib, K. Cheung, K. Milio, M. Zworth, J.J. Bartoszko, C. Valli, M. Rabassa, Y. Lee, J. Zajac, C. Lo, B.C. Johnston.
This article has been corrected. The original version (PDF) is appended to this article as a Supplement.
Dietary guidelines generally recommend limiting intake of red and processed meat. However, the quality of evidence implicating red and processed meat in adverse health outcomes remains unclear.
To evaluate the association between red and processed meat consumption and all-cause mortality, cardiometabolic outcomes, quality of life, and satisfaction with diet among adults.
EMBASE (Elsevier), Cochrane Central Register of Controlled Trials (Wiley), Web of Science (Clarivate Analytics), CINAHL (EBSCO), and ProQuest from inception until July 2018 and MEDLINE from inception until April 2019, without language restrictions, as well as bibliographies of relevant articles.
Cohort studies with at least 1000 participants that reported an association between unprocessed red or processed meat intake and outcomes of interest.
Teams of 2 reviewers independently extracted data and assessed risk of bias. One investigator assessed certainty of evidence, and the senior investigator confirmed the assessments.
Of 61 articles reporting on 55 cohorts with more than 4 million participants, none addressed quality of life or satisfaction with diet. Low-certainty evidence was found that a reduction in unprocessed red meat intake of 3 servings per week is associated with a very small reduction in risk for cardiovascular mortality, stroke, myocardial infarction (MI), and type 2 diabetes. Likewise, low-certainty evidence was found that a reduction in processed meat intake of 3 servings per week is associated with a very small decrease in risk for all-cause mortality, cardiovascular mortality, stroke, MI, and type 2 diabetes.
Inadequate adjustment for known confounders, residual confounding due to observational design, and recall bias associated with dietary measurement.
The magnitude of association between red and processed meat consumption and all-cause mortality and adverse cardiometabolic outcomes is very small, and the evidence is of low certainty.
None. (PROSPERO: CRD42017074074)
Table 1. Summary of Findings for Unprocessed Red Meat Intake (Reduction of 3 Servings per Week) and Risk for Cardiometabolic Outcomes
Nonlinear association between processed meat intake and type 2 diabetes.
The solid black line represents the point estimate, the shaded region represents the 95% CIs, and tick marks represent the positions of the study-specific estimates.
Table 2. Summary of Findings for Processed Red Meat Intake (Reduction of 3 Servings per Week) and Risk for Cardiometabolic Outcomes
Richard M. Fleming, PhD, MD
FHHI-OmnificImaging-Camelot El Segundo, CA, USA
October 2, 2019
Conflict of Interest:
FMTVDM patent was issued to primary author.
We Are All Now Dumber.
Atherosclerotic coronary artery disease (ASCAD) and cancer (CA) are the number 1 and 2 killers of people worldwide. Changes in dietary and lifestyle practices have undoubtedly played a major role in contributing to CA and CAD, but rather than conduct a prospective study measuring actual changes in disease [1,2], the authors  chose to do a retrospective look at studies biased towards meat consumption and concluded “The magnitude of association between red and processed meat consumption and all-cause mortality and adverse cardiometabolic outcomes is very small, and the evidence is of low certainty.” This conclusion provided immediately after noting there was “(i)nadequate adjustment for known confounders, residual confounding due to observational design, and recall bias associated with dietary measurement.”If the goal was to publish a study, which would receive media attention - the authors and journal have done just that. I wish I could reassure the general pubic and my medical colleagues that reading through the remainder of the paper produced a level of confidence in the conclusion – it does not!Both CAD and CA are associated with a series of inflammatory processes , which ultimately result in increased morbidity and mortality, missed by this retrospective review of what others have charted. This paper does a disservice to the journal, as well as medicine, the media and the general public. At a time when it is obvious that refined carbohydrates – including but not limited to sugar – as well as processed foods including meats and other foods, are a major contributor to CAD, CA, type 2 diabetes mellitus, high blood pressure and a host of other chronic inflammatory diseases, we should be conducting prospective research  into the causes and treatments of these diseases and not looking for retrospective support of bias. I am sadly reminded of these words from Billy Madison:“Mr. Madison, what you just said is one of the most insanely idiotic things I have ever heard. At no point in your rambling incoherent response were you even close to anything that could be considered a rational thought. Everyone in this room is now dumber for having listened to it. I award you no points and may God have mercy on your soul.” Based upon the media coverage of this paper, we are all now dumber for having read and listened to what the authors published in the Annals of Internal Medicine.Acknowledgment: FMTVDM  is issued to author.References:1. The Fleming Method for Tissue and Vascular Differentiation and Metabolism (FMTVDM) using same state single or sequential quantification comparisons. Patent Number 9566037. Issued 02/14/2017. 2. Fleming RM, Fleming MR, Chaudhuri TK. Are we prescribing the right diets and drugs for CAD, T2D, Cancer and Obesity? Int J Nuclear Med Radioactive Subs 2019;2(1):000115. 3. Zeraatkar D, Han A, Guyatt GH, et al. Red and Processed Meat Consumption and Risk for All-Cause Mortality and Cardiometabolic Outcomes. A Systematic Review and Meta-analysis of Cohort Studies. Ann Intern Med 2019. DOI:10.7326/M19-0655.4. Fleming RM. Chapter 64. The Pathogenesis of Vascular Disease. Textbook of Angiology. John C. Chang Editor, Springer-Verlag New York, NY. 1999, pp. 787-798.
Edward Giovannucci, Eric Rimm
Harvard T.H. Chan School of Public Health
October 24, 2019
Evaluation Criteria Doesn't Make the GRADE
A recent series of systematic reviews published in Annals of Internal Medicine on red meat intake concluded there was low- to very-low-certainty of evidence that higher red meat intake was associated with various health outcomes . The methods used the GRADE criteria, which were mainly developed for evaluating evidence from drug trials. GRADE has four levels of certainty in evidence: very low, low, moderate, and high. Evidence from randomized controlled trials (RCTs) starts at high quality, and evidence that includes observational data starts at low quality . There are criteria that allow one move down the evidence level, such as inconsistency or heterogeneity. The GRADE criteria state “In rare circumstances, certainty in the evidence can be rated up.” The strictness of these criteria will likely cause evidence for just about every dietary, lifestyle, and environmental topic for chronic disease to be graded as “low” or “very low”. If the GRADE criteria were used to evaluate the evidence for other dietary (e.g., low consumption of fruits and vegetables, high consumption of sugary beverages), lifestyle (e.g., alcohol, physical inactivity, safe sex, inadequate sleep), and environmental (e.g., passive smoking, air pollution) factors, none of the current recommendations on these would be supported by high or even moderate quality evidence. Even studies of current smoking and lung cancer have high heterogeneity (I2=97%), probably because in different populations, current smokers varied greatly in dose and duration, and other differences (age, ethnicity, follow-up periods, modifiers, etc). Yet, with strict adherence to GRADE guidelines, it would be questionable to move even smoking and lung cancer risk up from “low” evidence due to high heterogeneity. For dietary, lifestyle, and environmental factors, which are typically not amenable to large long-term RCTs, modified grading systems have been developed (e.g., Hierarchies of Evidence Applied to Lifestyle Medicine [HEALM] . The USDA has developed predefined criteria to evaluate and grade the strength of evidence for the Dietary Guidelines for Americans . Dr. Guyatt, one of the senior co-authors of the Annals’ papers, when asked if doctors can advise people on even something such as whether a salad is healthier than a bowl full of sugar, responded that they should tell them that “the quality of evidence is low, so it depends almost entirely on their preferences .” When GRADE criteria don’t allow us strongly to recommend against having a smoke with your bowl of sugar, we believe alternative grading systems are preferable.References1. Zeraatkar, D., et al., Red and processed meat consumption and risk for all-cause mortality and cardiometabolic outcomes: a systematic review and meta-analysis of cohort studies. Ann Intern Med, 2019.2. Siemieniuk, R. and G. Guyatt. What is GRADE? BMJ Best Practice [cited 2019; Available from: https://bestpractice.bmj.com/info/us/toolkit/learn-ebm/what-is-grade/.3. Katz, D.L., et al., Hierarchies of evidence applied to lifestyle medicine (HEALM): introduction of a strength-of-evidence approach based on a methodological systematic review. BMC Med Res Methodol, 2019. 19(1): p. 178.4. Dietary Guidelines Advisory Committee, Scientific Report of the 2015 Dietary Guidelines Advisory Committee, US Department of Agriculture and US Department of Health & Human Services, Editors. 2015.5. Hamblin, J., The actual reason meat is not healthy: nutrition studies leave out a crucial factor, in The Atlantic. 2019: Health.
Dena Zeraatkar, Gordon H. Guyatt, Pablo Alonso-Coello, Malgorzata M. Bala, Montserrat Rabassa, Mi Ah Han, Robin W.M. Vernooij, Claudia Valli, Bradley C. Johnston
November 7, 2019
The application of GRADE to nutritional epidemiology is appropriate
We would like to begin by correcting an error in Dr. Giovannuci and Rimm’s letter. They state: “it would be questionable to move even smoking and lung cancer risk up from ‘low’ evidence”. GRADE provides for high quality evidence from observational studies through two major considerations: convincing evidence of large effects and dose-response gradients. Unlike the association between red and processed meat and adverse health outcomes for which there is no convincing evidence of large relative risks (<0.5 or >2.0) (1), the magnitude of association between smoking and lung cancer is much larger, with relative risks ranging between 5 and 25 (2, 3)*. Moreover, there is also a credible dose-response gradient. Because smoking and lung cancer fulfills both criteria, the application of GRADE would infer high quality evidence.
We will now proceed to the major epistemological concern. The letter authors make the case that the epistemological rules by which we distinguish between what is true from what is speculative or untrue should differ depending on the type of evidence that is possible to bring to bear on an issue.
Consider the following situation. Two bodies of evidence, each a series of equally well-done observational studies. One body of evidence addresses a drug for which randomized trials are possible and another a nutritional intervention for which randomized trials are not feasible. According to the authors’ logic, the first would be considered low quality evidence and the second identical body of evidence would be considered high quality evidence.
This strikes us as fundamentally illogical. Rules for determining the quality of evidence should not depend on the feasibility of particular research designs and should be consistent across health fields.
If one accepts that standards of how we know what we know should be uniform across health fields, the next issue is what standards of evidence we should adopt.
The GRADE system, based on rigorous methodology described in a series of over 25 peer-reviewed articles, has been adopted by more than 110 international organizations (4, 5). These include Cochrane, and the World Health Organization, each of which regularly apply GRADE to systematic reviews of observational studies. In addressing the challenges of applying GRADE to observational studies, researchers from the environmental field have worked with GRADE on the application of the methods to environmental topics (6).
Until someone develops an alternative standard that surpasses the rigor of the GRADE system, and is more widely endorsed by the scientific community, GRADE should remain the current best approach to achieving consistency in epistemological inquiry across health fields.
*This sentence was corrected on March 16, 2020. The original sentence read: "Unlike the association between red and processed meat and adverse health outcomes for which relative risks range from 1.08 to 1.28 (1), the magnitude of association between smoking and lung cancer is much larger, with relative risks ranging between 5 and 25 (2,3)."
1) Johnston BC, Zeraatkar D, Han MA, Vernooij RW, Valli C, El Dib R, Marshall C, Stover PJ, Fairweather-Taitt S, Wójcik G, Bhatia F. Unprocessed red meat and processed meat consumption: dietary guideline recommendations from the nutritional recommendations (NutriRECS) consortium. Annals of Internal Medicine. 2019 Oct 1.
2) Ordóñez-Mena JM, Schöttker B, Mons U, Jenab M, Freisling H, Bueno-de-Mesquita B, O’Doherty MG, Scott A, Kee F, Stricker BH, Hofman A. Quantification of the smoking-associated cancer risk with rate advancement periods: meta-analysis of individual participant data from cohorts of the CHANCES consortium. BMC medicine. 2016 Dec;14(1):62.
3) Pesch B, Kendzia B, Gustavsson P, Jöckel KH, Johnen G, Pohlabeln H, Olsson A, Ahrens W, Gross IM, Brüske I, Wichmann HE. Cigarette smoking and lung cancer—relative risk estimates for the major histological types from a pooled analysis of case–control studies. International journal of cancer. 2012 Sep 1;131(5):1210-9.
4) Guyatt GH, Oxman AD, Vist GE, Kunz R, Falck-Ytter Y, Alonso-Coello P, Schünemann HJ. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ. 2008 Apr 24;336(7650):924-6.
5) Guyatt GH, Oxman AD, Schünemann HJ, Tugwell P, Knottnerus A. GRADE guidelines: a new series of articles in the Journal of Clinical Epidemiology. Journal of clinical epidemiology. 2011 Apr 1;64(4):380-2.
6) Morgan RL, Beverly B, Ghersi D, Schünemann HJ, Rooney AA, Whaley P, Zhu YG, Thayer KA. GRADE guidelines for environmental and occupational health: A new series of articles in Environment International. Environment international. 2019 Jul;128:11.
Zeraatkar D, Han MA, Guyatt GH, et al. Red and Processed Meat Consumption and Risk for All-Cause Mortality and Cardiometabolic Outcomes: A Systematic Review and Meta-analysis of Cohort Studies. Ann Intern Med. 2019;171:703–710. [Epub ahead of print 1 October 2019]. doi: https://doi.org/10.7326/M19-0655
Download citation file:
Published: Ann Intern Med. 2019;171(10):703-710.
Published at www.annals.org on 1 October 2019
Cardiology, Coronary Risk Factors, Diabetes, Endocrine and Metabolism, Neurology.
Results provided by:
Copyright © 2020 American College of Physicians. All Rights Reserved.
Print ISSN: 0003-4819 | Online ISSN: 1539-3704
Conditions of Use