Dena Zeraatkar, MSc; Mi Ah Han, MD, PhD; Gordon H. Guyatt, MD, MSc; Robin W.M. Vernooij, PhD; Regina El Dib, PhD; Kevin Cheung, MD, MSc; Kirolos Milio, BSc; Max Zworth, BASc; Jessica J. Bartoszko, HBSc; Claudia Valli, MSc; Montserrat Rabassa, PhD; Yung Lee, BHSc; Joanna Zajac, PhD; Anna Prokop-Dorner, PhD; Calvin Lo, BHSc; Malgorzata M. Bala, PhD; Pablo Alonso-Coello, MD, PhD; Steven E. Hanna, PhD; Bradley C. Johnston, PhD
Acknowledgment: The authors thank Thomasin Adams-Webber (Hospital for Sick Children) for her help in designing the search strategy.
Disclosures: Dr. El Dib received a São Paulo Research Foundation (FAPESP) (2018/11205-6) scholarship and funding from the National Council for Scientific and Technological Development (CNPq) (CNPq 310953/2015-4) and the Faculty of Medicine, Dalhousie University. Authors not named here have disclosed no conflicts of interest. Disclosures can also be viewed at www.acponline.org/authors/icmje/ConflictOfInterestForms.do?msNum=M19-0655.
Editors' Disclosures: Christine Laine, MD, MPH, Editor in Chief, reports that her spouse has stock options/holdings with Targeted Diagnostics and Therapeutics. Darren B. Taichman, MD, PhD, Executive Editor, reports that he has no financial relationships or interests to disclose. Cynthia D. Mulrow, MD, MSc, Senior Deputy Editor, reports that she has no relationships or interests to disclose. Jaya K. Rao, MD, MHS, Deputy Editor, reports that she has stock holdings/options in Eli Lilly and Pfizer. Catharine B. Stack, PhD, MS, Deputy Editor, Statistics, reports that she has stock holdings in Pfizer, Johnson & Johnson, and Colgate-Palmolive. Christina C. Wee, MD, MPH, Deputy Editor, reports employment with Beth Israel Deaconess Medical Center. Sankey V. Williams, MD, Deputy Editor, reports that he has no financial relationships or interests to disclose. Yu-Xiao Yang, MD, MSCE, Deputy Editor, reports that he has no financial relationships or interest to disclose.
>Reproducible Research Statement: Study protocol: Registered with PROSPERO (CRD42017074074). Statistical code and data set: Available from Ms. Zeraatkar (e-mail, email@example.com). For sample code, see Supplement 2.
Corresponding Author: Bradley C. Johnston, PhD, Department of Community Health and Epidemiology, Faculty of Medicine, Dalhousie University, Room 404, 5790 University Avenue, Halifax, Nova Scotia B3J 0E4, Canada; e-mail, firstname.lastname@example.org.
Current Author Addresses: Ms. Zeraatkar, Drs. Guyatt and Hanna, and Ms. Bartoszko: Department of Health Research Methods, Evidence, and Impact, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S 4K1, Canada.
Dr. Han: Department of Preventive Medicine, College of Medicine, Chosun University, 309 Philmun-daero, Dong-gu, Gwangju 61452, Korea.
Dr. Vernooij: Department of Research, Netherlands Comprehensive Cancer Organisation, Godebaldkwartier 419, Utrecht 3511 DT, the Netherlands.
Dr. El Dib: Institute of Science and Technology, São José dos Campos, Avenida Engenheiro Francisco José Longo, 777, Jardim São Dimas, São José dos Campos, 12245-000, Spain.
Dr. Cheung: 114 Loganberry Crescent, Toronto, Ontario M2H 3H1, Canada.
Mr. Milio: 592 Regal Place, Waterloo, Ontario N2V 2G3, Canada.
Mr. Zworth: 28 York Downs Drive, Toronto, Ontario M3H 1J1, Canada.
Ms. Valli and Drs. Rabassa and Alonso-Coello: Iberoamerican Cochrane Centre, (IIB Sant Pau-CIBERESP), Carrer de Sant Antoni Maria Claret, 167, Barcelona, 08025, Spain.
Mr. Lee: 30 White Lodge Crescent, Richmond Hill, Ontario L4C 9A1, Canada.
Drs. Zajac, Prokop-Dorner, and Bala: Jagiellonian University Medical College, Department of Hygiene and Dietetics, Kopernika 7 Street, 31-034 Krakow, Poland.
Mr. Lo: 556 Amarone Court, Mississauga, Ontario L5W 0A7, Canada.
Dr. Johnston: Department of Community Health and Epidemiology, Faculty of Medicine, Dalhousie University, Room 404, 5790 University Avenue, Halifax, Nova Scotia B3J 0E4, Canada.
Author Contributions: Conception and design: D. Zeraatkar, G.H. Guyatt, M.M. Bala, P. Alonso-Coello, S.E. Hanna, B.C. Johnston.
Analysis and interpretation of the data: D. Zeraatkar, M.A. Han, R.W.M. Vernooij, K. Milio, M. Rabassa, A. Prokop-Dorner, M.M. Bala, P. Alonso-Coello, S.E. Hanna, B.C. Johnston.
Drafting of the article: D. Zeraatkar, M.A. Han, S.E. Hanna, B.C. Johnston.
Critical revision for important intellectual content: D. Zeraatkar, G.H. Guyatt, R.W.M. Vernooij, M. Rabassa, Y. Lee, A. Prokop-Dorner, C. Lo, M.M. Bala, P. Alonso-Coello, S.E. Hanna, B.C. Johnston.
Final approval of the article: D. Zeraatkar, M.A. Han, G.H. Guyatt, R.W.M. Vernooij, R. El Dib, K. Cheung, K. Milio, M. Zworth, J.J. Bartoszko, C. Valli, M. Rabassa, Y. Lee, J. Zajac, A. Prokop-Dorner, C. Lo, M.M. Bala, P. Alonso-Coello, S.E. Hanna, B.C. Johnston.
Statistical expertise: D. Zeraatkar, S.E. Hanna, B.C. Johnston.
Administrative, technical, or logistic support: D. Zeraatkar, R. El Dib, Y. Lee, S.E. Hanna, B.C. Johnston.
Collection and assembly of data: D. Zeraatkar, M.A. Han, R.W.M. Vernooij, R. El Dib, K. Cheung, K. Milio, M. Zworth, J.J. Bartoszko, C. Valli, M. Rabassa, Y. Lee, J. Zajac, C. Lo, B.C. Johnston.
Dietary guidelines generally recommend limiting intake of red and processed meat. However, the quality of evidence implicating red and processed meat in adverse health outcomes remains unclear.
To evaluate the association between red and processed meat consumption and all-cause mortality, cardiometabolic outcomes, quality of life, and satisfaction with diet among adults.
EMBASE (Elsevier), Cochrane Central Register of Controlled Trials (Wiley), Web of Science (Clarivate Analytics), CINAHL (EBSCO), and ProQuest from inception until July 2018 and MEDLINE from inception until April 2019, without language restrictions, as well as bibliographies of relevant articles.
Cohort studies with at least 1000 participants that reported an association between unprocessed red or processed meat intake and outcomes of interest.
Teams of 2 reviewers independently extracted data and assessed risk of bias. One investigator assessed certainty of evidence, and the senior investigator confirmed the assessments.
Of 61 articles reporting on 55 cohorts with more than 4 million participants, none addressed quality of life or satisfaction with diet. Low-certainty evidence was found that a reduction in unprocessed red meat intake of 3 servings per week is associated with a very small reduction in risk for cardiovascular mortality, stroke, myocardial infarction (MI), and type 2 diabetes. Likewise, low-certainty evidence was found that a reduction in processed meat intake of 3 servings per week is associated with a very small decrease in risk for all-cause mortality, cardiovascular mortality, stroke, MI, and type 2 diabetes.
Inadequate adjustment for known confounders, residual confounding due to observational design, and recall bias associated with dietary measurement.
The magnitude of association between red and processed meat consumption and all-cause mortality and adverse cardiometabolic outcomes is very small, and the evidence is of low certainty.
None. (PROSPERO: CRD42017074074)
Table 1. Summary of Findings for Unprocessed Red Meat Intake (Reduction of 3 Servings per Week) and Risk for Cardiometabolic Outcomes
Nonlinear association between processed meat intake and type 2 diabetes.
The solid black line represents the point estimate, the shaded region represents the 95% CIs, and tick marks represent the positions of the study-specific estimates.
Table 2. Summary of Findings for Processed Red Meat Intake (Reduction of 3 Servings per Week) and Risk for Cardiometabolic Outcomes
The In the Clinic® slide sets are owned and copyrighted by the American College of Physicians (ACP). All text, graphics, trademarks, and other intellectual property incorporated into the slide sets remain the sole and exclusive property of the ACP. The slide sets may be used only by the person who downloads or purchases them and only for the purpose of presenting them during not-for-profit educational activities. Users may incorporate the entire slide set or selected individual slides into their own teaching presentations but may not alter the content of the slides in any way or remove the ACP copyright notice. Users may make print copies for use as hand-outs for the audience the user is personally addressing but may not otherwise reproduce or distribute the slides by any means or media, including but not limited to sending them as e-mail attachments, posting them on Internet or Intranet sites, publishing them in meeting proceedings, or making them available for sale or distribution in any unauthorized form, without the express written permission of the ACP. Unauthorized use of the In the Clinic slide sets will constitute copyright infringement.
Richard M. Fleming, PhD, MD
FHHI-OmnificImaging-Camelot El Segundo, CA, USA
October 2, 2019
Conflict of Interest:
FMTVDM patent was issued to primary author.
We Are All Now Dumber.
Atherosclerotic coronary artery disease (ASCAD) and cancer (CA) are the number 1 and 2 killers of people worldwide. Changes in dietary and lifestyle practices have undoubtedly played a major role in contributing to CA and CAD, but rather than conduct a prospective study measuring actual changes in disease [1,2], the authors  chose to do a retrospective look at studies biased towards meat consumption and concluded “The magnitude of association between red and processed meat consumption and all-cause mortality and adverse cardiometabolic outcomes is very small, and the evidence is of low certainty.” This conclusion provided immediately after noting there was “(i)nadequate adjustment for known confounders, residual confounding due to observational design, and recall bias associated with dietary measurement.”If the goal was to publish a study, which would receive media attention - the authors and journal have done just that. I wish I could reassure the general pubic and my medical colleagues that reading through the remainder of the paper produced a level of confidence in the conclusion – it does not!Both CAD and CA are associated with a series of inflammatory processes , which ultimately result in increased morbidity and mortality, missed by this retrospective review of what others have charted. This paper does a disservice to the journal, as well as medicine, the media and the general public. At a time when it is obvious that refined carbohydrates – including but not limited to sugar – as well as processed foods including meats and other foods, are a major contributor to CAD, CA, type 2 diabetes mellitus, high blood pressure and a host of other chronic inflammatory diseases, we should be conducting prospective research  into the causes and treatments of these diseases and not looking for retrospective support of bias. I am sadly reminded of these words from Billy Madison:“Mr. Madison, what you just said is one of the most insanely idiotic things I have ever heard. At no point in your rambling incoherent response were you even close to anything that could be considered a rational thought. Everyone in this room is now dumber for having listened to it. I award you no points and may God have mercy on your soul.” Based upon the media coverage of this paper, we are all now dumber for having read and listened to what the authors published in the Annals of Internal Medicine.Acknowledgment: FMTVDM  is issued to author.References:1. The Fleming Method for Tissue and Vascular Differentiation and Metabolism (FMTVDM) using same state single or sequential quantification comparisons. Patent Number 9566037. Issued 02/14/2017. 2. Fleming RM, Fleming MR, Chaudhuri TK. Are we prescribing the right diets and drugs for CAD, T2D, Cancer and Obesity? Int J Nuclear Med Radioactive Subs 2019;2(1):000115. 3. Zeraatkar D, Han A, Guyatt GH, et al. Red and Processed Meat Consumption and Risk for All-Cause Mortality and Cardiometabolic Outcomes. A Systematic Review and Meta-analysis of Cohort Studies. Ann Intern Med 2019. DOI:10.7326/M19-0655.4. Fleming RM. Chapter 64. The Pathogenesis of Vascular Disease. Textbook of Angiology. John C. Chang Editor, Springer-Verlag New York, NY. 1999, pp. 787-798.
Edward Giovannucci, Eric Rimm
Harvard T.H. Chan School of Public Health
October 24, 2019
Evaluation Criteria Doesn't Make the GRADE
A recent series of systematic reviews published in Annals of Internal Medicine on red meat intake concluded there was low- to very-low-certainty of evidence that higher red meat intake was associated with various health outcomes . The methods used the GRADE criteria, which were mainly developed for evaluating evidence from drug trials. GRADE has four levels of certainty in evidence: very low, low, moderate, and high. Evidence from randomized controlled trials (RCTs) starts at high quality, and evidence that includes observational data starts at low quality . There are criteria that allow one move down the evidence level, such as inconsistency or heterogeneity. The GRADE criteria state “In rare circumstances, certainty in the evidence can be rated up.” The strictness of these criteria will likely cause evidence for just about every dietary, lifestyle, and environmental topic for chronic disease to be graded as “low” or “very low”. If the GRADE criteria were used to evaluate the evidence for other dietary (e.g., low consumption of fruits and vegetables, high consumption of sugary beverages), lifestyle (e.g., alcohol, physical inactivity, safe sex, inadequate sleep), and environmental (e.g., passive smoking, air pollution) factors, none of the current recommendations on these would be supported by high or even moderate quality evidence. Even studies of current smoking and lung cancer have high heterogeneity (I2=97%), probably because in different populations, current smokers varied greatly in dose and duration, and other differences (age, ethnicity, follow-up periods, modifiers, etc). Yet, with strict adherence to GRADE guidelines, it would be questionable to move even smoking and lung cancer risk up from “low” evidence due to high heterogeneity. For dietary, lifestyle, and environmental factors, which are typically not amenable to large long-term RCTs, modified grading systems have been developed (e.g., Hierarchies of Evidence Applied to Lifestyle Medicine [HEALM] . The USDA has developed predefined criteria to evaluate and grade the strength of evidence for the Dietary Guidelines for Americans . Dr. Guyatt, one of the senior co-authors of the Annals’ papers, when asked if doctors can advise people on even something such as whether a salad is healthier than a bowl full of sugar, responded that they should tell them that “the quality of evidence is low, so it depends almost entirely on their preferences .” When GRADE criteria don’t allow us strongly to recommend against having a smoke with your bowl of sugar, we believe alternative grading systems are preferable.References1. Zeraatkar, D., et al., Red and processed meat consumption and risk for all-cause mortality and cardiometabolic outcomes: a systematic review and meta-analysis of cohort studies. Ann Intern Med, 2019.2. Siemieniuk, R. and G. Guyatt. What is GRADE? BMJ Best Practice [cited 2019; Available from: https://bestpractice.bmj.com/info/us/toolkit/learn-ebm/what-is-grade/.3. Katz, D.L., et al., Hierarchies of evidence applied to lifestyle medicine (HEALM): introduction of a strength-of-evidence approach based on a methodological systematic review. BMC Med Res Methodol, 2019. 19(1): p. 178.4. Dietary Guidelines Advisory Committee, Scientific Report of the 2015 Dietary Guidelines Advisory Committee, US Department of Agriculture and US Department of Health & Human Services, Editors. 2015.5. Hamblin, J., The actual reason meat is not healthy: nutrition studies leave out a crucial factor, in The Atlantic. 2019: Health.
Dena Zeraatkar, Gordon H. Guyatt, Pablo Alonso-Coello, Malgorzata M. Bala, Montserrat Rabassa, Mi Ah Han, Robin W.M. Vernooij, Claudia Valli, Bradley C. Johnston
November 7, 2019
The application of GRADE to nutritional epidemiology is appropriate
We would like to begin by correcting an error in Dr. Giovannuci and Rimm’s letter. They state: “it would be questionable to move even smoking and lung cancer risk up from “low” evidence due to high heterogeneity”. GRADE provides for high quality evidence from observational studies through two major considerations: large or very large effects and dose-response gradients. Unlike the association between red and processed meat and adverse health outcomes for which relative risks range from 1.06 to 1.28 (Johnston et al., 2019), the magnitude of association between smoking and lung cancer is much larger, with reported relative risks ranging between 5 and 25 (Ordóñez-Mena et al., 2016; Pesch et al., 2012). Moreover, there is also a dose-response association between smoking and adverse health outcomes. Because it fulfills both criteria, the application of the GRADE approach would infer high quality evidence that smoking results in an important increase in lung cancer.We will now proceed to the major issue, which is one of epistemology: how we distinguish what is true from what is speculative or untrue, or how do we distinguish between justified belief from opinion. The letter authors make the case that the epistemological rules by which we make these distinctions should differ depending on what evidence is possible to bring to bear on an issue.Consider the following situation. Two bodies of evidence, each a series of well-done observational studies, are identical in their methods. One body of evidence addresses a drug in which randomized trials are possible, and another a nutritional intervention in which randomized trials are not feasible. According to the authors’ logic, the first would be considered low quality evidence resulting in only weak inferences regarding causation, and the second identical body of evidence would be considered high quality evidence in which we are confident of causal inferences (Katz et al, 2019).This strikes us as fundamentally illogical. Our rules for distinguishing quality of evidence regarding health claims, from inferences in which we can be confident to those that remain uncertain, should not depend on the feasibility of particular research designs. Epistemological standards should be consistent across health fields. If one rejects the position that we should apply epistemological standards across health claims, one can make up whatever standards one likes to be able to claim high quality evidence in one’s particular field. If one accepts the position that standards of how we know what we know should be uniform across fields of health inquiry, the next issue is what standards of evidence we should adopt.The GRADE system, based on comprehensive methodology described in detail in a series of 8 BMJ publications and, thus far, 22 articles in the Journal of Clinical Epidemiology, has been adopted by more than 110 international organizations (Guyatt et al., 2008; Guyatt et al., 2011). These include the Cochrane Collaboration, the World Health Organization, the Joanna Briggs Institute, and the National Institute for Health and Care Excellence, each of which regularly apply GRADE to systematic reviews of observational studies (http://www.gradeworkinggroup.org/). In addressing the challenges of applying GRADE to observational studies, researchers from the environmental field have worked closely with the GRADE working group on the application of the GRADE system to improve the rigor and transparency of systematic reviews and guidelines addressing environmental topics (Morgan et al., 2019).The logic of scientific inquiry demands consistent standards of causal inference across health claims. Until someone develops an alternative standard that surpasses the rigor and transparency of the GRADE system, and outdoes GRADE in terms of endorsement by the scientific community, GRADE should remain the current best approach to achieving, in health fields, consistency in epistemological inquiry. ReferencesGuyatt GH, Oxman AD, Vist GE, Kunz R, Falck-Ytter Y, Alonso-Coello P, Schünemann HJ. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. Bmj. 2008 Apr 24;336(7650):924-6.Guyatt GH, Oxman AD, Schünemann HJ, Tugwell P, Knottnerus A. GRADE guidelines: a new series of articles in the Journal of Clinical Epidemiology. Journal of clinical epidemiology. 2011 Apr 1;64(4):380-2.Johnston BC, Zeraatkar D, Han MA, Vernooij RW, Valli C, El Dib R, Marshall C, Stover PJ, Fairweather-Taitt S, Wójcik G, Bhatia F. Unprocessed red meat and processed meat consumption: dietary guideline recommendations from the nutritional recommendations (NutriRECS) consortium. Annals of Internal Medicine. 2019 Oct 1.Katz DL, Karlsen MC, Chung M, Shams-White MM, Green LW, Fielding J, Saito A, Willett W. Hierarchies of evidence applied to lifestyle Medicine (HEALM): introduction of a strength-of-evidence approach based on a methodological systematic review. BMC medical research methodology. 2019 Dec 1;19(1):178.Morgan RL, Beverly B, Ghersi D, Schünemann HJ, Rooney AA, Whaley P, Zhu YG, Thayer KA. GRADE guidelines for environmental and occupational health: A new series of articles in Environment International. Environment international. 2019 Jul;128:11.Ordóñez-Mena JM, Schöttker B, Mons U, Jenab M, Freisling H, Bueno-de-Mesquita B, O’Doherty MG, Scott A, Kee F, Stricker BH, Hofman A. Quantification of the smoking-associated cancer risk with rate advancement periods: meta-analysis of individual participant data from cohorts of the CHANCES consortium. BMC medicine. 2016 Dec;14(1):62.Pesch B, Kendzia B, Gustavsson P, Jöckel KH, Johnen G, Pohlabeln H, Olsson A, Ahrens W, Gross IM, Brüske I, Wichmann HE. Cigarette smoking and lung cancer—relative risk estimates for the major histological types from a pooled analysis of case–control studies. International journal of cancer. 2012 Sep 1;131(5):1210-9.
Zeraatkar D, Han MA, Guyatt GH, et al. Red and Processed Meat Consumption and Risk for All-Cause Mortality and Cardiometabolic Outcomes: A Systematic Review and Meta-analysis of Cohort Studies. Ann Intern Med. 2019;171:703–710. [Epub ahead of print 1 October 2019]. doi: https://doi.org/10.7326/M19-0655
Download citation file:
Published: Ann Intern Med. 2019;171(10):703-710.
Published at www.annals.org on 1 October 2019
Cardiology, Coronary Risk Factors, Diabetes, Endocrine and Metabolism, Neurology.
Results provided by:
Copyright © 2019 American College of Physicians. All Rights Reserved.
Print ISSN: 0003-4819 | Online ISSN: 1539-3704
Conditions of Use