Steve Yadlowsky, MS; Rodney A. Hayward, MD; Jeremy B. Sussman, MD, MS; Robyn L. McClelland, PhD; Yuan-I Min, PhD; Sanjay Basu, MD, PhD
Disclaimer: The content is solely the responsibility of the authors and does not necessarily represent the official views or opinions of the National Institutes of Health; National Heart, Lung, and Blood Institute (NHLBI); CHS; CARDIA; or FHS. This manuscript has been reviewed by MESA and JHS.
Acknowledgment: The authors thank Lu Tian, John Ioannidis, John Duchi, and Doug Owens for their input and reviews of the methods and manuscript. This manuscript was prepared using research materials from ARIC, CHS, CARDIA, and FHS obtained from the NHLBI Biologic Specimen and Data Repository Information Coordinating Center.
Financial Support: By awards DP2MD010478 and U54MD010724 from the National Institute on Minority Health and Health Disparities of the National Institutes of Health and award K08HL121056 from the NHLBI of the National Institutes of Health. Mr. Yadlowsky is supported by the Stanford University Graduate Fellowship. Dr. Hayward is supported by grant P30DK092926 (Michigan Center for Diabetes Translational Research Methods Core) from the National Institute of Diabetes and Digestive and Kidney Diseases. The JHS is supported by contracts HHSN268201300046C, HHSN268201300047C, HHSN268201300048C, HHSN268201300049C, and HHSN268201300050C from the NHLBI and the National Institute on Minority Health and Health Disparities, with additional support from the National Institute of Biomedical Imaging and Bioengineering. MESA is supported by contracts HHSN268201500003I, N01-HC-95159, N01-HC-95160, N01-HC-95161, N01-HC-95162, N01-HC-95163, N01-HC-95164, N01-HC-95165, N01-HC-95166, N01-HC-95167, N01-HC-95168, and N01-HC-95169 from the NHLBI and grants UL1-TR-000040, UL1-TR-001079, and UL1-TR-001420 from the National Center for Advancing Translational Sciences.
Disclosures: Dr. Sussman reports grants from the U.S. Department of Veterans Affairs during the conduct of the study. Dr. McClelland reports grants from the NHLBI during the conduct of the study. Authors not named here have disclosed no conflicts of interest. Disclosures can also be viewed at www.acponline.org/authors/icmje/ConflictOfInterestForms.do?msNum=M17-3011.
Editors' Disclosures: Christine Laine, MD, MPH, Editor in Chief, reports that her spouse has stock options/holdings with Targeted Diagnostics and Therapeutics. Darren B. Taichman, MD, PhD, Executive Editor, reports that he has no financial relationships or interests to disclose. Cynthia D. Mulrow, MD, MSc, Senior Deputy Editor, reports that she has no relationships or interests to disclose. Deborah Cotton, MD, MPH, Deputy Editor, reports that she has no financial relationships or interest to disclose. Jaya K. Rao, MD, MHS, Deputy Editor, reports that she has stock holdings/options in Eli Lilly and Pfizer. Sankey V. Williams, MD, Deputy Editor, reports that he has no financial relationships or interests to disclose. Catharine B. Stack, PhD, MS, Deputy Editor for Statistics, reports that she has stock holdings in Pfizer and Johnson & Johnson.
Reproducible Research Statement:Study protocol: Not applicable. Statistical code: Available at https://github.com/sanjaybasu/revised-pooled-ascvd, with accompanying online calculator available at https://sanjaybasu.shinyapps.io/ascvd. Data set: Available at https://biolincc.nhlbi.nih.gov.
Corresponding Author: Sanjay Basu, MD, PhD, Encina Commons, 117 Crothers Way, Stanford University, Stanford, CA 94305-6055; e-mail, email@example.com.
Current Author Addresses: Mr. Yadlowsky: Stanford University Center for Primary Care and Outcomes Research, 616 Serra Street, Room E012, Stanford, CA 94305.
Dr. Hayward: VA Ann Arbor Health Care, 2800 Plymouth Road, Building 14, G100-36, Ann Arbor, MI 48109.
Dr. Sussman: VA Ann Arbor Health Care, 2800 Plymouth Road, Building 16, Room 335E, Ann Arbor, MI 48109.
Dr. McClelland: University of Washington, 6200 NE 74th Street, Seattle, WA 98115.
Dr. Min: University of Mississippi Medical Center, 350 West Woodrow Wilson Avenue, Suite 701, Jackson, MS 39213.
Dr. Basu: Encina Commons, 117 Crothers Way, Stanford University, Stanford, CA 94305-6055.
Author Contributions: Conception and design: S. Yadlowsky, J.B. Sussman, S. Basu.
Analysis and interpretation of the data: S. Yadlowsky, R.A. Hayward, R.L. McClelland, Y.I. Min, S. Basu.
Drafting of the article: S. Yadlowsky, S. Basu.
Critical revision of the article for important intellectual content: S. Yadlowsky, R.A. Hayward, J.B. Sussman, R.L. McClelland.
Final approval of the article: S. Yadlowsky, R.A. Hayward, J.B. Sussman, R.L. McClelland, Y.I. Min, S. Basu.
Provision of study materials or patients: Y.I. Min.
Statistical expertise: S. Yadlowsky, R.A. Hayward, J.B. Sussman, R.L. McClelland, Y.I. Min, S. Basu.
Obtaining of funding: S. Basu.
Administrative, technical, or logistic support: S. Basu.
Collection and assembly of data: Y.I. Min, S. Basu.
The 2013 pooled cohort equations (PCEs) are central in prevention guidelines for cardiovascular disease (CVD) but can misestimate CVD risk.
To improve the clinical accuracy of CVD risk prediction by revising the 2013 PCEs using newer data and statistical methods.
Derivation and validation of risk equations.
26 689 adults aged 40 to 79 years without prior CVD from 6 U.S. cohorts.
Nonfatal myocardial infarction, death from coronary heart disease, or fatal or nonfatal stroke.
The 2013 PCEs overestimated 10-year risk for atherosclerotic CVD by an average of 20% across risk groups. Misestimation of risk was particularly prominent among black adults, of whom 3.9 million (33% of eligible black persons) had extreme risk estimates (<70% or >250% those of white adults with otherwise-identical risk factor values). Updating these equations improved accuracy among all race and sex subgroups. Approximately 11.8 million U.S. adults previously labeled high-risk (10-year risk ≥7.5%) by the 2013 PCEs would be relabeled lower-risk by the updated equations.
Updating the 2013 PCEs with data from modern cohorts reduced the number of persons considered to be at high risk. Clinicians and patients should consider the potential benefits and harms of reducing the number of persons recommended aspirin, blood pressure, or statin therapy. Our findings also indicate that risk equations will generally become outdated over time and require routine updating.
Revised PCEs can improve the accuracy of CVD risk estimates.
National Institutes of Health.
Conceptual illustration of overfitting.
Suppose we have data correlating a theoretical risk factor (e.g., a biomarker), x, to 10-y CVD rate, y. The equation (fitted curve) on the left may appear to be “better fit” to the data than that on the right, but we would not expect the curve on the left to reliably predict CVD rate y given some new values of risk factor x from another cohort, because the curve has fitted random error in the data set. In addition, the equation will extremely under- or overestimate CVD rates when applied to new cohorts that may have lower or higher values of x than the derivation data set. By contrast, the simpler equation (line) on the right may have poorer fit to the derivation data but better captures a more generalizable relationship between risk factor x and actual CVD rate y. CVD = cardiovascular disease.
Table 1. Model Performance Metrics*
Model error in the internal cross-validation sample (left) and prospective holdout validation sample (right).
Error was calculated as the χ2 statistic for distance between expected (predicted) and observed event rates of atherosclerotic CVD in the derivation sample, using the Greenwood–Nam–D'Agostino method for time-to-event data (27). Values <20 (dashed line) are generally considered to indicate acceptably low error. Model set 1 was the 2013 PCEs updated with newer cohort data; model set 2 additionally used a newer derivation method to avoid overfitting and the proportional hazards assumption. See Table 1 for additional calibration statistics and Supplement Figure 1 for calibration plots. CVD = cardiovascular disease; PCE = pooled cohort equation.
Observed versus expected CVD event risk, by risk group, in the internal cross-validation sample (top) and prospective holdout validation sample (bottom).
Sample sizes were n = 8329 for <5%, n = 2510 for 5% to <7.5%, n = 1899 for 7.5% to <10%, and n = 7145 for ≥10% 10-year expected risk by the 2013 PCEs in the derivation sample. The risk groups defined by the 2013 PCEs were used consistently across model comparisons to keep the same number of participants in each group, for fair comparison. Model set 1 was the 2013 PCEs updated with newer cohort data; model set 2 additionally used a newer derivation method to avoid overfitting and the proportional hazards assumption. See also Supplement Figure 3. CVD = cardiovascular disease; PCE = pooled cohort equation.
Table 2. Examples of Highly Variable Risk Ratios for Black Versus White Adults From the 2013 PCEs Versus Our Proposed Alternative, Model Set 2*
Effect of black race on 10-y estimated risk for atherosclerotic cardiovascular disease, per alternative models.
Risk ratios were calculated for all participants in the National Health and Nutrition Examination Survey, 2013–2014, aged 40–79 y with no history of cardiovascular disease. We calculated their 10-y event risk if they were black and divided by the same risk if they were white (i.e., all other risk factors being held constant). We omitted participants who were of neither white nor black race, had missing values, were receiving statins, or had extreme values for input variables (high-density lipoprotein cholesterol level <0.52 or >2.59 mmol/L [<20 or >100 mg/dL], total cholesterol level <3.37 or >8.29 mmol/L [<130 or >320 mg/dL], or systolic blood pressure <90 or >200 mm Hg). Note the log scale of the x-axis. ACC = American College of Cardiology; AHA = American Heart Association.
Appendix Table. Example Calculation for Model Set 2, the Proposed Revision of the PCEs for Estimating ASCVD Risk*
The In the Clinic® slide sets are owned and copyrighted by the American College of Physicians (ACP). All text, graphics, trademarks, and other intellectual property incorporated into the slide sets remain the sole and exclusive property of the ACP. The slide sets may be used only by the person who downloads or purchases them and only for the purpose of presenting them during not-for-profit educational activities. Users may incorporate the entire slide set or selected individual slides into their own teaching presentations but may not alter the content of the slides in any way or remove the ACP copyright notice. Users may make print copies for use as hand-outs for the audience the user is personally addressing but may not otherwise reproduce or distribute the slides by any means or media, including but not limited to sending them as e-mail attachments, posting them on Internet or Intranet sites, publishing them in meeting proceedings, or making them available for sale or distribution in any unauthorized form, without the express written permission of the ACP. Unauthorized use of the In the Clinic slide sets will constitute copyright infringement.
Yadlowsky S, Hayward RA, Sussman JB, McClelland RL, Min Y, Basu S. Clinical Implications of Revised Pooled Cohort Equations for Estimating Atherosclerotic Cardiovascular Disease Risk. Ann Intern Med. ;169:20–29. doi: 10.7326/M17-3011
Download citation file:
Published: Ann Intern Med. 2018;169(1):20-29.
Published at www.annals.org on 5 June 2018
Copyright © 2019 American College of Physicians. All Rights Reserved.
Print ISSN: 0003-4819 | Online ISSN: 1539-3704
Conditions of Use