## Abstract

The 2013 pooled cohort equations (PCEs) are central in prevention guidelines for cardiovascular disease (CVD) but can misestimate CVD risk.

To improve the clinical accuracy of CVD risk prediction by revising the 2013 PCEs using newer data and statistical methods.

Derivation and validation of risk equations.

Population-based.

26 689 adults aged 40 to 79 years without prior CVD from 6 U.S. cohorts.

Nonfatal myocardial infarction, death from coronary heart disease, or fatal or nonfatal stroke.

The 2013 PCEs overestimated 10-year risk for atherosclerotic CVD by an average of 20% across risk groups. Misestimation of risk was particularly prominent among black adults, of whom 3.9 million (33% of eligible black persons) had extreme risk estimates (<70% or >250% those of white adults with otherwise-identical risk factor values). Updating these equations improved accuracy among all race and sex subgroups. Approximately 11.8 million U.S. adults previously labeled high-risk (10-year risk ≥7.5%) by the 2013 PCEs would be relabeled lower-risk by the updated equations.

Updating the 2013 PCEs with data from modern cohorts reduced the number of persons considered to be at high risk. Clinicians and patients should consider the potential benefits and harms of reducing the number of persons recommended aspirin, blood pressure, or statin therapy. Our findings also indicate that risk equations will generally become outdated over time and require routine updating.

Revised PCEs can improve the accuracy of CVD risk estimates.

National Institutes of Health.

## Methods

### Data

### Participants

*N*

*=*26 689) (Supplement Tables 1 and 2). The sample included only participants with complete data on predictor variables, matching 2013 derivation procedures (11); 4.9% of cohort participants were excluded because they were missing at least 1 predictor.

### Outcome

### Predictors

### Analysis

#### Model Set 1

*p*value for the interaction term was less than .01, or the

*p*value was .01 to .05 and the continuous net reclassification improvement for nonevents was 15 percent or greater, or the integrated discrimination improvement index … was statistically significant” (11). We replicated the derivation approach used by the 2013 PCE committee to test the hypothesis that updating the data without updating the derivation method would improve estimation by the PCEs. The resulting equations were labeled model set 1

*.*

#### Model Set 2

*P*values and model fit can cause overfitting, particularly for subpopulations with fewer participants, such as black adults (Figure 1) (13). Second, the 2013 derivation method used a Cox proportional hazards model, a traditional survival model that requires the proportional hazards assumption. We tested this assumption (because it was not previously tested to our knowledge) and found that the Cox proportional hazards assumption was violated by the cohort data used to derive the original 2013 PCEs (Supplement Table 3).

*P*values to select predictors and coefficients (19, 20). We also avoided overfitting by deriving only 2 equations (1 for men and 1 for women, with potential black race coefficients and interaction terms with race) rather than 4 (1 for each combination of men or women and black or white race), because deriving 4 equations assumes that black race necessarily interacts with each other predictor in the model (thus predisposing to overfitting). The revised equations in model set 2 also followed recommendations that when the proportional hazards assumption is violated, a logistic regression model adjusted for censoring can produce more accurate coefficient estimates than a Cox proportional hazards model (13, 15, 21). Hence, 2 logistic equations were selected through elastic net regularization, 1 for men and 1 for women.

### Validation

### Performance Measures

*Calibration*(how well estimated CVD event rates correspond to observed rates) was assessed by the Greenwood–Nam–D'Agostino (GND) test (27). This test assesses the significance of χ

^{2}differences between expected and observed event rates (ideally having a large

*P*value), the calibration slope of the regression line between expected and observed Kaplan–Meier CVD event rates (ideally 1), and the observed versus expected CVD rate by expected risk group (10-year expected risk of <5%, 5% to <7.5%, 7.5% to <10%, or ≥10%).

*Discrimination*(how likely the equations are to correctly pick the higher-risk person in a pair) was measured by the c-statistic (ideally 1) (28). We used a reclassification table to assess discrimination improvement, tabulating persons who did and did not have a CVD event and were classified as high or low risk (≥7.5% or <7.5% expected 10-year risk, respectively [1]) by 1 model and correctly or incorrectly reclassified as high or low risk by an alternative model. We did sensitivity analyses using 10-year risk of 5% or 10% rather than 7.5% as the threshold for “high risk.” We avoided the net reclassification index because of multiple reports that it is biased toward overfitted models even when independent test data sets are used (29, 30); hence, fully disaggregated reclassification tables were presented.

*black–white expected risk ratio*was estimated, which is a black adult's estimated risk divided by that of a white adult with otherwise-identical risk factor values. We aimed to detect implausible risk estimates for black adults that may be due to overfitting by comparing the black–white expected risk ratio with the empirical ratio range of 70% to 250% (12). We estimated the ratio among adults aged 40 to 79 years in NHANES who met the inclusion criteria above after excluding participants receiving statins and those with extreme biomarker values (as defined by the 2013 PCE derivation committee [11]: systolic blood pressure <90 or >200 mm Hg, total cholesterol level <3.37 or >8.29 mmol/L [<130 or >320 mg/dL], or high-density lipoprotein cholesterol level <0.52 or >2.59 mmol/L [<20 or >100 mg/dL]). The 10-year CVD risk for each person in NHANES was calculated using the equations for blacks and then reestimated using the equations for whites to calculate the expected risk ratio.

### Role of the Funding Source

## Results

### Original 2013 PCEs

### Updated Model Set 1: 2013 PCE Derivation Method With Newer Cohort Data

### Updated Model Set 2: Newer Cohort Data and an Updated Derivation Method

### Alternative Derivations and Validations

## Discussion

*P*values and model fit to estimate risk equations without a penalty for producing equations with many terms, the resulting equations can be overfitted. Overfitting may cause miscalibration and extreme risk estimates, particularly for subpopulations with smaller sample sizes. Second, although use of a Cox proportional hazards model is common for medical risk estimation, the proportional hazards assumption may be violated and should be tested to avoid miscalibration before the Cox model formula is applied. Finally, in our alternative derivation and validation experiments, we found that simply updating our statistical methods was not sufficient to correct misestimation by the 2013 PCEs; both updated data and revised statistical methods were necessary.

## References

- ,StoneNJ,RobinsonJG,LichtensteinAH,BaireyMerzCN,BlumCB.EckelRHet alAmerican College of Cardiology/American Heart Association Task Force on Practice Guidelines2013 ACC/AHA guideline on the treatment of blood cholesterol to reduce atherosclerotic cardiovascular risk in adults: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines.Circulation2014129S145
- ,WheltonPK,CareyRM,AronowWS,CaseyDEJr,CollinsKJ.Dennison HimmelfarbCet al2017 ACC/AHA/AAPA/ABC/ACPM/AGS/APhA/ASH/ASPC/NMA/PCNA guideline for the prevention, detection, evaluation, and management of high blood pressure in adults: a report of the American College of Cardiology/American Heart Association Task Force on Clinical Practice Guidelines.J Am Coll Cardiol2017
- ,DeFilippisAP,YoungR,McEvoyJW,MichosED,SandfortV.KronmalRAet alRisk score overestimation: the impact of individual cardiovascular risk factors and preventive therapies on the performance of the American Heart Association-American College of Cardiology-Atherosclerotic Cardiovascular Disease risk score in a modern multi-ethnic cohort.Eur Heart J201738598608
- ,GoffDCJr,Lloyd-JonesDM,BennettG,CoadyS,D'AgostinoRB.GibbonsRet alAmerican College of Cardiology/American Heart Association Task Force on Practice Guidelines2013 ACC/AHA guideline on the assessment of cardiovascular risk: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines.Circulation2014129S4973
- ,HastieT,TibshiraniR.FriedmanJThe Elements of Statistical Learning: Data Mining, Inference, and Prediction. 2nd ed.New YorkSpringer2011
- .BasuSModeling Public Health and Healthcare Systems. 1st ed.New YorkOxford Univ Pr2017
- ,HosmerDWJr,LemeshowS.SturdivantRXApplied Logistic Regression. Vol. 398.New YorkJ Wiley2013
- ,HillerL,MarshallA.DunnJAssessing violations of the proportional hazards assumption in Cox regression: does the chosen method matter?Trials201516P134
- .BoruckaJExtensions of Cox model for non-proportional hazards purpose.Ekonometria2014385101
- .KohaviRProceedings of the 14th International Joint Conference on Artificial Intelligence. Vol. 2.San FranciscoMorgan Kaufmann1995113743
- .CooperHMResearch Synthesis and Meta-analysis. 5th ed.Los AngelesSage2016

## This feature is available only to Registered Users

Subscribe/Learn More## 0 Comments