### Context

The value of adding high-sensitivity C-reactive protein (hsCRP) to a global risk assessment model is unknown.

### Contribution

The authors used the Women's Health Study, a nationwide cohort of 15048 initially healthy women, to develop a cardiovascular disease (CVD) risk prediction model using hsCRP and Framingham risk model predictors. While hsCRP improved overall model fit, the clinical utility of hsCRP in terms of reclassification was most substantial for those with a 5% or greater 10-year risk based on traditional risk factors.

### Cautions

The study does not address the clinical value of lowering hsCRP level.

### Implications

In this largely low-risk population, adding hsCRP to the Framingham model reclassified patients into groups that better reflected their actual CVD risk. This effect was most clinically relevant for those at intermediate risk.

## Methods

*n*= 15048 with data on all variables) and were then applied to all nondiabetic women (

*n*= 26927) for clinical risk prediction.

### Development of Risk Prediction Models

### Measures of Model Fit

^{2}(For comparison, we also computed several other measures of global model fit (provided in the Appendix), including other likelihood-based measures such as model weights for the Bayes information criterion, which provide an estimate of the posterior probability of each model given the set of candidate models considered (29, 32); the Akaike information criterion and its corresponding model weights (32); and Nagelkerke's generalized model R2(33, 34). We computed the D-statistic of Royston and Sauerbrei (35), based on the separation of survival curves by predictor variables, and again adjusted for optimism. Differences in statistics between nested models were tested with a 1-sided test using bootstrap sampling (31). We also calculated the Brier score (28), which directly compares the observed outcomes with the fitted probabilities.-34). We computed the D-statistic of Royston and Sauerbrei (For comparison, we also computed several other measures of global model fit (provided in the Appendix), including other likelihood-based measures such as model weights for the Bayes information criterion, which provide an estimate of the posterior probability of each model given the set of candidate models considered (29, 32); the Akaike information criterion and its corresponding model weights (32); and Nagelkerke's generalized model R2(33, 34). We computed the D-statistic of Royston and Sauerbrei (35), based on the separation of survival curves by predictor variables, and again adjusted for optimism. Differences in statistics between nested models were tested with a 1-sided test using bootstrap sampling (31). We also calculated the Brier score (28), which directly compares the observed outcomes with the fitted probabilities.), based on the separation of survival curves by predictor variables, and again adjusted for optimism. Differences in statistics between nested models were tested with a 1-sided test using bootstrap sampling (For comparison, we also computed several other measures of global model fit (provided in the Appendix), including other likelihood-based measures such as model weights for the Bayes information criterion, which provide an estimate of the posterior probability of each model given the set of candidate models considered (29, 32); the Akaike information criterion and its corresponding model weights (32); and Nagelkerke's generalized model R2(33, 34). We computed the D-statistic of Royston and Sauerbrei (35), based on the separation of survival curves by predictor variables, and again adjusted for optimism. Differences in statistics between nested models were tested with a 1-sided test using bootstrap sampling (31). We also calculated the Brier score (28), which directly compares the observed outcomes with the fitted probabilities.). We also calculated the Brier score (For comparison, we also computed several other measures of global model fit (provided in the Appendix), including other likelihood-based measures such as model weights for the Bayes information criterion, which provide an estimate of the posterior probability of each model given the set of candidate models considered (29, 32); the Akaike information criterion and its corresponding model weights (32); and Nagelkerke's generalized model R2(33, 34). We computed the D-statistic of Royston and Sauerbrei (35), based on the separation of survival curves by predictor variables, and again adjusted for optimism. Differences in statistics between nested models were tested with a 1-sided test using bootstrap sampling (31). We also calculated the Brier score (28), which directly compares the observed outcomes with the fitted probabilities.), which directly compares the observed outcomes with the fitted probabilities.

### Role of the Funding Sources

## Results

### Relative Contributions to Global Risk of Age, Blood Pressure, Smoking, Lipids, and hsCRP

### Discrimination and Calibration in Models with and without hsCRP

*P*< 0.001). Of note, the Bayes information criterion indicated a strong preference for the inclusion of hsCRP (Table 1) after adjustment for adding a variable. This suggests that the model including hsCRP provided better fit, even after adjustment for the increase in number of predictors. Similarly, models that included hsCRP demonstrated better calibration (higher

*P*value for calibration), while models without hsCRP had larger deviations between the observed and predicted probabilities in the higher-risk categories (Figure 3). By contrast, the c-index again showed minimal ability to detect differences in model fit. As shown in the Appendix Table, all global measures of fit showed improvement when hsCRP was added to prediction models based on Framingham covariables alone.

### Clinical Risk Classification and Accuracy

*n*= 26927) into 4 risk groups defined by the ATP III categories of 10-year risk for CVD of 0% to less than 5%, 5% to less than 10%, 10% to less than 20%, and 20% or greater. We then compared the WHS models with and without hsCRP by cross-classifying expected risks and comparing these to the observed proportions of events in each group. While there was general agreement between these classifications (weighted κ= 0.86), the predicted risk categories changed substantially with the addition of hsCRP for women with at least a 5% 10-year risk according to only the Framingham risk variables (Table 3). Specifically, more than 20% of all participants with intermediate risk were reclassified with the addition of hsCRP; among those originally classified as having 5% to less than 10% risk, 12% moved down a category in risk and 10% moved up. Among those originally classified as having 10% to less than 20% risk, 19% were reclassified: 14% to a lower and 5% to a higher category. Among those at high risk (≥20% risk), 14% were reclassified into a lower-risk category. By contrast, among those with less than 5% risk according to Framingham covariables, only 2% were reclassified. Thus, overall in this low-risk cohort, with 88% of women in the lowest risk group, 4% were reclassified. However, this overall percentage depends heavily on the underlying risk and would be substantially greater in an older or higher-risk population. For comparison, according to ATP III risk categories based on National Cholesterol Education Program β-coefficients rather than those derived from the WHS, among women originally classified as having less than 5%, 5% to less than 10%, 10% to less than 20%, and 20% or greater 10-year risk, 4%, 38%, 42%, and 20%, respectively, were reclassified in the WHS model that included hsCRP.

## Discussion

## References

- WilsonPWD'AgostinoRBLevyDBelangerAMSilbershatzHKannelWBPrediction of coronary heart disease using risk factor categories.Circulation199897183747
- Executive Summary of The Third Report of The National Cholesterol Education Program (NCEP) Expert Panel on Detection, Evaluation, And Treatment of High Blood Cholesterol In Adults (Adult Treatment Panel III).JAMA2001285248697
- D'AgostinoRBSrGrundySSullivanLMWilsonPValidation of the Framingham coronary heart disease prediction scores: results of a multiple ethnic groups investigation.JAMA20012861807
- RidkerPMBrownNJVaughanDEHarrisonDGMehtaJLEstablished and emerging plasma biomarkers in the prediction of first atherothrombotic events.Circulation2004109IV619
- KhotUNKhotMBBajzerCTSappSKOhmanEMBrenerSJet al.Prevalence of conventional risk factors in patients with coronary heart disease.JAMA2003290898904
- GreenlandPKnollMDStamlerJNeatonJDDyerARGarsideDBet al.Major risk factors as antecedents of fatal and nonfatal coronary heart disease events.JAMA20032908917
- RidkerPMCushmanMStampferMJTracyRPHennekensCHInflammation, aspirin, and the risk of cardiovascular disease in apparently healthy men.N Engl J Med19973369739
- RidkerPMRifaiNRoseLBuringJECookNRComparison of C-reactive protein and low-density lipoprotein cholesterol levels in the prediction of first cardiovascular events.N Engl J Med2002347155765
- KoenigWLowelHBaumertJMeisingerCC-reactive protein modulates risk prediction based on the Framingham Score: implications for future risk assessment: results from a large cohort study in southern Germany.Circulation2004109134953
- BallantyneCMHoogeveenRCBangHCoreshJFolsomARHeissGet al.Lipoprotein-associated phospholipase A2, high-sensitivity C-reactive protein, and risk for incident coronary heart disease in middle-aged men and women in the Atherosclerosis Risk in Communities (ARIC) study.Circulation200410983742
- DaneshJWheelerJGHirschfieldGMEdaSEiriksdottirGRumleyAet al.C-reactive protein and other circulating markers of inflammation in the prediction of coronary heart disease.N Engl J Med2004350138797
- PaiJKPischonTMaJMansonJEHankinsonSEJoshipuraKet al.Inflammatory markers and the risk of coronary heart disease in men and women.N Engl J Med20043512599610
- RidkerPMWilsonPWGrundySMShould C-reactive protein be added to metabolic syndrome and to assessment of global cardiovascular risk?Circulation2004109281825
- CushmanMArnoldAMPsatyBMManolioTAKullerLHBurkeGLet al.C-reactive protein and the 10-year incidence of coronary heart disease in older men and women: the cardiovascular health study.Circulation20051122531
- RidkerPMRifaiNCookNRBradwinGBuringJENon-HDL cholesterol, apolipoproteins A-I and B100, standard lipid measures, lipid ratios, and CRP as risk factors for cardiovascular disease in women.JAMA200529432633
- LaaksonenDENiskanenLNisskanenKPunnonenKTuomainenTPSalonenJTC-reactive protein in the prediction of cardiovascular and overall mortality in middle-aged men: a population-based cohort study.Eur Heart J20052617839
- FestaAD'AgostinoRJrHowardGMykkanenLTracyRPHaffnerSMChronic subclinical inflammation as part of the insulin resistance syndrome: the Insulin Resistance Atherosclerosis Study (IRAS).Circulation2000102427
- FestaAD'AgostinoRJrTracyRPHaffnerSMElevated levels of acute-phase proteins and plasminogen activator inhibitor-1 predict the development of type 2 diabetes: the insulin resistance atherosclerosis study.Diabetes20025111317
- PradhanADMansonJERifaiNBuringJERidkerPMC-reactive protein, interleukin 6, and risk of developing type 2 diabetes mellitus.JAMA200128632734
- RidkerPMBuringJECookNRRifaiNC-reactive protein, the metabolic syndrome, and risk of incident cardiovascular events: an 8-year follow-up of 14 719 initially healthy American women.Circulation20031073917
- PearsonTAMensahGAAlexanderRWAndersonJLCannonRO3rdCriquiMet al.Markers of inflammation and cardiovascular disease: application to clinical and public health practice: a statement for healthcare professionals from the Centers for Disease Control and Prevention and the American Heart Association.Circulation2003107499511
- BuringJEHennekensCHThe Women's Health Study: summary of the study design.Journal of Myocardial Ischemia19924279
- RexrodeKMLeeIMCookNRHennekensCHBuringJEBaseline characteristics of participants in the Women's Health Study.J Womens Health Gend Based Med200091927
- RidkerPMCookNRLeeIMGordonDGazianoJMMansonJEet al.A randomized trial of low-dose aspirin in the primary prevention of cardiovascular disease in women.N Engl J Med20053521293304
- Hainline A, Karon J, Lippel KManual of Laboratory Operations, Lipid Research Clinics Program, and Lipid and Lipoprotein Analysis. 2nd ed.Bethesda, MDDepartment of Health and Human Services1982
- CoxDRRegression models and life tables (with discussion).Journal of the Royal Statistical Society B197234187220
- HarrellFEJr.Regression Modeling Strategies.New YorkSpringer2001
- HastieTTibshiraniRFriedmanJThe Elements of Statistical Learning: Data Mining, Inference and Prediction.New YorkSpringer-Verlag2001
- HanleyJAMcNeilBJThe meaning and use of the area under a receiver operating characteristic (ROC) curve.Radiology19821432936
- EfronBTibshiraniRAn Introduction to the Bootstrap.New YorkChapman & Hall1993
- BurnhamKPAndersonDRModel Selection and Inference: A Practical Information-Theoretic Approach.New YorkSpringer-Verlag1998
- NagelkerkeNJA note on a general definition of the coefficient of determination.Biomka1991786912
- AshAShwartzMR2: a useful measure of model performance when predicting a dichotomous outcome.Stat Med19991837584
- HosmerDWHosmerTLeCessieSLemeshowSA comparison of goodness-of-fit tests for the logistic regression model.Stat Med19971696580
- FleissJLStatistical Methods for Rates and Proportions. 2nd ed.New YorkWiley1981
- GreenlandPSmithSCJrGrundySMImproving coronary heart disease risk assessment in asymptomatic people: role of traditional risk factors and noninvasive cardiovascular tests.Circulation200110418637
- SullivanLMMassaroJMD'AgostinoRBSr.Presentation of multivariate data for clinical use: The Framingham Study risk score functions.Stat Med200423163160
- MoonsKGHarrellFESensitivity and specificity should be de-emphasized in diagnostic accuracy studies.Acad Radiol2003106702
- RidkerPMRifaiNPfefferMASacksFMMoyeLAGoldmanSet al.Inflammation, pravastatin, and the risk of coronary events after myocardial infarction in patients with average cholesterol levels. Cholesterol and Recurrent Events (CARE) Investigators.Circulation19989883944
- RidkerPMRifaiNClearfieldMDownsJRWeisSEMilesJSet al.Measurement of C-reactive protein for the targeting of statin therapy in the primary prevention of acute coronary events.N Engl J Med2001344195965
- KentSMTaylorAJUsefulness of lowering low-density lipoprotein cholesterol to <70 mg/dl and usefulness of C-reactive protein in patient selection.Am J Cardiol20039212247
- RidkerPMCannonCPMorrowDRifaiNRoseLMMcCabeCHet al.C-reactive protein levels and outcomes after statin therapy.N Engl J Med2005352208
- NissenSETuzcuEMSchoenhagenPCroweTSasielaWJTsaiJet al.Statin therapy, LDL cholesterol, C-reactive protein, and coronary artery disease.N Engl J Med20053522938
- BoekholdtSMHackCESandhuMSLubenRBinghamSAWarehamNJet al.C-reactive protein levels and coronary artery disease incidence and mortality in apparently healthy men and women: The EPIC-Norfolk prospective population study 1993-2003.Atherosclerosis2005
- HarrellFEJrLeeKLMarkDBMultivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors.Stat Med19961536187

### Appendix

#### Computation of 10-Year Risk

*x*by the appropriate coefficient in Table 1 and sum these (= Σβ ×

*x*). The risk may then be computed from the following equation:

^{exp(Σβ × x − 8.795)}

#### Additional Measures of Model Fit

The likelihood ratio chi-square provides a global test of model fit. It is a function of the degrees of freedom, or number of terms in the model. The difference between chi-square values provides a test of the model improvement with hsCRP (

*P*< 0.0001 for both the WHS and ATP III models).The Bayes information criterion is a function of the log likelihood but adds a penalty for added variables based on the sample size (The Bayes information criterion is a function of the log likelihood but adds a penalty for added variables based on the sample size (28). It is not influenced by the number of predictors, so models can thus be compared directly. Lower values reflect better fit, suggesting improvement with the addition of hsCRP.). It is not influenced by the number of predictors, so models can thus be compared directly. Lower values reflect better fit, suggesting improvement with the addition of hsCRP.

The Bayes information criterion weight provides an estimate of the posterior probability of each model given the set of candidate models considered (The Bayes information criterion weight provides an estimate of the posterior probability of each model given the set of candidate models considered (29, 32). The weights suggest a much higher probability that the WHS model that includes hsCRP is correct., 32. The weights suggest a much higher probability that the WHS model that includes hsCRP is correct.

The Akaike information criterion is a function of the log likelihood that adds a penalty of 2 for each added variable (The Akaike information criterion is a function of the log likelihood that adds a penalty of 2 for each added variable (32), less extreme than the penalty used in the Bayes information criterion. Lower values are better, again suggesting improvement with hsCRP.), less extreme than the penalty used in the Bayes information criterion. Lower values are better, again suggesting improvement with hsCRP.

The Akaike information criterion weights reflect the relative likelihood of a model given the data and the set of models (The Akaike information criterion weights reflect the relative likelihood of a model given the data and the set of models (32). These weights display a clear preference for the models with hsCRP.). These weights display a clear preference for the models with hsCRP.

Nagelkerke's generalized model R

^{2}(Nagelkerke's generalized model R2(33, 34) is a measure of the fraction of the−2 log likelihood explained by the predictors, analogous to the percentage of variance explained in a linear model. It is adjusted to a range of 0 to 1 and is higher for models with hsCRP, both in the original data and after adjustment for optimism using the bootstrap (31, 48).-34) is a measure of the fraction of the−2 log likelihood explained by the predictors, analogous to the percentage of variance explained in a linear model. It is adjusted to a range of 0 to 1 and is higher for models with hsCRP, both in the original data and after adjustment for optimism using the bootstrap (Nagelkerke's generalized model R2(33, 34) is a measure of the fraction of the−2 log likelihood explained by the predictors, analogous to the percentage of variance explained in a linear model. It is adjusted to a range of 0 to 1 and is higher for models with hsCRP, both in the original data and after adjustment for optimism using the bootstrap (31, 48)., 48.The D-statistic of Royston and Sauerbrei (The D-statistic of Royston and Sauerbrei (35) measures the separation of survival curves across levels of the predictor variables, analogous to distance between Kaplan–Meier curves. This is higher for models that included hsCRP, even after adjustment for optimism, suggesting better prediction for these models.) measures the separation of survival curves across levels of the predictor variables, analogous to distance between Kaplan–Meier curves. This is higher for models that included hsCRP, even after adjustment for optimism, suggesting better prediction for these models.

The Brier score (The Brier score (28) computes the sum of squared differences between the observed outcome and the fitted probability. It is lower for models that included hsCRP, indicating that the predicted probabilities are closer to the observed outcomes.) computes the sum of squared differences between the observed outcome and the fitted probability. It is lower for models that included hsCRP, indicating that the predicted probabilities are closer to the observed outcomes.

The c-index represents the area under the receiver-operating characteristic curve (The c-index represents the area under the receiver-operating characteristic curve (30), allowing for censored data. This is a measure of discrimination based on ranks and is similar but slightly higher for models that included hsCRP, even after adjustment for optimism. The c-statistic is the probability that, for a randomly selected pair of subjects, one diseased and the other nondiseased, the person with disease will have the higher estimated disease probability according to the model.), allowing for censored data. This is a measure of discrimination based on ranks and is similar but slightly higher for models that included hsCRP, even after adjustment for optimism. The c-statistic is the probability that, for a randomly selected pair of subjects, one diseased and the other nondiseased, the person with disease will have the higher estimated disease probability according to the model.

The Hosmer–Lemeshow calibration statistic (The Hosmer–Lemeshow calibration statistic (36) classifies predicted probabilities into categories and compares the mean predicted probability with the observed risk within each category. A P value representing a significant difference indicates a lack of fit. When decile categories are used, the predicted probability is less than 5% for the first 9 of 10 categories. Calibration is adequate for all models that use this measure and is somewhat better for models without hsCRP. The calibration statistic based on risk percentage compares observed and predicted risk by using 10 categories based on 2–percentage point increments in predicted risk, from 0% to 2% risk to 18% or greater risk. This statistic indicates significant deviation of observed and predicted values in models without hsCRP, suggesting a lack of fit in higher-risk categories.) classifies predicted probabilities into categories and compares the mean predicted probability with the observed risk within each category. A

*P*value representing a significant difference indicates a lack of fit. When decile categories are used, the predicted probability is less than 5% for the first 9 of 10 categories. Calibration is adequate for all models that use this measure and is somewhat better for models without hsCRP. The calibration statistic based on risk percentage compares observed and predicted risk by using 10 categories based on 2–percentage point increments in predicted risk, from 0% to 2% risk to 18% or greater risk. This statistic indicates significant deviation of observed and predicted values in models without hsCRP, suggesting a lack of fit in higher-risk categories.

## This feature is available only to Registered Users

Subscribe/Learn More## 0 Comments