The full content of Annals is available to subscribers

Subscribe/Learn More  >
Research and Reporting Methods |

Net Reclassification Improvement: Computation, Interpretation, and Controversies: A Literature Review and Clinician's Guide

Maarten J.G. Leening, MD, MSc; Moniek M. Vedder, MSc; Jacqueline C.M. Witteman, PhD; Michael J. Pencina, PhD; and Ewout W. Steyerberg, PhD
[+] Article, Author, and Disclosure Information

From Erasmus MC - University Medical Center Rotterdam, Rotterdam, the Netherlands, and Duke University, Durham, North Carolina.

Grant Support: By the Netherlands Organisation for Health Research and Development (ZonMw) and the Netherlands Organisation for Scientific Research (NWO) (ZonMw HTA grant 80-82500-98-10208; Vici grant 918-76-619) and the Center for Translational Molecular Medicine (PCMM project grant). The funding sources had no role in the design or conduct of the study; the collection, management, analysis, or interpretation of the data; or the preparation, review, or approval of the manuscript.

Potential Conflicts of Interest: Disclosures can be viewed at www.acponline.org/authors/icmje/ConflictOfInterestForms.do?msNum=M13-1522.

Requests for Single Reprints: Maarten J.G. Leening, MD, MSc, Department of Epidemiology, Erasmus MC - University Medical Center Rotterdam, PO Box 2040, 3000 CA Rotterdam, the Netherlands; e-mail, m.leening@erasmusmc.nl.

Current Author Addresses: Drs. Leening and Witteman: Department of Epidemiology, Erasmus MC - University Medical Center Rotterdam, Dr. Molenwaterplein 50, 3015 GE Rotterdam, the Netherlands.

Ms. Vedder and Dr. Steyerberg: Department of Public Health, Erasmus MC - University Medical Center Rotterdam, Dr. Molenwaterplein 50, 3015 GE Rotterdam, the Netherlands.

Dr. Pencina: Department of Biostatistics and Bioinformatics, Duke Clinical Research Institute, Duke University, 2400 Pratt Street, Durham, NC 27715.

Author Contributions: Conception and design: M.J.G. Leening, E.W. Steyerberg.

Analysis and interpretation of the data: M.J.G. Leening, M.M. Vedder, E.W. Steyerberg.

Drafting of the article: M.J.G. Leening.

Critical revision of the article for important intellectual content: M.J.G. Leening, M.M. Vedder, J.C.M. Witteman, M.J. Pencina, E.W. Steyerberg.

Final approval of the article: M.J.G. Leening, M.M. Vedder, J.C.M. Witteman, M.J. Pencina, E.W. Steyerberg.

Statistical expertise: M.J.G. Leening, M.J. Pencina, E.W. Steyerberg.

Obtaining of funding: E.W. Steyerberg.

Administrative, technical, or logistic support: M.J.G. Leening.

Collection and assembly of data: M.J.G. Leening, M.M. Vedder.

Ann Intern Med. 2014;160(2):122-131. doi:10.7326/M13-1522
Text Size: A A A

The net reclassification improvement (NRI) is an increasingly popular measure for evaluating improvements in risk predictions. This article details a review of 67 publications in high-impact general clinical journals that considered the NRI. Incomplete reporting of NRI methods, incorrect calculation, and common misinterpretations were found. To aid improved applications of the NRI, the article elaborates on several aspects of the computation and interpretation in various settings. Limitations and controversies are discussed, including the effect of miscalibration of prediction models, the use of the continuous NRI and “clinical NRI,” and the relation with decision analytic measures. A systematic approach toward presenting NRI analysis is proposed: Detail and motivate the methods used for computation of the NRI, use clinically meaningful risk cutoffs for the category-based NRI, report both NRI components, address issues of calibration, and do not interpret the overall NRI as a percentage of the study population reclassified. Promising NRI findings need to be followed with decision analytic or formal cost-effectiveness evaluations.


Grahic Jump Location
Appendix Figure 1.

Summary of evidence search and selection.

The search was last updated on 23 April 2013.

Grahic Jump Location
Grahic Jump Location
Appendix Figure 2.

Example of a reclassification graph with superimposed cut points of predicted risk.

The graph shows 10-y risk for incident CHD in women from the ARIC (Atherosclerosis Risk in Communities) Study predicted by a model containing only the Framingham risk score variables (horizontal axis) against risk predicted by a model containing Framingham risk score variables and retinal arteriolar caliber (vertical axis). Lines at predicted risks of 10% and 20% are superimposed to show reclassification over clinically relevant cut points (2, 89) and thereby create a visual representation of a reclassification table (Appendix Table 3). Of note, most women in this study have a low (<10%) predicted risk for CHD, both with the Framingham variables and with the model that includes retinal arteriolar caliber. The graph also shows that a limited number of women are reclassified over the cut points (i.e., only a small proportion of dots lies in the off-diagonal cells of the graph). CHD = coronary heart disease. (Reproduced from McGeechan and colleagues [130] with permission of the American Journal of Cardiology.)

Grahic Jump Location




Citing articles are presented as examples only. In non-demo SCM6 implementation, integration with CrossRef’s "Cited By" API will populate this tab (http://www.crossref.org/citedby.html).


Submit a Comment/Letter
Submit a Comment/Letter

Summary for Patients

Clinical Slide Sets

Terms of Use

The In the Clinic® slide sets are owned and copyrighted by the American College of Physicians (ACP). All text, graphics, trademarks, and other intellectual property incorporated into the slide sets remain the sole and exclusive property of the ACP. The slide sets may be used only by the person who downloads or purchases them and only for the purpose of presenting them during not-for-profit educational activities. Users may incorporate the entire slide set or selected individual slides into their own teaching presentations but may not alter the content of the slides in any way or remove the ACP copyright notice. Users may make print copies for use as hand-outs for the audience the user is personally addressing but may not otherwise reproduce or distribute the slides by any means or media, including but not limited to sending them as e-mail attachments, posting them on Internet or Intranet sites, publishing them in meeting proceedings, or making them available for sale or distribution in any unauthorized form, without the express written permission of the ACP. Unauthorized use of the In the Clinic slide sets will constitute copyright infringement.


Buy Now for $32.00

to gain full access to the content and tools.

Want to Subscribe?

Learn more about subscription options

Related Articles
Topic Collections
PubMed Articles
Forgot your password?
Enter your username and email address. We'll send you a reminder to the email address on record.