Andrew J. Vickers, PhD; Margaret Pepe, PhD
Grant Support: In part by funds from David H. Koch provided through the Prostate Cancer Foundation, the Sidney Kimmel Center for Prostate and Urologic Cancers, and SPORE grant P50-CA92629 from the National Cancer Institute to Dr. Howard Scher. This work was also supported in part by a National Institutes of Health/National Cancer Institute Cancer Center Support Grant to Memorial Sloan-Kettering Cancer Center under award P30 CA00874 and by grant R01CA152089 from the National Institutes of Health.
Potential Conflicts of Interest: None disclosed. Forms can be viewed at www.acponline.org/authors/icmje/ConflictOfInterestForms.do?msNum=M13-2841.
Requests for Single Reprints: Andrew J. Vickers, PhD, Department of Epidemiology & Biostatistics, Memorial Sloan-Kettering Cancer Center, 307 East 63rd Street, 2nd Floor, New York, NY 10021; e-mail, firstname.lastname@example.org.
Current Author Addresses: Dr. Vickers: Department of Epidemiology & Biostatistics, Memorial Sloan-Kettering Cancer Center, 307 East 63rd Street, 2nd Floor, New York, NY 10021.
Dr. Pepe: Fred Hutchinson Cancer Research Center, 1100 Fairview Avenue North, PO Box 19024, Seattle, WA 98109-1024.
Vickers A., Pepe M.; Does the Net Reclassification Improvement Help Us Evaluate Models and Markers?. Ann Intern Med. 2014;160:136-137. doi: 10.7326/M13-2841
Download citation file:
Published: Ann Intern Med. 2014;160(2):136-137.
Few statistical methods have been as rapidly adopted as the net reclassification improvement (NRI). The reasons that the NRI has become so popular so quickly are not hard to fathom. First, it is an appealing concept to investigators. Instead of reporting that their new marker improved the area under the receiver-operating characteristic curve (AUC)—a statistical abstraction at the best of times—they can say that their marker led to “reclassification,” suggesting clinical value. The popularity of the NRI also stems from a belief that the AUC is insensitive. For example, in an early paper on the NRI, Pencina and colleagues stated that the AUC is “too conservative” because it “hardly moves after a few good risk factors are already included in the model” (1). The reason that the AUC “hardly moves” may not be because we are evaluating inappropriately but because it is not easy to improve on a good model given the inherent uncertainties of medical prediction. The belief that the AUC is insensitive also resulted from the practice of reporting P values comparing the AUC of a model with and without a marker, an approach that is statistically invalid and associated with extremely low power (2). It is not surprising that investigators would prefer the NRI when a marker that barely shifts the AUC can be said to lead to “statistically significant reclassification,” but it is also a red flag when the reason for the adoption of a test is more favorable results rather than a careful consideration of statistical validity.
to gain full access to the content and tools.
Learn more about subscription options.
Register Now for a free account.
Results provided by:
Copyright © 2016 American College of Physicians. All Rights Reserved.
Print ISSN: 0003-4819 | Online ISSN: 1539-3704
Conditions of Use
This PDF is available to Subscribers Only