0
Research and Reporting Methods |

Strengthening the Reporting of Genetic Risk Prediction Studies: The GRIPS Statement FREE

A. Cecile J.W. Janssens, PhD; John P.A. Ioannidis, MD, DSc; Cornelia M. van Duijn, PhD; Julian Little, PhD; Muin J. Khoury, MD, PhD, for the GRIPS Group
[+] Article and Author Information

For members of the GRIPS (Genetic RIsk Prediction Studies) Group, see the Appendix.


From Erasmus University Medical Center, Rotterdam, the Netherlands; University of Ioannina School of Medicine and Biomedical Research Institute, Ioannina, Greece; Tufts University School of Medicine, Boston, Massachusetts; Stanford University School of Medicine, Stanford, California; University of Ottawa, Ottawa, Ontario, Canada; and Centers for Disease Control and Prevention, Atlanta, Georgia.


Disclaimer: The opinions, findings, and conclusions expressed here are those of the authors and do not reflect the views of the Department of Health and Human Services or represent the official position or policies of the Tufts Clinical and Translational Science Institute.

Grant Support: The Centers for Disease Control and Prevention sponsored the workshop on behalf of the Human Genome Epidemiology Network. Dr. Janssens is supported by grants from Erasmus University Medical Center, the Center for Medical Systems Biology in the framework of the Netherlands Genomics Initiative, and the VIDI grant of the Netherlands Organisation for Scientific Research. Dr. Ioannidis is supported by Tufts Clinical and Translational Science Institute, which is supported by the National Institutes of Health, National Center for Research Resources (UL1 RR025752). Dr. Little holds a Canada Research Chair in Human Genome Epidemiology. The funding sources had no role in the study design, data collection, analysis, preparation of the manuscript, or decision to submit the manuscript for publication.

Potential Conflicts of Interest: Disclosures can be viewed at www.acponline.org/authors/icmje/ConflictOfInterestForms.do?msNum=M10-2517.

Requests for Single Reprints: A. Cecile J.W. Janssens, PhD, Department of Epidemiology, Erasmus University Medical Center, PO Box 2040, 3000 CA Rotterdam, the Netherlands; e-mail, a.janssens@erasmusmc.nl.

Current Author Addresses: Drs. Janssens and van Duijn: Department of Epidemiology, Erasmus University Medical Center, PO Box 2040, 3000 CA Rotterdam, the Netherlands.

Dr. Ioannidis: Department of Hygiene and Epidemiology, University of Ioannina School of Medicine, Ioannina 45110, Greece.

Dr. Little: Department of Epidemiology and Community Medicine, University of Ottawa, 451 Smyth Road, Ottawa, Ontario K1H 8M5, Canada.

Dr. Khoury: Office of Public Health Genomics, Centers for Disease Control and Prevention, 1600 Clifton Road, Atlanta, GA 30333.

Author Contributions: Conception and design: A.C.J.W. Janssens, J.P.A. Ioannidis, C.M. van Duijn, J. Little, M.J. Khoury.

Analysis and interpretation of the data: A.C.J.W. Janssens, J.P.A. Ioannidis, C.M. van Duijn, J. Little, M.J. Khoury.

Drafting of the article: A.C.J.W. Janssens, J.P.A. Ioannidis, C.M. van Duijn, M.J. Khoury.

Critical revision of the article for important intellectual content: A.C.J.W. Janssens, J.P.A. Ioannidis, C.M. van Duijn, J. Little, M.J. Khoury.

Final approval of the article: A.C.J.W. Janssens, J.P.A. Ioannidis, J. Little, M.J. Khoury.

Statistical expertise: J.P.A. Ioannidis.

Obtaining of funding: M.J. Khoury.

Administrative, technical, or logistic support: M.J. Khoury.


Ann Intern Med. 2011;154(6):421-425. doi:10.7326/0003-4819-154-6-201103150-00008
Text Size: A A A
Key Summary Points

  • The rapid and continuing progress in gene discovery for complex diseases is fueling interest in the potential application of genetic risk models for clinical and public health practice.

  • The number of studies assessing the predictive ability is steadily increasing, but the quality and completeness of reporting varies.

  • A multidisciplinary workshop sponsored by the Human Genome Epidemiology Network developed a checklist of 25 items recommended for strengthening the reporting of Genetic RIsk Prediction Studies (GRIPS), building on the principles established by prior reporting guidelines.

  • These recommendations aim to enhance the transparency of study reporting, and thereby to improve the synthesis and application of information from multiple studies that might differ in design, conduct, or analysis.

  • A detailed Explanation and Elaboration document is published at www.plosmedicine.org.

Editor's Note: In order to encourage dissemination of the GRIPS Statement, this article is freely accessible onwww.annals.organd will also be published by PLoS Medicine, BMJ, Circulation: Cardiovascular Genetics, European Journal of Clinical Investigation, European Journal of Epidemiology, European Journal of Human Genetics, Genetics in Medicine, Genome Medicine, and Journal of Clinical Epidemiology.

The recent successes of genome-wide association studies and the promises of whole genome sequencing fuel interest in the translation of this new wave of basic genetic knowledge to health care practice. Knowledge about genetic risk factors may be used to target diagnostic, preventive, and therapeutic interventions for complex disorders based on a person's genetic risk or to complement existing risk models based on classical nongenetic factors, such as the Framingham risk score for cardiovascular disease. Implementation of genetic risk prediction in health care requires a series of studies that encompass all phases of translational research (12), starting with a comprehensive evaluation of genetic risk prediction.

With increasing numbers of discovered genetic markers that can be used in future genetic risk prediction studies, it is crucial to enhance the quality of the reporting of these studies since valid interpretation could be compromised by the lack of reporting of key information. Information that is often missing includes details in the description of how the study was designed and conducted (for example, how genetic variants were selected and coded, how risk models or genetic risk scores were constructed, and how risk categories were chosen) or how the results should be interpreted. An appropriate assessment of the study's strengths and weaknesses is not possible without this information. There is ample evidence that prediction research often suffers from poor design and bias, and these may also have an impact on the results of the studies and on models of disease outcomes based on these studies (35). Although most prognostic studies published to date claim significant results (67), very few translate to clinically useful applications. Just as for observational epidemiological studies (8), poor reporting complicates the use of the specific study for research, clinical, or public health purposes and hampers the synthesis of evidence across studies.

Reporting guidelines have been published for various research designs (9), and these contain many items that are also relevant to genetic risk prediction studies. In particular, the guidelines for genetic association studies (STREGA) have relevant items on the assessment of genetic variants, and the guidelines for observational studies (STROBE) have relevant items about the reporting of study design. The guidelines for diagnostic studies (STARD) and those for tumor marker prognostic studies (REMARK) include relevant items about test evaluation; the REMARK guidelines also have relevant items about risk prediction (5, 1012). However, none of these guidelines are fully suited to genetic risk prediction studies, an emerging field of investigation with specific methodological issues that need to be addressed, such as the handling of large numbers of genetic variants (from 10s to 10 000s) and flexibility in handling such large numbers in analyses. We organized a 2-day workshop with an international group of risk prediction researchers, epidemiologists, geneticists, methodologists, statisticians, and journal editors to develop recommendations for the reporting of Genetic RIsk Prediction Studies (GRIPS).

Genetic risk prediction studies typically develop or validate models that predict the risk of disease, but they are also being investigated for use in predicting prognostic outcome, treatment response, or treatment-related harms. Risk prediction models are statistical algorithms, which may be simple genetic risk scores (e.g., risk allele counts), regression analyses (e.g., weighted risk scores or predicted risks), or more complex analytic approaches, such as support vector machine learning or classification trees. The risk models may be based on genetic variants only or include both genetic and nongenetic risk factors (13).

The 25 items of the GRIPS statement are intended to maximize the transparency, quality, and completeness of reporting on research methodology and findings in a particular study. It is important to emphasize that these recommendations are guidelines only for how to report research and do not prescribe how to perform genetic risk prediction studies. The guidelines do not support or oppose the choice of any particular study design or method, e.g., the guidelines recommend that the study population should be described but do not specify which population is preferred in a particular study.

The intended audience for the reporting guidelines is broad and includes epidemiologists, geneticists, statisticians, clinician scientists, and laboratory-based investigators who undertake genetic risk prediction studies, as well as journal editors and reviewers who have to appraise the design, conduct, and analysis of such studies. In addition, it includes “users” of such studies who wish to understand the basic premise, design, and limitations of genetic prediction studies in order to interpret the results for their potential application in health care. These guidelines are also intended to ensure that essential data from future genetic risk prediction studies are presented in standardized form, which will facilitate information synthesis as part of systematic reviews and meta-analyses.

Items presented in the checklist are relevant for a wide array of risk prediction studies because GRIPS focuses on the main aspects of the design and analysis of risk prediction studies. GRIPS does not address randomized trials that may be performed to test risk models, nor does it specifically address decision analyses, cost-effectiveness analyses, assessment of health care needs, or assessment of barriers to health care implementation (14). Once the performance of a risk model has been established, these next steps toward implementation require further evaluation (10, 15). For the reporting of these studies, which go beyond the assessment of genetic risk models as such, additional requirements apply. However, proper documentation of genetic predictive research according to GRIPS might facilitate the translation of research findings into clinical and public health practice.

The GRIPS statement was developed by a multidisciplinary panel of 25 risk prediction researchers, epidemiologists, geneticists, methodologists, statisticians, and journal editors, 7 of whom were also part of the STREGA initiative (11). They attended a 2-day meeting in Atlanta, Georgia (United States) in December 2009 that was sponsored by the U.S. Centers for Disease Control and Prevention on behalf of the Human Genome Epidemiology Network (HuGENet) (16). Participants discussed a draft version of the guidelines that was prepared and distributed before the meeting. This draft version was developed on the basis of existing reporting guidelines, namely STREGA (11), REMARK (5), and STARD (12). These were selected out of all available guidelines (www.equator-network.org) because of their focus on observational study designs and genetic factors (STREGA), prediction models (REMARK), and test evaluation (REMARK and STARD). During the meeting, methodological issues pertinent to risk prediction studies were addressed in presentations. Workshop participants were asked to change, combine, or delete proposed items and add additional items if necessary. Participants had extensive electronic correspondence after the meeting. To harmonize our recommendations for genetic risk prediction studies with previous guidelines, we chose the same wording for the items wherever possible. Finally, we tried to create consistency with previous guidelines for the evaluation of risk prediction studies of cardiovascular diseases and cancer (2, 17). The final version of the checklist is presented in the Table.

Table Jump PlaceholderTable.  Reporting Recommendations for Evaluations of Risk Prediction Models That Include Genetic Variants

Accompanying this GRIPS statement, an Explanation and Elaboration document has been written (www.plosmedicine.org), modeled after those developed for other reporting guidelines (1821). The Explanation and Elaboration document illustrates each item with at least 1 published example that we consider transparent in reporting, explains the rationale for its inclusion in the checklist, and presents details of the items that need to be addressed to ensure transparent reporting. The Explanation and Elaboration document was produced after the meeting. The document was prepared by a small subgroup and shared with all workshop participants for additional revisions and final approval.

High-quality reporting reveals the strengths and weaknesses of empirical studies, facilitates the interpretation of the scientific and health care relevance of the results—especially within the framework of systematic reviews and meta-analyses—and helps build a solid evidence base for moving genomic discoveries into applications in health care practice. The GRIPS guidelines were developed to improve the transparency, quality, and completeness of the reporting of genetic risk prediction studies. As outlined in the introduction, GRIPS does not prescribe how studies should be designed, conducted, or analyzed, and therefore, the guidelines should not be used to assess the quality of empirical studies (22). The guidelines should be used only to check whether all essential items are adequately reported.

Finally, the methodology for designing and assessing genetic risk prediction models is still developing. For example, newer measures of reclassification were first introduced in 2007 (23), and several alternative reclassification measures have been proposed (24). Which measures to apply and when to use measures of reclassification are still subject to ongoing evaluation and discussion (25). Furthermore, alternative strategies for constructing risk models other than simple regression analyses are being explored, and these may add increased complexity to the reporting. In formulating the items of the GRIPS statement, these methodological advances were anticipated. It is for this reason that the GRIPS statement recommends how a study should be reported and not how a study should be conducted or analyzed. Therefore, methodological and analytical developments will not immediately impact the validity and relevance of the items, but the GRIPS statement will be updated when this is warranted by essential new developments in the construction and evaluation of genetic risk models.

Khoury MJ, Gwinn M, Yoon PW, Dowling N, Moore CA, Bradley L.  The continuum of translation research in genomic medicine: how can we accelerate the appropriate integration of human genome discoveries into health care and disease prevention? Genet Med. 2007; 9:665-74.
PubMed
CrossRef
 
Hlatky MA, Greenland P, Arnett DK, Ballantyne CM, Criqui MH, Elkind MS, et al. American Heart Association Expert Panel on Subclinical Atherosclerotic Diseases and Emerging Risk Factors and the Stroke Council.  Criteria for evaluation of novel markers of cardiovascular risk: a scientific statement from the American Heart Association. Circulation. 2009; 119:2408-16.
PubMed
 
Kyzas PA, Denaxa-Kyza D, Ioannidis JP.  Quality of reporting of cancer prognostic marker studies: association with reported prognostic effect. J Natl Cancer Inst. 2007; 99:236-43.
PubMed
 
Kyzas PA, Loizou KT, Ioannidis JP.  Selective reporting biases in cancer prognostic factor studies. J Natl Cancer Inst. 2005; 97:1043-55.
PubMed
 
McShane LM, Altman DG, Sauerbrei W, Taube SE, Gion M, Clark GM, Statistics Subcommittee of the NCI-EORTC Working Group on Cancer Diagnostics.  REporting recommendations for tumor MARKer prognostic studies (REMARK). Nat Clin Pract Urol. 2005; 2:416-22.
PubMed
 
Kyzas PA, Denaxa-Kyza D, Ioannidis JP.  Almost all articles on cancer prognostic markers report statistically significant results. Eur J Cancer. 2007; 43:2559-79.
PubMed
 
Tzoulaki I, Liberopoulos G, Ioannidis JP.  Assessment of claims of improved prediction beyond the Framingham risk score. JAMA. 2009; 302:2345-52.
PubMed
 
von Elm E, Egger M.  The scandal of poor epidemiological research [Editorial]. BMJ. 2004; 329:868-9.
PubMed
 
Simera I, Moher D, Hoey J, Schulz KF, Altman DG.  A catalogue of reporting guidelines for health research. Eur J Clin Invest. 2010; 40:35-53.
PubMed
 
von Elm E, Altman DG, Egger M, Pocock SJ, Gøtzsche PC, Vandenbroucke JP, STROBE Initiative.  The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. PLoS Med. 2007; 4:296.
PubMed
 
Little J, Higgins JP, Ioannidis JP, Moher D, Gagnon F, von Elm E, et al. STrengthening the REporting of Genetic Association Studies.  STrengthening the REporting of Genetic Association Studies (STREGA): an extension of the STROBE statement. PLoS Med. 2009; 6:22.
PubMed
 
Bossuyt PM, Reitsma JB, Bruns DE, Gatsonis CA, Glasziou PP, Irwig LM, et al. Standards for Reporting of Diagnostic Accuracy.  Towards complete and accurate reporting of studies of diagnostic accuracy: the STARD initiative. BMJ. 2003; 326:41-4.
PubMed
 
Janssens AC, van Duijn CM.  Genome-based prediction of common diseases: methodological considerations for future research. Genome Med. 2009; 1:20.
PubMed
 
Khoury MJ, Gwinn M, Ioannidis JP.  The emergence of translational epidemiology: from scientific discovery to population health impact. Am J Epidemiol. 2010; 172:517-24.
PubMed
 
Moons KG, Altman DG, Vergouwe Y, Royston P.  Prognosis and prognostic research: application and impact of prognostic models in clinical practice. BMJ. 2009; 338:b606.
PubMed
 
Khoury MJ, Dorman JS.  The Human Genome Epidemiology Network [Editorial]. Am J Epidemiol. 1998; 148:1-3.
PubMed
 
Freedman AN, Seminara D, Gail MH, Hartge P, Colditz GA, Ballard-Barbash R. et al.  Cancer risk prediction models: a workshop on development, evaluation, and application. J Natl Cancer Inst. 2005; 97:715-23.
PubMed
 
Altman DG, Schulz KF, Moher D, Egger M, Davidoff F, Elbourne D, et al. CONSORT GROUP (Consolidated Standards of Reporting Trials).  The revised CONSORT statement for reporting randomized trials: explanation and elaboration. Ann Intern Med. 2001; 134:663-94.
PubMed
 
Bossuyt PM, Reitsma JB, Bruns DE, Gatsonis CA, Glasziou PP, Irwig LM, et al. Standards for Reporting of Diagnostic Accuracy.  The STARD statement for reporting studies of diagnostic accuracy: explanation and elaboration. Ann Intern Med. 2003; 138:W1-12.
PubMed
 
Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC, Ioannidis JP. et al.  The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. PLoS Med. 2009; 6:1000100.
PubMed
 
Vandenbroucke JP, von Elm E, Altman DG, Gøtzsche PC, Mulrow CD, Pocock SJ, et al. STROBE Initiative.  Strengthening the Reporting of Observational Studies in Epidemiology (STROBE): explanation and elaboration. PLoS Med. 2007; 4:297.
PubMed
 
Vandenbroucke JP.  STREGA, STROBE, STARD, SQUIRE, MOOSE, PRISMA, GNOSIS, TREND, ORION, COREQ, QUOROM, REMARK … and CONSORT: for whom does the guideline toll? J Clin Epidemiol. 2009; 62:594-6.
PubMed
 
Cook NR.  Use and misuse of the receiver operating characteristic curve in risk prediction. Circulation. 2007; 115:928-35.
PubMed
 
Pencina MJ, D'Agostino RB Sr, D'Agostino RB Jr, Vasan RS.  Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond. Stat Med. 2008; 27:157-72.
PubMed
 
Janssens AC, Khoury MJ.  Assessment of improved prediction beyond traditional risk factors: when does a difference make a difference? [Editorial]. Circ Cardiovasc Genet. 2010; 3:3-5.
PubMed
 
Appendix: Members of the GRIPS Group

A. Cecile J.W. Janssens, John P.A. Ioannidis, Sara Bedrosian, Paolo Boffetta, Siobhan M. Dolan, Nicole Dowling, Isabel Fortier, Andrew N. Freedman, Jeremy M. Grimshaw, Jeffrey Gulcher, Marta Gwinn, Mark A. Hlatky, Holly Janes, Peter Kraft, Stephanie Melillo, Christopher J. O'Donnell, Michael J. Pencina, David Ransohoff, Sheri D. Schully, Daniela Seminara, Deborah M. Winn, Caroline F. Wright, Cornelia M. van Duijn, Julian Little, and Muin J. Khoury.

Figures

Tables

Table Jump PlaceholderTable.  Reporting Recommendations for Evaluations of Risk Prediction Models That Include Genetic Variants

References

Khoury MJ, Gwinn M, Yoon PW, Dowling N, Moore CA, Bradley L.  The continuum of translation research in genomic medicine: how can we accelerate the appropriate integration of human genome discoveries into health care and disease prevention? Genet Med. 2007; 9:665-74.
PubMed
CrossRef
 
Hlatky MA, Greenland P, Arnett DK, Ballantyne CM, Criqui MH, Elkind MS, et al. American Heart Association Expert Panel on Subclinical Atherosclerotic Diseases and Emerging Risk Factors and the Stroke Council.  Criteria for evaluation of novel markers of cardiovascular risk: a scientific statement from the American Heart Association. Circulation. 2009; 119:2408-16.
PubMed
 
Kyzas PA, Denaxa-Kyza D, Ioannidis JP.  Quality of reporting of cancer prognostic marker studies: association with reported prognostic effect. J Natl Cancer Inst. 2007; 99:236-43.
PubMed
 
Kyzas PA, Loizou KT, Ioannidis JP.  Selective reporting biases in cancer prognostic factor studies. J Natl Cancer Inst. 2005; 97:1043-55.
PubMed
 
McShane LM, Altman DG, Sauerbrei W, Taube SE, Gion M, Clark GM, Statistics Subcommittee of the NCI-EORTC Working Group on Cancer Diagnostics.  REporting recommendations for tumor MARKer prognostic studies (REMARK). Nat Clin Pract Urol. 2005; 2:416-22.
PubMed
 
Kyzas PA, Denaxa-Kyza D, Ioannidis JP.  Almost all articles on cancer prognostic markers report statistically significant results. Eur J Cancer. 2007; 43:2559-79.
PubMed
 
Tzoulaki I, Liberopoulos G, Ioannidis JP.  Assessment of claims of improved prediction beyond the Framingham risk score. JAMA. 2009; 302:2345-52.
PubMed
 
von Elm E, Egger M.  The scandal of poor epidemiological research [Editorial]. BMJ. 2004; 329:868-9.
PubMed
 
Simera I, Moher D, Hoey J, Schulz KF, Altman DG.  A catalogue of reporting guidelines for health research. Eur J Clin Invest. 2010; 40:35-53.
PubMed
 
von Elm E, Altman DG, Egger M, Pocock SJ, Gøtzsche PC, Vandenbroucke JP, STROBE Initiative.  The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. PLoS Med. 2007; 4:296.
PubMed
 
Little J, Higgins JP, Ioannidis JP, Moher D, Gagnon F, von Elm E, et al. STrengthening the REporting of Genetic Association Studies.  STrengthening the REporting of Genetic Association Studies (STREGA): an extension of the STROBE statement. PLoS Med. 2009; 6:22.
PubMed
 
Bossuyt PM, Reitsma JB, Bruns DE, Gatsonis CA, Glasziou PP, Irwig LM, et al. Standards for Reporting of Diagnostic Accuracy.  Towards complete and accurate reporting of studies of diagnostic accuracy: the STARD initiative. BMJ. 2003; 326:41-4.
PubMed
 
Janssens AC, van Duijn CM.  Genome-based prediction of common diseases: methodological considerations for future research. Genome Med. 2009; 1:20.
PubMed
 
Khoury MJ, Gwinn M, Ioannidis JP.  The emergence of translational epidemiology: from scientific discovery to population health impact. Am J Epidemiol. 2010; 172:517-24.
PubMed
 
Moons KG, Altman DG, Vergouwe Y, Royston P.  Prognosis and prognostic research: application and impact of prognostic models in clinical practice. BMJ. 2009; 338:b606.
PubMed
 
Khoury MJ, Dorman JS.  The Human Genome Epidemiology Network [Editorial]. Am J Epidemiol. 1998; 148:1-3.
PubMed
 
Freedman AN, Seminara D, Gail MH, Hartge P, Colditz GA, Ballard-Barbash R. et al.  Cancer risk prediction models: a workshop on development, evaluation, and application. J Natl Cancer Inst. 2005; 97:715-23.
PubMed
 
Altman DG, Schulz KF, Moher D, Egger M, Davidoff F, Elbourne D, et al. CONSORT GROUP (Consolidated Standards of Reporting Trials).  The revised CONSORT statement for reporting randomized trials: explanation and elaboration. Ann Intern Med. 2001; 134:663-94.
PubMed
 
Bossuyt PM, Reitsma JB, Bruns DE, Gatsonis CA, Glasziou PP, Irwig LM, et al. Standards for Reporting of Diagnostic Accuracy.  The STARD statement for reporting studies of diagnostic accuracy: explanation and elaboration. Ann Intern Med. 2003; 138:W1-12.
PubMed
 
Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC, Ioannidis JP. et al.  The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. PLoS Med. 2009; 6:1000100.
PubMed
 
Vandenbroucke JP, von Elm E, Altman DG, Gøtzsche PC, Mulrow CD, Pocock SJ, et al. STROBE Initiative.  Strengthening the Reporting of Observational Studies in Epidemiology (STROBE): explanation and elaboration. PLoS Med. 2007; 4:297.
PubMed
 
Vandenbroucke JP.  STREGA, STROBE, STARD, SQUIRE, MOOSE, PRISMA, GNOSIS, TREND, ORION, COREQ, QUOROM, REMARK … and CONSORT: for whom does the guideline toll? J Clin Epidemiol. 2009; 62:594-6.
PubMed
 
Cook NR.  Use and misuse of the receiver operating characteristic curve in risk prediction. Circulation. 2007; 115:928-35.
PubMed
 
Pencina MJ, D'Agostino RB Sr, D'Agostino RB Jr, Vasan RS.  Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond. Stat Med. 2008; 27:157-72.
PubMed
 
Janssens AC, Khoury MJ.  Assessment of improved prediction beyond traditional risk factors: when does a difference make a difference? [Editorial]. Circ Cardiovasc Genet. 2010; 3:3-5.
PubMed
 

Letters

NOTE:
Citing articles are presented as examples only. In non-demo SCM6 implementation, integration with CrossRef’s "Cited By" API will populate this tab (http://www.crossref.org/citedby.html).

Comments

Submit a Comment
Submit a Comment

Summary for Patients

Clinical Slide Sets

Terms of Use

The In the Clinic® slide sets are owned and copyrighted by the American College of Physicians (ACP). All text, graphics, trademarks, and other intellectual property incorporated into the slide sets remain the sole and exclusive property of the ACP. The slide sets may be used only by the person who downloads or purchases them and only for the purpose of presenting them during not-for-profit educational activities. Users may incorporate the entire slide set or selected individual slides into their own teaching presentations but may not alter the content of the slides in any way or remove the ACP copyright notice. Users may make print copies for use as hand-outs for the audience the user is personally addressing but may not otherwise reproduce or distribute the slides by any means or media, including but not limited to sending them as e-mail attachments, posting them on Internet or Intranet sites, publishing them in meeting proceedings, or making them available for sale or distribution in any unauthorized form, without the express written permission of the ACP. Unauthorized use of the In the Clinic slide sets will constitute copyright infringement.

Toolkit

Want to Subscribe?

Learn more about subscription options

Advertisement
Related Articles
Journal Club
Topic Collections
PubMed Articles
Forgot your password?
Enter your username and email address. We'll send you a reminder to the email address on record.
(Required)
(Required)