0

The full content of Annals is available to subscribers

Subscribe/Learn More  >
Research and Reporting Methods |

Models in the Development of Clinical Practice GuidelinesModels in the Development of Clinical Practice Guidelines

J. Dik F. Habbema, PhD; Timothy J. Wilt, MD, MPH; Ruth Etzioni, PhD; Heidi D. Nelson, MD, MPH; Clyde B. Schechter, MD, MA; William F. Lawrence, MD, MS; Joy Melnikow, MD, MPH; Karen M. Kuntz, ScD; Douglas K. Owens, MD, MS; and Eric J. Feuer, PhD
[+] Article, Author, and Disclosure Information

From Erasmus MC University Medical Center, Rotterdam, the Netherlands; University of Minnesota School of Medicine, Minneapolis, Minnesota; Fred Hutchinson Cancer Research Center, Seattle, Washington; Oregon Health & Science University, Portland, Oregon; Albert Einstein College of Medicine, New York, New York; Agency for Healthcare Research and Quality, Rockville, Maryland; UC Davis Medical Center, Sacramento, California; Stanford University, Stanford, California; and National Cancer Institute, Bethesda, Maryland.

Disclaimer: This paper reflects the opinions of the authors only and not of guideline development groups in which they are involved; organizations with which they are affiliated; or the Agency for Healthcare Research and Quality, the National Cancer Institute, the U.S. Department of Health and Human Services, or the U.S. Department of Veterans Affairs.

Acknowledgment: The authors thank Therese Miller for her valuable comments.

Financial Support: Dr. Wilt is supported by grants from the U.S. Department of Veterans Affairs and the Agency for Healthcare Research and Quality and contracts to conduct evidence synthesis from the American College of Physicians, the National Kidney Foundation, and Kidney Diseases International. Dr. Nelson is supported by the Agency for Healthcare Research and Quality and the National Institutes of Health. Dr. Schechter is supported by grants from the National Cancer Institute to develop and apply models of breast cancer epidemiology. Dr. Melnikow receives extramural support from the California Health Benefits Review Program and the National Institutes of Health. Dr. Owens is supported by the U.S. Department of Veterans Affairs.

Disclosures: Disclosures can be viewed at www.acponline.org/authors/icmje/ConflictOfInterestForms.do?msNum=M14-0845.

Corresponding Author: J. Dik F. Habbema, PhD, Department of Public Health, Erasmus MC University Medical Center, Wytemaweg 80, 3015 CN, Rotterdam, the Netherlands; e-mail, j.d.f.habbema@erasmusmc.nl.

Current Author Addresses: Dr. Habbema: Department of Public Health, Erasmus MC University Medical Center, Wytemaweg 80, 3015 CN, Rotterdam, the Netherlands.

Dr. Wilt: Minneapolis VA Center for Chronic Disease Outcomes Research and the University of Minnesota School of Medicine, 1 Veterans Drive (111-0), Minneapolis, MN 55417.

Dr. Etzioni: Fred Hutchinson Cancer Research Center, 1100 Fairview Avenue North, M2-B230, PO Box 19024, Seattle, WA 98109-1024.

Dr. Nelson: Pacific Northwest Evidence-based Practice Center, Oregon Health & Science University, Mailcode BICC, 3181 SW Sam Jackson Park Road, Portland, OR 97239.

Dr. Schechter: Department of Family & Social Medicine, Albert Einstein College of Medicine, 1300 Morris Park Avenue, Bronx, NY 10461.

Dr. Lawrence: Agency for Healthcare Research and Quality, 540 Gaither Road, Rockville, MD 20850.

Dr. Melnikow: Center for Healthcare Policy and Research, University of California, Davis, 4860 Y Street, Sacramento, CA 95817.

Dr. Kuntz: School of Public Health, University of Minnesota, 420 Delaware Street SE, D360, Mayo MMC 729, Minneapolis, MN 55455.

Dr. Owens: Center for Health Policy/Center for Primary Care and Outcomes Research, Stanford University, 117 Encina Commons, Stanford, CA 94305.

Dr. Feuer: Division of Cancer Control and Population Sciences, National Cancer Institute, 9609 Medical Center Drive, Bethesda, MD 20892.

Author Contributions: Conception and design: J.D.F. Habbema, R. Etzioni, H.D. Nelson, W.F. Lawrence, J. Melnikow, K.M. Kuntz, D.K. Owens, E.J. Feuer.

Analysis and interpretation of the data: J.D.F. Habbema, T.J. Wilt, R. Etzioni, C.B. Schechter, J. Melnikow, K.M. Kuntz, D.K. Owens, E.J. Feuer.

Drafting of the article: J.D.F. Habbema, T.J. Wilt, R. Etzioni, H.D. Nelson, C.B. Schechter, J. Melnikow, D.K. Owens, E.J. Feuer.

Critical revision of the article for important intellectual content: J.D.F. Habbema, T.J. Wilt, R. Etzioni, H.D. Nelson, C.B. Schechter, W.F. Lawrence, J. Melnikow, K.M. Kuntz, D.K. Owens, E.J. Feuer.

Final approval of the article: J.D.F. Habbema, R. Etzioni, H.D. Nelson, C.B. Schechter, W.F. Lawrence, J. Melnikow, K.M. Kuntz, D.K. Owens, E.J. Feuer.

Provision of study materials or patients: C.B. Schechter.

Statistical expertise: R. Etzioni, C.B. Schechter, K.M. Kuntz.

Administrative, technical, or logistic support: E.J. Feuer.

Collection and assembly of data: J.D.F. Habbema, R. Etzioni.


Ann Intern Med. 2014;161(11):812-818. doi:10.7326/M14-0845
Text Size: A A A

Clinical practice guidelines should be based on the best scientific evidence derived from systematic reviews of primary research. However, these studies often do not provide evidence needed by guideline development groups to evaluate the tradeoffs between benefits and harms. In this article, the authors identify 4 areas where models can bridge the gaps between published evidence and the information needed for guideline development applying new or updated information on disease risk, diagnostic test properties, and treatment efficacy; exploring a more complete array of alternative intervention strategies; assessing benefits and harms over a lifetime horizon; and projecting outcomes for the conditions for which the guideline is intended. The use of modeling as an approach to bridge these gaps (provided that the models are high-quality and adequately validated) is considered. Colorectal and breast cancer screening are used as examples to show the utility of models for these purposes. The authors propose that a modeling study is most useful when strong primary evidence is available to inform the model but critical gaps remain between the evidence and the questions that the guideline group must address. In these cases, model results have a place alongside the findings of systematic reviews to inform health care practice and policy.

Figures

Grahic Jump Location
Figure.

Illustration of the inverse relationship between the credibility of a modeled result and the required amount of extrapolation from primary data sources.

Examples A, B, C, D, and G concern mortality reduction and/or life-years gained as the primary modeled outcomes. For examples E and F, the primary modeled outcomes are incidental findings and overdiagnosis, respectively. Examples A and B require relatively little extrapolation from primary data, whereas example E requires extensive extrapolation. The other examples require an intermediate amount of extrapolation. CT = computed tomography; DCIS = ductal carcinoma in situ; FOBT = fecal occult blood test; gFOBT = guaiac fecal occult blood test.

Grahic Jump Location

Tables

References

Letters

NOTE:
Citing articles are presented as examples only. In non-demo SCM6 implementation, integration with CrossRef’s "Cited By" API will populate this tab (http://www.crossref.org/citedby.html).

Comments

Submit a Comment/Letter
Models in the Development of Clinical Practice Guidelines
Posted on December 30, 2014
Ioannis M. Zacharioudakis, Fainareti N. Zervou, Eleftherios Mylonakis
Infectious Diseases Division, Warren Alpert Medical School of Brown University, Providence, RI
Conflict of Interest: None Declared
Habbema and colleagues have presented a particularly interesting article regarding the position of models in clinical practice guidelines (1). Indeed models can bridge the gap between direct evidence provided from randomized controlled trials (RCTs) and/or observational studies and questions raised in daily practice.
In order to estimate the degree of utilization of models in the development of current guidelines, we used the list of the 100 most cited guidelines of the National Guideline Clearinghouse (http://www.guideline.gov) that we had identified for the purposes of a previous study of ours. The number of citations was extracted from the ISI Web of Science. The search of the most cited guidelines was done to select those most influential in clinical practice and research development. Using the results of that search we performed a secondary analysis with major outcome the degree of incorporation of mathematical models in guidelines justification. Surprisingly, we found that 54% of these guidelines use conclusions derived from modeling studies to support their recommendations.
Given this observed magnitude of models in the development of practice guidelines and their expected growing use in the future, a need for an objective, to the extent possible, assessment of their merits, in terms of internal validity and reproducibility, raises. Habbema and colleagues mention reasonable criteria for the assessment of the credibility of models. However, so far, consensus statements have mostly focused on the quality of presentation and reporting, such as the recently published CHEERS checklist for cost-effectiveness studies (2); while statements for the assessment of the internal validity of the reporting outcomes, similar to the Cochrane collaboration tool for RCTs (3), or the Newcastle-Ottawa scale for observational studies (4), are currently lacking.
Furthermore, given that the model upon which a guideline is based should be calibrated to every meaningful measurement that is published, there might be a need for a systematic and predefined verification and update of included models at fixed intervals separately from the overall assessment of the guidelines.
In summary, more than 50% of the examined guidelines of the National Guideline Clearinghouse included results derived from mathematical models to support their recommendations. Admittedly, models can be of significant help in guiding daily clinical practice regarding questions that cannot be answered directly by conventional RCTS and/ or observational studies. However, an objective assessment of their internal validity should be established. Otherwise we risk undermining the high quality data provided by RCTs.


REFERENCES

1. Habbema JD, Wilt TJ, Etzioni R, Nelson HD, Schechter CB, Lawrence WF, et al. Models in the development of clinical practice guidelines. Ann Intern Med. 2014;161:812-8.
2. Husereau D, Drummond M, Petrou S, Carswell C, Moher D, Greenberg D, et al. Consolidated Health Economic Evaluation Reporting Standards (CHEERS) statement. BMJ. 2013;346:f1049.
3. Higgins JP, Altman DG, Gotzsche PC, Juni P, Moher D, Oxman AD, et al. The Cochrane Collaboration's tool for assessing risk of bias in randomised trials. BMJ. 2011;343:d5928.
4. Wells GA SB, O’Connell D, Peterson J, Welch V, Losos M. he Newcastle-Ottawa Scale (NOS) for assessing the quality of nonrandomised studies in meta-analyses.
Submit a Comment/Letter

Summary for Patients

Clinical Slide Sets

Terms of Use

The In the Clinic® slide sets are owned and copyrighted by the American College of Physicians (ACP). All text, graphics, trademarks, and other intellectual property incorporated into the slide sets remain the sole and exclusive property of the ACP. The slide sets may be used only by the person who downloads or purchases them and only for the purpose of presenting them during not-for-profit educational activities. Users may incorporate the entire slide set or selected individual slides into their own teaching presentations but may not alter the content of the slides in any way or remove the ACP copyright notice. Users may make print copies for use as hand-outs for the audience the user is personally addressing but may not otherwise reproduce or distribute the slides by any means or media, including but not limited to sending them as e-mail attachments, posting them on Internet or Intranet sites, publishing them in meeting proceedings, or making them available for sale or distribution in any unauthorized form, without the express written permission of the ACP. Unauthorized use of the In the Clinic slide sets will constitute copyright infringement.

Toolkit

Buy Now for $32.00

to gain full access to the content and tools.

Want to Subscribe?

Learn more about subscription options

Advertisement
Related Articles
Related Point of Care
Topic Collections
PubMed Articles
Forgot your password?
Enter your username and email address. We'll send you a reminder to the email address on record.
(Required)
(Required)