George F. Sawaya, MD; Janelle Guirguis-Blake, MD, MPH; Michael LeFevre, MD, MSPH; Russell Harris, MD, MPH; Diana Petitti, MD, MPH; U.S. Preventive Services Task Force
Sawaya GF, Guirguis-Blake J, LeFevre M, Harris R, Petitti D, U.S. Preventive Services Task Force. Update on the Methods of the U.S. Preventive Services Task Force: Estimating Certainty and Magnitude of Net Benefit. Ann Intern Med. 2007;147:871-875. doi: 10.7326/0003-4819-147-12-200712180-00007
Download citation file:
Published: Ann Intern Med. 2007;147(12):871-875.
The major goal of the U.S. Preventive Services Task Force (USPSTF) is to provide a reliable and accurate source of evidence-based recommendations on a wide range of preventive services. In this article, the USPSTF updates and reviews the process by which it evaluates evidence, determines the certainty and magnitude of net benefit, and gives a final letter grade to recommendations. Because direct evidence about prevention is often unavailable, the Task Force usually considers indirect evidence. To guide its selection of indirect evidence, a â€œchain of evidenceâ€ is constructed within an analytic framework. The Task Force examines evidence of various research designs that addresses the key questions within the framework. New terms have been added to describe the USPSTF's judgment about the evidence for each key question: â€œconvincing,â€ â€œadequate,â€ or â€œinadequate.â€ For increased clarity, the USPSTF has changed its description of overall evidence of net benefit for the preventive service from â€œgood,â€ â€œfair,â€ or â€œpoorâ€ quality to â€œhigh,â€ â€œmoderate,â€ or â€œlowâ€ certainty. This rating considers the extent to which an uninterrupted chain of evidence exists across the analytic framework. Individual studies will continue to be judged as being of â€œgood,â€ â€œfair,â€ or â€œpoorâ€ quality. Using outcomes tables, the USPSTF estimates the magnitude of benefits and the magnitude of harms, and synthesizes them into an estimate of the magnitude of net benefit. Although some judgment is required at all steps, the USPSTF strives to make the process as explicit and transparent as possible. The USPSTF anticipates that its methods for making evidence-based recommendations will continue to evolve.
The In the Clinic® slide sets are owned and copyrighted by the American College of Physicians (ACP). All text, graphics, trademarks, and other intellectual property incorporated into the slide sets remain the sole and exclusive property of the ACP. The slide sets may be used only by the person who downloads or purchases them and only for the purpose of presenting them during not-for-profit educational activities. Users may incorporate the entire slide set or selected individual slides into their own teaching presentations but may not alter the content of the slides in any way or remove the ACP copyright notice. Users may make print copies for use as hand-outs for the audience the user is personally addressing but may not otherwise reproduce or distribute the slides by any means or media, including but not limited to sending them as e-mail attachments, posting them on Internet or Intranet sites, publishing them in meeting proceedings, or making them available for sale or distribution in any unauthorized form, without the express written permission of the ACP. Unauthorized use of the In the Clinic slide sets will constitute copyright infringement.
David L. Hahn
Dean Medical Center, Madison WI
January 13, 2008
USPSTF "B" recommendations and the "chain of evidence" method
Clinicians and policymakers rely on the United States Preventive Service Task Force (USPSTF) for reliable and accurate information on preventive services. In their recent update on USPSTF methods, Sawaya et al. (1) described USPSTF criteria for constructing and evaluating an indirect "chain of evidence" for a preventive service when direct evidence from one or more randomized effectiveness studies is unavailable. They also stated that they anticipated that their methods for making evidence- based recommendations would continue to evolve. I have a comment about the "chain of evidence" approach, and a suggestion for the next step in the evolution of the recommendation process.
Positive recommendations based on indirect "chains of evidence" in the absence of confirmatory effectiveness trials is analogous to basing therapy on surrogate outcomes (e.g., blood test results, histopathologic findings, etc) instead of on patient-oriented outcomes (e.g., morbidity, mortality, quality-of-life). This is an important distinction because basing interventions on surrogates can result in unexpected net harm because the mechanistic hypotheses (and by extension the "chain of evidence" assumptions) may often be incomplete or incorrect (2).
I am not suggesting that the USPSTF should abandon its "chain of evidence" approach, but I do suggest that the USPSTF should acknowledge that it may be risky to base positive recommendations on this method without confirmatory evidence from direct clinical effectiveness trials in appropriate settings. Specifically, I suggest that the USPSTF consider revising the "B" recommendations (Recommendation Grid, Table 1) (1). High certainty evidence (i.e., direct evidence) with moderate magnitude of net benefit might continue to receive a "B" recommendation. Moderate certainty evidence (i.e., indirect evidence) with moderate magnitude might be reclassified to indicate the possibility of unanticipated harm. Perhaps this category could be changed from "recommend" to "consider" accompanied by the appropriate caveats.
I have spoken with policymakers who believe that the public, the press, patients and many physicians do not distinguish the sometimes very significant differences in certainty and magnitude of net benefit between "A" and "B" rated USPSTF recommendations. Refining the Recommendation Grid to better communicate these differences could contribute to enhancing the delivery of high value clinical preventive services.
1. Sawaya GF, Guirguis-Blake J, LeFevre M, Harris R, Petitti D. Update on the methods of the U.S. Preventive Services Task Force: Estimating certainty and magnitude of benefit. Ann Intern Med 2007; 147:871-5.
2. Ebell MH, Siwek J, Weiss B, Woolf SH, Susman J, Ewigman B, et al. Strength of Recommendation Taxonomy (SORT): A patient-centered approach to grading evidence in the medical literature. J Am Board Fam Pract 2004; 17:59-67.
Mary B Barton
Agency for Healthcare Research and Quality
February 26, 2008
Response to Dr. Hahn on USPSTF methods
We appreciate Dr. Hahn's thoughtful consideration of the potential distinctions between, and within, USPSTF recommendation grade letters, and the role of certainty in arriving at the recommendation grade.
His suggestion that the USPSTF never assign any grade higher than "B" to a recommendation based on the "chain of evidence" approach (in fact, he proposes a new "B minus" category for any positive recommendation made on the basis of indirect evidence) implies that only evidence from randomized trials merits a judgment that "certainty of net benefit is high." In at least some cases, randomized trials of preventive services are not now necessary to draw this conclusion. Support for smoking cessation counseling in primary care is, for example, a high certainty of net benefit, "grade A," recommendation. It is based on the "chain of evidence" approach"”counseling results in smoking cessation"”smoking cessation prevents lung cancer"”prevention of lung cancer increases longevity.
The USPSTF also recognizes that the randomized trial as evidence standard is unattainable for many clinical preventive services. It is not feasible, for example, to conduct a randomized trial of the effect of HIV screening in pregnant women on childhood morbidity and mortality for ethical reasons and because the sample size and cost of such a trial is prohibitive. The USPSTF rated its recommendations for universal HIV screening in pregnancy as a high certainty of net benefit, "A," recommendation using the "chain of evidence" approach"”screening detects HIV infection in the mother"”treatment of the mother during pregnancy prevents HIV transmission to the newborn"”prevention of newborn HIV infection prevents HIV morbidity and mortality.
The USPSTF is not, however, in any way indifferent to Dr. Hahn's concern about the use of surrogate outcomes instead of patient-oriented outcomes as the desirable basis for recommending interventions, including preventive interventions. If randomized trials are necessary to draw a conclusion that has high certainty, and if the trials are feasible, they should be conducted and conclusions about use of the intervention delayed pending results.
The USPSTF appreciates that much of its intended audience distinguishes little between grade A and grade B recommendations âˆ’ both are recommended for use in routine practice. As a potential guide to policy makers, those who design quality metrics, or even individual clinicians who may use the distinction between A and B recommendations for prioritization purposes, the USPSTF will continue to use both, and reserve the grade A recommendation for those services with the most ironclad evidence, and therefore high certainty, about the impact of the service on health outcomes.
Ned Calonge, MD, MPH, Chair, USPSTF Diana B. Petitti, MD, MPH, Vice Chair, USPSTF Mary B. Barton, MD, MPP, Scientific Director, USPSTF
Results provided by:
Copyright © 2016 American College of Physicians. All Rights Reserved.
Print ISSN: 0003-4819 | Online ISSN: 1539-3704
Conditions of Use
This PDF is available to Subscribers Only