Diana B. Petitti, MD, MPH; Steven M. Teutsch, MD; Mary B. Barton, MD, MPP; George F. Sawaya, MD; Judith K. Ockene, PhD, MEd; Thomas DeWitt, MD; on behalf of the U.S. Preventive Services Task Force
Potential Financial Conflicts of Interest: None disclosed.
Requests for Single Reprints: Reprints are available on the AHRQ Web site (http://www.preventiveservices.ahrq.gov).
Current Author Addresses: Dr. Petitti: Department of Biomedical Informatics, Arizona State University, Phoenix, AZ 85041.
Dr. Teutsch: Outcomes Research, Merck & Co., West Point, PA 19486.
Dr. Barton: Center for Primary Care, Prevention and Clinical Partnerships, Agency for Healthcare Research and Quality, Rockville, MD 20850.
Dr. Sawaya: Department of Obstetrics, Gynecology and Reproductive Science, University of California, San Francisco, San Francisco, CA 94143.
Dr. Ockene: Department of Medicine, University of Massachusetts Medical School, Worcester, MA 01655.
Dr. DeWitt: Department of Pediatrics, Cincinnati Children's, University of Cincinnati College of Medicine, Cincinnati, OH 45229.
Petitti D., Teutsch S., Barton M., Sawaya G., Ockene J., DeWitt T., ; Update on the Methods of the U.S. Preventive Services Task Force: Insufficient Evidence. Ann Intern Med. 2009;150:199-205. doi: 10.7326/0003-4819-150-3-200902030-00010
Download citation file:
Published: Ann Intern Med. 2009;150(3):199-205.
The U.S. Preventive Services Task Force (USPSTF) seeks to provide reliable and accurate evidence-based recommendations to primary care clinicians. However, clinicians indicate frustration with the lack of guidance provided by the USPSTF when the evidence is insufficient to make a recommendation. This article describes a new USPSTF plan to commission its Evidence-based Practice Centers to collect information in 4 domains pertinent to clinical decisions about prevention and to report this information routinely. The 4 domains are potential preventable burden, potential harm of the intervention, costs (both monetary and opportunity), and current practice. The process and rationale used to select these domains are presented, along with examples of how clinicians might use the information to guide clinical decision making when evidence is insufficient.
Gordon H Guyatt
McMaster University Health Sciences Center
May 21, 2009
In their article focusing on the "insufficient evidence" category of the USPSTF system, the authors make three statements about the GRADE (www.gradeworkinggroup.org) approach to rating quality of evidence and strength of recommendations that may mislead. We would therefore like to clarify the characteristics of the GRADE approach at issue.
First, the authors state that GRADE has no equivalent to the USPSTF "I statement". The "I statement" accompanies a USPSTF decision not to recommend either in favor or against the intervention because "the current evidence is insufficient to assess the balance of benefits and harms of the service". With GRADE, guideline panels may choose to rate the quality of evidence as low or very low, and decide not to offer a recommendation and to provide clarifying statements. Thus GRADE accommodates the situation in which the USPSTF uses an "I statement".
Second, the authors accurately state that the GRADE approach to recommendations considers the magnitude of the benefits and harms and the values and preferences patients place on those benefits and harms. The authors state that GRADE also takes into account "the importance of the outcome that the treatment prevents, (and) the burden of therapy" as if these were something different than the magnitude of benefits and harms and the relative values placed on them. They are not.
Third, in contrast to what the USPSTF authors suggest, GRADE explicitly takes potential harms and resource use (costs) into account in grading the strength of a recommendation. GRADE considers both the seriousness of a condition and its incidence or prevalence (the two factors that determine burden of disease) as factors that policymakers must consider beyond the strength of a recommendation when deciding on priorities. From the perspective of individual patients, both the severity and the incidence of a condition (and the proportion of that burden of disease that is potentially preventable) are incorporated into estimates of the effect of an intervention and into judgments about the balance between desirable and undesirable effects. This is explicitly addressed in GRADE's approach to determining the strength of a recommendation. As with burden of disease, how widely a service is used is a factor that policymakers must consider when deciding priorities.
We trust that this clarification will prevent misunderstanding by any groups considering joining the over 30 organizations (including UpToDate, the World Health Organization, and the American College of Physicians) currently using GRADE.
1. Petitti, D.B., et al., Update on the methods of the U.S. Preventive Services Task Force: insufficient evidence. Ann Intern Med, 2009. 150(3): p. 199-205.
2. Guyatt, G.H., et al., GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. Bmj, 2008. 336(7650): p. 924-6.
3. Jaeschke, R., et al., Use of GRADE grid to reach decisions on clinical practice guidelines when consensus is elusive. Bmj, 2008. 337: p. a744.
4. Guyatt, G.H., et al., Going from evidence to recommendations. Bmj, 2008. 336(7652): p. 1049-51.
5. Guyatt, G.H., et al., Incorporating considerations of resources use into grading recommendations. Bmj, 2008. 336(7654): p. 1170-3.
Diana B. Petitti
Vice Chair, U.S. Preventive Services Task Force
June 3, 2009
Response: Comparing the U.S. Preventive Services Task Force (USPSTF) and GRADE Approaches to Recomme
Response: Comparing the U.S. Preventive Services Task Force (USPSTF) and GRADE Approaches to Recommendations
The U.S. Preventive Services Task Force (USPSTF) welcomes this opportunity to further explicate the similarities and differences between the GRADE (http://www.gradeworkinggroup.org> ) and USPSTF approaches to making recommendations.
The GRADE Working Group clarifies that guideline panels using its approach may decide not to offer a recommendation and to provide clarifying statements. We would agree that GRADE thus accommodates the situation in which the USPSTF uses an "I statement."
However, the USPSTF "rules of evidence" dictate that an "I statement" must always be issued in situations when a letter grade of A, B, C, or D cannot be assigned (1, 2). This occurs when the USPSTF judges that the evidence about health benefits and harms does not permit at least moderate certainty that the balance of health benefits and harms is favorable, or that there is no net benefit, or that the harms of the intervention outweigh the benefits (3). The values and preferences individual patients place on benefits and harms are not considered by the USPSTF when deciding whether to assign a letter grade to a recommendation. Also not considered in the decision to assign a letter grade are the cost or cost effectiveness of the service. These are important differences between the USPSTF and the GRADE approaches.
GRADE states that the importance of the outcome the intervention prevents and the burden of the intervention therapy are not different from benefits and harms. The USPSTF agrees that importance and burden could be conceptualized as benefits and harms; we do not agree that it is necessary to conceptualize them as such.
The USPSTF agrees with GRADE's interpretation of the implications of the USPSTF insufficient evidence publication (4) "”that the differences between treatment and diagnosis on the one hand, and prevention on the other, may make GRADE less applicable to prevention decisions. The USPSTF believes that certainty that there is a net health benefit must be higher when asymptomatic people are encouraged by their physician to undergo a preventive intervention than when patients seek help for a problem and are advised to undergo a diagnostic intervention or to partake of a treatment. We would welcome the GRADE Working Group's comment on the implications of our belief that certainty of net benefit should be higher for preventive interventions than for diagnostic interventions and treatment.
The USPSTF did not intend to imply that GRADE does not consider the four issues the USPSTF describes as pertinent to decision making in the face of insufficient evidence. The GRADE Working Group has published widely about its methods, and these publications describe the uses of information on burden of illness, cost, and use in the community (5-12).
The difference between the USPSTF and the GRADE approaches lies in the role that burden of illness, cost, and community use play in moving from evidence about net harms and benefits to suggested actions. For the USPSTF, the assignment of a letter grade to a recommendation carries a suggested action: "Do it routinely" (letter grade A and B); "Don't do it routinely" (letter grade C), or "Don't do it" (letter grade D) (1). Topics not assigned a letter grade"”all topics with an associated "I statement""”do not have a suggested action.
It is our understanding that, in addition to information on the quality of the evidence about net benefits and harms, GRADE could use information on patient preferences, burden of illness, cost, and use in the community to classify a recommendation as weak or strong. It is also our understanding that a GRADE classification as a weak or strong recommendation determines the linkage of the recommendation with suggested actions (5-6).
The difference in the kinds of information that are used to move from evidence about harms and benefits to suggested action is, indeed, a large difference in approach between the USPSTF and GRADE. We believe that the USPSTF approach to insufficient evidence is well suited to the needs of the primary care community, the public, and policy-makers in the United States for topics in prevention. This does not mean that the USPSTF approach is applicable outside the United States or that it is applicable to diagnosis or treatment.
The USPSTF is aware of the important work that GRADE is doing to try to reduce inconsistencies among different approaches to assessing the quality of evidence and moving from evidence to recommended actions. GRADE and the USPSTF use nearly identical criteria to rate the quality of studies that provide information about health benefits and harms (2, 7- 11). The USPSTF and GRADE both attempt to make their methods and processes transparent and to remain scrupulously free of financial conflicts of interests. The USPSTF looks forward to further productive dialogue about and convergence of the methods and processes of the USPSTF, GRADE, and other authoritative groups. We are eager to engage in an ongoing inquiry about how best to rate, review, and use evidence and about the kinds of evidence that should figure in suggestions for action.
From the Methods Workgroup of the U.S. Preventive Services Task Force
1. Barton MB, Miller T, Wolff T, Petitti D, LeFevre M, Sawaya G, et al.; U.S. Preventive Services Task Force. How to read the new recommendation statement: methods update from the U.S. Preventive Services Task Force. Ann Intern Med. 2007;147:123-7.
2. U.S. Preventive Services Task Force Procedure Manual. AHRQ Publication No. 08-05118-EF. Rockville, Maryland: Agency for Healthcare Research and Quality, July 2008. http://www.ahrq.gov/clinic/uspstf08/methods/procmanual.html, accessed May 26, 2009.
3. Sawaya GF, Guirguis-Blake J, LeFevre M, Harris R, Petitti D; U.S. Preventive Services Task Force. Update on the methods of the U.S. Preventive Services Task Force: estimating certainty and magnitude of net benefit. Ann Intern Med. 2007;147:871-5.
4. Petitti DB, Teutsch SM, Barton MB, Sawaya GF, Ockene JK, DeWitt T; U.S. Preventive Services Task Force. Update on the methods of the U.S. Preventive Services Task Force: insufficient evidence. Ann Intern Med. 2009;150:199-205.
5. Guyatt GH, Oxman AD, Kunz R, Jaeschke R, Helfand M, Liberati A, et al.; GRADE Working Group. Incorporating considerations of resources use into grading recommendations. BMJ. 2008;336:1170-3.
6. Guyatt GH, Oxman AD, Kunz R, Falck-Ytter Y, Vist GE, Liberati A, et al.; GRADE Working Group. Going from evidence to recommendations. BMJ. 2008;336:1049-51.
7. Guyatt GH, Oxman AD, Vist GE, Kunz R, Falck-Ytter Y, Alonso-Coello P, et al.; GRADE Working Group. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ. 2008;336:924-6.
8. Guyatt GH, Oxman AD, Kunz R, Vist GE, Falck-Ytter Y, SchÃ¼nemann HJ; GRADE Working Group. What is "quality of evidence" and why is it important to clinicians?. BMJ. 2008;336:995-8.
9. Atkins D, Briss PA, Eccles M, Flottorp S, Guyatt GH, Harbour RT, et al.; GRADE Working Group. Systems for grading the quality of evidence and the strength of recommendations II: pilot study of a new system. BMC Health Serv Res. 2005;5:25.
10. Atkins D, Eccles M, Flottorp S, Guyatt GH, Henry D, Hill S, et al.; GRADE Working Group. Systems for grading the quality of evidence and the strength of recommendations I: critical appraisal of existing approaches The GRADE Working Group. BMC Health Serv Res. 2004;4:38.
11. Atkins D, Best D, Briss PA, Eccles M, Falck-Ytter Y, Flottorp S, et al.; GRADE Working Group. Grading quality of evidence and strength of recommendations. BMJ. 2004;328:1490.
12. SchÃ¼nemann HJ, Oxman AD, Brozek J, Glasziou P, Jaeschke R, Vist GE, et al.; GRADE Working Group. Grading quality of evidence and strength of recommendations for diagnostic tests and strategies. BMJ. 2008;336:1106- 10.
to gain full access to the content and tools.
Learn more about subscription options.
Register Now for a free account.
Results provided by:
Copyright © 2016 American College of Physicians. All Rights Reserved.
Print ISSN: 0003-4819 | Online ISSN: 1539-3704
Conditions of Use
This PDF is available to Subscribers Only