0
Challenges of Summarizing Better Information for Better Health: The Evidence-based Practice Center Experience |

A Guide to This Supplement FREE

Mark Helfand, MD, MPH; Sally Morton, PhD; Eliseo Guallar, MD, PhD; and Cynthia Mulrow, MD, MSc, Deputy Editor
[+] Article and Author Information

From Portland Veterans Affairs Medical Center and Oregon Health & Science University, Portland, OR 97239; RAND Corporation, Santa Monica, CA 90401; Johns Hopkins University, Baltimore, MD 21205; and Annals of Internal Medicine, Philadelphia, PA 19106.


Potential Financial Conflicts of Interest: Authors of this paper have received funding for Evidence-based Practice Center reports.

Requests for Single Reprints: Customer Service, American College of Physicians, 190 N. Independence Mall West, Philadelphia, PA 19106.

Current Author Addresses: Dr. Helfand: Oregon Health & Science University, Mail code BICC, 3181 SW Sam Jackson Park Road, Portland, OR 97239.

Dr. Morton: RAND Corporation, 1776 Main Street, Santa Monica, CA 90401.

Dr. Guallar: Johns Hopkins University, 2024 East Monument Street, Room 2-639, Baltimore, MD 21205.

Dr. Mulrow: Annals of Internal Medicine, 190 N. Independence Mall West, Philadelphia, PA 19106.


Ann Intern Med. 2005;142(12_Part_2):1033-1034. doi:10.7326/0003-4819-142-12_Part_2-200506211-00001
Text Size: A A A

In 1997, the Agency for Healthcare Research and Quality (AHRQ) initiated a network of North American Evidence-based Practice Centers (EPCs). The EPC staff and collaborators have the following assets: clinical expertise; knowledge of research design; critical appraisal skills; knowledge of bibliographic sources and searching techniques; and multiple analytic skills, including meta-analysis, decision analysis, and cost-effectiveness analysis. The Agency charged EPCs to synthesize information important for the effective and efficient practice of medicine and public health. It asked EPCs to help clinicians, providers, and health plans improve quality of health care by conducting state-of-the-art syntheses of scientific information.

The EPCs produce multiple products for many decision makers. The main product is an evidence report or technology assessment that is designed to address the needs of users. The evidence report may serve as the scientific foundation for public- and private-sector organizations to develop tools and strategies for improving the quality of health care services that they provide. Some technology assessments, such as those conducted for the Centers for Medicare & Medicaid Services, give health plans and payers information they need to make informed decisions about covering new and changing medical devices and procedures. Other evidence reports provide organizations, such as the U.S. Preventive Services Task Force, the National Institutes of Health's Consensus Development Conferences, and the American College of Physicians' Clinical Efficacy Assessment Program, with evidence-based systematic reviews that are used to help formulate guidelines or recommendations.

Evidence reports typically tackle several specific questions about topics within such broad areas as adult health, child and adolescent health, maternal health, geriatrics, rehabilitation, dental health, mental health and substance abuse, alternative care, and preventive care. Public- and private-sector organizations nominate topics for evidence reports, and content experts help formulate the questions addressed in the reports. The EPCs compile the report within several months. Drafts are peer reviewed by content experts, representatives of relevant specialty or federal agencies, and AHRQ scientific staff. Revised reports are released on the AHRQ Web site, and parts of the reports deemed most important are often adapted for publication as journal articles.

As of 2004, EPCs had produced more than 100 evidence reports. They learned much from this experience. They worked with many groups to formulate questions relevant to users and decision makers. They learned about specialized bibliographic sources other than MEDLINE, EMBASE, and the Cochrane Library. They used critical appraisal skills to evaluate diverse literature and various types of studies. They sometimes found existing techniques for conducting systematic reviews to be inadequate and had to explore new ways of summarizing or presenting information.

Thus, over the past decade, EPCs accumulated a wealth of knowledge about conducting systematic reviews and compiling evidence reports. In this supplement, they share with readers some of the most difficult methodologic challenges that they faced. They give many examples to illustrate multiple approaches for addressing the challenges, and they make recommendations for future reviewers to improve processes and presentation of reviews and evidence reports.

Between an introductory article that presents the evolution and future of EPCs (1) and a final article that describes dissemination of EPC reports (2) are 9 articles that explicate methodologic challenges faced by reviewers. Articles address a smorgasbord of topics: complementary and alternative medicine (3); diagnostic technologies (4); health care delivery, organization, and financing (5); drug efficacy (6); economic analyses (7); educational interventions (8); assessing harms using observational studies (9); therapeutic devices and procedures (10); and assessment of treatment interventions from nonrandomized studies (11). Each article begins with a short introduction to provide readers relevant background context. Next, authors describe a few methodologic challenges and give examples. The Table in this paper and a puzzle icon at the bottom right-hand corner of the first page of each article highlight the challenges that particular articles address. All articles end with summary recommendations for reviewers or researchers interested in improving the quality of future reviews.

Table Jump PlaceholderTable.  Challenges in Conducting Systematic Reviews That Are Addressed in the Supplement Articles

There are many resources to aid people interested in conducting systematic reviews (1217). We hope that this supplement, packed with pragmatic examples from experienced reviewers, is a valuable addition that helps readers, reviewers, teachers, and decision makers become more familiar with the processes and challenges of preparing high-quality systematic reviews.

Mark Helfand, MD, MPH

Portland Veterans Affairs Medical Center and

Oregon Health & Science University

Portland, OR 97239

Sally Morton, PhD

RAND Corporation

Santa Monica, CA 90401

Eliseo Guallar, MD, PhD

Johns Hopkins University

Baltimore, MD 21205

Cynthia Mulrow, MD, MSc

Annals of Internal Medicine

Philadelphia, PA 19106

Atkins D, Fink K, Slutsky J.  Better information for better health care: the Evidence-based Practice Center Program and the Agency for Healthcare Research and Quality. Ann Intern Med. 2005; 142:1035-41.
 
Matchar DB, Westermann-Clark EV, McCrory DC, Patwardhan M, Samsa G, Kulasingam S. et al.  Dissemination of Evidence-based Practice Center reports. Ann Intern Med. 2005; 142:1120-5.
 
Shekelle PG, Morton SC, Suttorp MJ, Buscemi N, Friesen C.  Challenges in systematic reviews of complementary and alternative medicine topics. Ann Intern Med. 2005; 142:1042-7.
 
Tatsioni A, Zarin DA, Aronson N, Samson DJ, Flamm CR, Schmid C. et al.  Challenges in systematic reviews of diagnostic technologies. Ann Intern Med. 2005; 142:1048-55.
 
Bravata DM, McDonald KM, Shojania KG, Sundaram V, Owens DK.  Challenges in systematic reviews: synthesis of topics related to the delivery, organization, and financing of health care. Ann Intern Med. 2005; 142:1056-65.
 
Santaguida PL, Helfand M, Raina P.  Challenges in systematic reviews that evaluate drug efficacy or effectiveness. Ann Intern Med. 2005; 142:1066-72.
 
Pignone M, Saha S, Hoerger T, Lohr KN, Teutsch S, Mandelblatt J.  Challenges in systematic reviews of economic analyses. Ann Intern Med. 2005; 142:1073-9.
 
Reed D, Price EG, Windish DM, Wright SM, Gozu A, Hsu EB. et al.  Challenges in systematic reviews of educational intervention studies. Ann Intern Med. 2005; 142:1080-9.
 
Chou R, Helfand M.  Challenges in systematic reviews that assess treatment harms. Ann Intern Med. 2005; 142:1090-9.
 
Hartling L, McAlister FA, Rowe BH, Ezekowitz J, Friesen C, Klassen TP.  Challenges in systematic reviews of therapeutic devices and procedures. Ann Intern Med. 2005; 142:1100-11.
 
Norris SL, Atkins D.  Challenges in using nonrandomized studies in systematic reviews of treatment interventions. Ann Intern Med. 2005; 142:1112-9.
 
Green S, Higgins J, eds.  Cochrane Reviewers' Handbook 4.2.4. In: The Cochrane Library. Chichester, United Kingdom: J Wiley; 2005: Issue 1.
 
Egger M, Davey-Smith G, Altman DG.  Systematic Reviews in Health Care: Meta-analysis in Context. London: BMJ Books; 2001.
 
Glasziou P, Irwig L, Bain C, Colditz G.  Systematic Reviews in Health Care: A Practical Guide. Cambridge, United Kingdom: Cambridge Univ Pr; 2001.
 
. Mulrow C, Cook D Systematic Reviews: Synthesis of Best Evidence for Health Care Decisions. Philadelphia: American College of Physicians; 1998.
 
Eddy DM, Hasselblad V, Schacter R.  Meta-Analysis by the Confidence Profile Method. Boston: Academic Pr; 1992.
 
Hunter JE, Schmidt FL.  Methods of Meta-Anaysis: Correcting Error and Bias in Research Findings. Newbury Park, CA: Sage Publications; 1990.
 

Figures

Tables

Table Jump PlaceholderTable.  Challenges in Conducting Systematic Reviews That Are Addressed in the Supplement Articles

References

Atkins D, Fink K, Slutsky J.  Better information for better health care: the Evidence-based Practice Center Program and the Agency for Healthcare Research and Quality. Ann Intern Med. 2005; 142:1035-41.
 
Matchar DB, Westermann-Clark EV, McCrory DC, Patwardhan M, Samsa G, Kulasingam S. et al.  Dissemination of Evidence-based Practice Center reports. Ann Intern Med. 2005; 142:1120-5.
 
Shekelle PG, Morton SC, Suttorp MJ, Buscemi N, Friesen C.  Challenges in systematic reviews of complementary and alternative medicine topics. Ann Intern Med. 2005; 142:1042-7.
 
Tatsioni A, Zarin DA, Aronson N, Samson DJ, Flamm CR, Schmid C. et al.  Challenges in systematic reviews of diagnostic technologies. Ann Intern Med. 2005; 142:1048-55.
 
Bravata DM, McDonald KM, Shojania KG, Sundaram V, Owens DK.  Challenges in systematic reviews: synthesis of topics related to the delivery, organization, and financing of health care. Ann Intern Med. 2005; 142:1056-65.
 
Santaguida PL, Helfand M, Raina P.  Challenges in systematic reviews that evaluate drug efficacy or effectiveness. Ann Intern Med. 2005; 142:1066-72.
 
Pignone M, Saha S, Hoerger T, Lohr KN, Teutsch S, Mandelblatt J.  Challenges in systematic reviews of economic analyses. Ann Intern Med. 2005; 142:1073-9.
 
Reed D, Price EG, Windish DM, Wright SM, Gozu A, Hsu EB. et al.  Challenges in systematic reviews of educational intervention studies. Ann Intern Med. 2005; 142:1080-9.
 
Chou R, Helfand M.  Challenges in systematic reviews that assess treatment harms. Ann Intern Med. 2005; 142:1090-9.
 
Hartling L, McAlister FA, Rowe BH, Ezekowitz J, Friesen C, Klassen TP.  Challenges in systematic reviews of therapeutic devices and procedures. Ann Intern Med. 2005; 142:1100-11.
 
Norris SL, Atkins D.  Challenges in using nonrandomized studies in systematic reviews of treatment interventions. Ann Intern Med. 2005; 142:1112-9.
 
Green S, Higgins J, eds.  Cochrane Reviewers' Handbook 4.2.4. In: The Cochrane Library. Chichester, United Kingdom: J Wiley; 2005: Issue 1.
 
Egger M, Davey-Smith G, Altman DG.  Systematic Reviews in Health Care: Meta-analysis in Context. London: BMJ Books; 2001.
 
Glasziou P, Irwig L, Bain C, Colditz G.  Systematic Reviews in Health Care: A Practical Guide. Cambridge, United Kingdom: Cambridge Univ Pr; 2001.
 
. Mulrow C, Cook D Systematic Reviews: Synthesis of Best Evidence for Health Care Decisions. Philadelphia: American College of Physicians; 1998.
 
Eddy DM, Hasselblad V, Schacter R.  Meta-Analysis by the Confidence Profile Method. Boston: Academic Pr; 1992.
 
Hunter JE, Schmidt FL.  Methods of Meta-Anaysis: Correcting Error and Bias in Research Findings. Newbury Park, CA: Sage Publications; 1990.
 

Letters

NOTE:
Citing articles are presented as examples only. In non-demo SCM6 implementation, integration with CrossRef’s "Cited By" API will populate this tab (http://www.crossref.org/citedby.html).

Comments

Submit a Comment
Submit a Comment

Summary for Patients

Clinical Slide Sets

Terms of Use

The In the Clinic® slide sets are owned and copyrighted by the American College of Physicians (ACP). All text, graphics, trademarks, and other intellectual property incorporated into the slide sets remain the sole and exclusive property of the ACP. The slide sets may be used only by the person who downloads or purchases them and only for the purpose of presenting them during not-for-profit educational activities. Users may incorporate the entire slide set or selected individual slides into their own teaching presentations but may not alter the content of the slides in any way or remove the ACP copyright notice. Users may make print copies for use as hand-outs for the audience the user is personally addressing but may not otherwise reproduce or distribute the slides by any means or media, including but not limited to sending them as e-mail attachments, posting them on Internet or Intranet sites, publishing them in meeting proceedings, or making them available for sale or distribution in any unauthorized form, without the express written permission of the ACP. Unauthorized use of the In the Clinic slide sets will constitute copyright infringement.

Toolkit

Want to Subscribe?

Learn more about subscription options

Advertisement
Related Articles
Topic Collections
Forgot your password?
Enter your username and email address. We'll send you a reminder to the email address on record.
(Required)
(Required)