A Guide to This Supplement FREE

In 1997, the Agency for Healthcare Research and Quality (AHRQ) initiated a network of North American Evidence-based Practice Centers (EPCs). The EPC staff and collaborators have the following assets: clinical expertise; knowledge of research design; critical appraisal skills; knowledge of bibliographic sources and searching techniques; and multiple analytic skills, including meta-analysis, decision analysis, and cost-effectiveness analysis. The Agency charged EPCs to synthesize information important for the effective and efficient practice of medicine and public health. It asked EPCs to help clinicians, providers, and health plans improve quality of health care by conducting state-of-the-art syntheses of scientific information.

The EPCs produce multiple products for many decision makers. The main product is an evidence report or technology assessment that is designed to address the needs of users. The evidence report may serve as the scientific foundation for public- and private-sector organizations to develop tools and strategies for improving the quality of health care services that they provide. Some technology assessments, such as those conducted for the Centers for Medicare & Medicaid Services, give health plans and payers information they need to make informed decisions about covering new and changing medical devices and procedures. Other evidence reports provide organizations, such as the U.S. Preventive Services Task Force, the National Institutes of Health's Consensus Development Conferences, and the American College of Physicians' Clinical Efficacy Assessment Program, with evidence-based systematic reviews that are used to help formulate guidelines or recommendations.

Evidence reports typically tackle several specific questions about topics within such broad areas as adult health, child and adolescent health, maternal health, geriatrics, rehabilitation, dental health, mental health and substance abuse, alternative care, and preventive care. Public- and private-sector organizations nominate topics for evidence reports, and content experts help formulate the questions addressed in the reports. The EPCs compile the report within several months. Drafts are peer reviewed by content experts, representatives of relevant specialty or federal agencies, and AHRQ scientific staff. Revised reports are released on the AHRQ Web site, and parts of the reports deemed most important are often adapted for publication as journal articles.

As of 2004, EPCs had produced more than 100 evidence reports. They learned much from this experience. They worked with many groups to formulate questions relevant to users and decision makers. They learned about specialized bibliographic sources other than MEDLINE, EMBASE, and the Cochrane Library. They used critical appraisal skills to evaluate diverse literature and various types of studies. They sometimes found existing techniques for conducting systematic reviews to be inadequate and had to explore new ways of summarizing or presenting information.

Thus, over the past decade, EPCs accumulated a wealth of knowledge about conducting systematic reviews and compiling evidence reports. In this supplement, they share with readers some of the most difficult methodologic challenges that they faced. They give many examples to illustrate multiple approaches for addressing the challenges, and they make recommendations for future reviewers to improve processes and presentation of reviews and evidence reports.

Between an introductory article that presents the evolution and future of EPCs (1) and a final article that describes dissemination of EPC reports (2) are 9 articles that explicate methodologic challenges faced by reviewers. Articles address a smorgasbord of topics: complementary and alternative medicine (3); diagnostic technologies (4); health care delivery, organization, and financing (5); drug efficacy (6); economic analyses (7); educational interventions (8); assessing harms using observational studies (9); therapeutic devices and procedures (10); and assessment of treatment interventions from nonrandomized studies (11). Each article begins with a short introduction to provide readers relevant background context. Next, authors describe a few methodologic challenges and give examples. The Table in this paper and a puzzle icon at the bottom right-hand corner of the first page of each article highlight the challenges that particular articles address. All articles end with summary recommendations for reviewers or researchers interested in improving the quality of future reviews.

There are many resources to aid people interested in conducting systematic reviews (12 - 17). We hope that this supplement, packed with pragmatic examples from experienced reviewers, is a valuable addition that helps readers, reviewers, teachers, and decision makers become more familiar with the processes and challenges of preparing high-quality systematic reviews.

Mark Helfand, MD, MPH

Portland Veterans Affairs Medical Center and

Oregon Health & Science University

Portland, OR 97239

Sally Morton, PhD

RAND Corporation

Santa Monica, CA 90401

Eliseo Guallar, MD, PhD

Johns Hopkins University

Baltimore, MD 21205

Cynthia Mulrow, MD, MSc

Annals of Internal Medicine

Philadelphia, PA 19106