0
Academia and the Profession |

The PRISMA Statement for Reporting Systematic Reviews and Meta-Analyses of Studies That Evaluate Health Care Interventions: Explanation and Elaboration FREE

Alessandro Liberati, MD, DrPH; Douglas G. Altman, DSc; Jennifer Tetzlaff, BSc; Cynthia Mulrow, MD, MSc; Peter C. Gøtzsche, MD, DrMedSci, MSc; John P.A. Ioannidis, MD; Mike Clarke, BA, DPhil; P. J. Devereaux, MD, BSc, PhD; Jos Kleijnen, MD, PhD; and David Moher, PhD
[+] Article and Author Information

From Università di Modena e Reggio Emilia, Modena, Italy; Centro Cochrane Italiano, Istituto Ricerche Farmacologiche Mario Negri, Milan, Italy; Centre for Statistics in Medicine, University of Oxford and UK Cochrane Centre, Oxford, United Kingdom; Ottawa Methods Centre, Ottawa Hospital Research Institute, and University of Ottawa, Ottawa, Ontario, Canada; Annals of Internal Medicine, Philadelphia, Pennsylvania; The Nordic Cochrane Centre, Copenhagen, Denmark; University of Ioannina School of Medicine, Ioannina, Greece; School of Nursing and Midwifery, Trinity College, Dublin, Ireland; McMaster University, Hamilton, Ontario, Canada; Kleijnen Systematic Reviews Ltd, York, United Kingdom; School for Public Health and Primary Care (CAPHRI) and University of Maastricht, Maastricht, The Netherlands.


Acknowledgments: The following people contributed to this paper: Doug Altman, DSc, Centre for Statistics in Medicine (Oxford, United Kingdom); Gerd Antes, PhD, University Hospital Freiburg (Freiburg, Germany); David Atkins, MD, MPH, Health Services Research & Development Service, Veterans Health Administration (Washington, DC); Virginia Barbour, MRCP, DPhil, PLoS Medicine (Cambridge, United Kingdom); Nick Barrowman, PhD, Children's Hospital of Eastern Ontario (Ottawa, Ontario, Canada); Jesse A. Berlin, ScD, Johnson & Johnson Pharmaceutical Research and Development (Titusville, New Jersey); Jocalyn Clark, PhD, PLoS Medicine (at the time of writing, BMJ; London, United Kingdom); Mike Clarke, PhD, UK Cochrane Centre (Oxford, United Kingdom), and School of Nursing and Midwifery, Trinity College (Dublin, Ireland); Deborah Cook, MD, Departments of Medicine, Clinical Epidemiology and Biostatistics, McMaster University (Hamilton, Ontario, Canada); Roberto D'Amico, PhD, Università di Modena e Reggio Emilia (Modena, Italy) and Centro Cochrane Italiano, Istituto Ricerche Farmacologiche Mario Negri (Milan, Italy); Jonathan J. Deeks, PhD, University of Birmingham (Birmingham, United Kingdom); P.J. Devereaux, MD, PhD, Departments of Medicine, Clinical Epidemiology and Biostatistics, McMaster University (Hamilton, Ontario, Canada); Kay Dickersin, PhD, Johns Hopkins Bloomberg School of Public Health (Baltimore, Maryland); Matthias Egger, MD, Department of Social and Preventive Medicine, University of Bern (Bern, Switzerland); Edzard Ernst, MD, PhD, FRCP, FRCP(Edin), Peninsula Medical School (Exeter, United Kingdom); Peter C. Gøtzsche, MD, MSc, The Nordic Cochrane Centre (Copenhagen, Denmark); Jeremy Grimshaw, MBChB, PhD, FRCFP, Ottawa Hospital Research Institute (Ottawa, Ontario, Canada); Gordon Guyatt, MD, Departments of Medicine, Clinical Epidemiology and Biostatistics, McMaster University (Hamilton, Ontario, Canada); Julian Higgins, PhD, MRC Biostatistics Unit (Cambridge, United Kingdom); John P.A. Ioannidis, MD, University of Ioannina Campus (Ioannina, Greece); Jos Kleijnen, MD, PhD, Kleijnen Systematic Reviews Ltd (York, United Kingdom), and School for Public Health and Primary Care (CAPHRI), University of Maastricht (Maastricht, the Netherlands); Tom Lang, MA, Tom Lang Communications and Training (Davis, California); Alessandro Liberati, MD, Università di Modena e Reggio Emilia (Modena, Italy), and Centro Cochrane Italiano, Istituto Ricerche Farmacologiche Mario Negri (Milan, Italy); Nicola Magrini, MD, NHS Centre for the Evaluation of the Effectiveness of Health Care–CeVEAS (Modena, Italy); David McNamee, PhD, The Lancet (London, United Kingdom); Lorenzo Moja, MD, MSc, Centro Cochrane Italiano, Istituto Ricerche Farmacologiche Mario Negri (Milan, Italy); David Moher, PhD, Ottawa Methods Centre, Ottawa Hospital Research Institute (Ottawa, Ontario, Canada); Cynthia Mulrow, MD, MSc, Annals of Internal Medicine (Philadelphia, Pennsylvania); Maryann Napoli, Center for Medical Consumers (New York, New York); Andy Oxman, MD, Norwegian Health Services Research Centre (Oslo, Norway); Ba' Pham, MMath, Toronto Health Economics and Technology Assessment Collaborative (Toronto, Ontario, Canada; at the time of the first meeting of the group, GlaxoSmithKline Canada [Mississauga, Ontario, Canada]); Drummond Rennie, MD, FRCP, FACP, University of California, San Francisco (San Francisco, California); Margaret Sampson, MLIS, Children's Hospital of Eastern Ontario (Ottawa, Ontario, Canada); Kenneth F. Schulz, PhD, MBA, Family Health International (Durham, North Carolina); Paul G. Shekelle, MD, PhD, Southern California Evidence-Based Practice Center (Santa Monica, California); Jennifer Tetzlaff, BSc, Ottawa Methods Centre, Ottawa Hospital Research Institute (Ottawa, Ontario, Canada); David Tovey, FRCGP, The Cochrane Library, Cochrane Collaboration (Oxford, United Kingdom; at the time of the first meeting of the group, BMJ [London, United Kingdom]); and Peter Tugwell, MD, MSc, FRCPC, Institute of Population Health (Ottawa, Ontario, Canada). Dr. Lorenzo Moja helped with the preparation and the several updates of the manuscript and assisted with the preparation of the reference list. Dr. Liberati is the guarantor of the manuscript.

Grant Support: PRISMA was funded by the Canadian Institutes of Health Research; Università di Modena e Reggio Emilia, Italy; Cancer Research UK; Clinical Evidence BMJ Knowledge; The Cochrane Collaboration; and GlaxoSmithKline, Canada. Dr. Liberati is funded, in part, through grants of the Italian Ministry of University (COFIN-PRIN 2002 prot. 2002061749 and COFIN-PRIN 2006 prot. 2006062298). Dr. Altman is funded by Cancer Research UK. Dr. Moher is funded by a University of Ottawa Research Chair. None of the sponsors had any involvement in the planning, execution, or write-up of the PRISMA documents. Additionally, no funder played a role in drafting the manuscript.

Potential Financial Conflicts of Interest:Employment: M. Clarke (His employment is as Director of the UK Cochrane Centre. He is employed by the Oxford Radcliffe Hospitals Trust on behalf of the Department of Health and the National Institute for Health Research in England. This is a fixed-term contract, the renewal of which is dependent upon the value placed upon his work, that of the UK Cochrane Centre and of The Cochrane Collaboration more widely by the Department of Health. His work involves the conduct of systematic reviews and the support of the conduct and use of systematic reviews. Therefore, work—such as this manuscript—relating to systematic reviews might have an impact on his employment).

Corresponding Author: Alessandro Liberati, MD Università di Modena e Reggio Emilia, Centro Cochrane Italiano, Istituto Ricerche Farmacologiche Mario Negri, Via La Masa 19, 20156 Milan, Italy; e-mail, alesslib@mailbase.it.

Current Author Addresses: Dr. Liberati: Università di Modena e Reggio Emilia, Centro Cochrane Italiano, Istituto Ricerche Farmacologiche Mario Negri, Via La Masa 19, 20156 Milan, Italy.

Dr. Altman: Centre for Statistics in Medicine, University of Oxford, Wolfson College Annexe, Linton Road, Oxford OX2 6UD, United Kingdom.

Ms. Tetzlaff and Dr. Moher: Ottawa Methods Centre, Ottawa Hospital Research Institute, The Ottawa Hospital, General Campus, Critical Care Wing (Eye Institute), 6th Floor, 501 Smyth Road, Ottawa, Ontario K1H 8L6, Canada.

Dr. Mulrow: Annals of Internal Medicine, 190 N. Independence Mall West, Philadelphia, PA 19106.

Dr. Gøtzsche: Nordic Cochrane Centre, Rigshospitàlet, Dept 3343, Blegdamsvej 9, DK-2100 Copenhagen, Denmark.

Dr. Ioannidis: Department of Hygiene and Epidemiology, University of Ioannina School of Medicine, University Campus, Ioannina 45110, Greece.

Dr. Clarke: School of Nursing and Midwifery, Trinity College, 24 D'Olier Street, Dublin 2, Ireland.

Dr. Devereaux: Clinical Epidemiology and Biostatistics, McMaster University Health Sciences Centre, Room 2C8 1200 Main Street West, Hamilton, Ontario L8N 3Z5, Canada.

Dr. Kleijnen: Kleijnen Systematic Reviews Ltd, Westminister Business Centre, 10 Great North Way, Nether Poppleton, York YO26 6RB, United Kingdom.


Ann Intern Med. 2009;151(4):W-65-W-94. doi:10.7326/0003-4819-151-4-200908180-00136
Text Size: A A A

Systematic reviews and meta-analyses are essential to summarize evidence relating to efficacy and safety of health care interventions accurately and reliably. The clarity and transparency of these reports, however, is not optimal. Poor reporting of systematic reviews diminishes their value to clinicians, policy makers, and other users.

Since the development of the QUOROM (QUality Of Reporting Of Meta-analysis) Statement—a reporting guideline published in 1999—there have been several conceptual, methodological, and practical advances regarding the conduct and reporting of systematic reviews and meta-analyses. Also, reviews of published systematic reviews have found that key information about these studies is often poorly reported. Realizing these issues, an international group that included experienced authors and methodologists developed PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analyses) as an evolution of the original QUOROM guideline for systematic reviews and meta-analyses of evaluations of health care interventions.

The PRISMA Statement consists of a 27-item checklist and a four-phase flow diagram. The checklist includes items deemed essential for transparent reporting of a systematic review. In this Explanation and Elaboration document, we explain the meaning and rationale for each checklist item. For each item, we include an example of good reporting and, where possible, references to relevant empirical studies and methodological literature. The PRISMA Statement, this document, and the associated Web site (www.prisma-statement.org) should be helpful resources to improve reporting of systematic reviews and meta-analyses.

Editor's Note: In order to encourage dissemination of the PRISMA explanatory paper, this article is freely accessible on the Annals of Internal Medicine, PLoS Medicine, and BMJ Web sites. The authors jointly hold the copyright of this article. For details on further use, see the PRISMA Web site (www.prisma-statement.org).

Systematic reviews and meta-analyses are essential tools for summarizing evidence accurately and reliably. They help clinicians keep up-to-date; provide evidence for policy makers to judge risks, benefits, and harms of health care behaviors and interventions; gather together and summarize related research for patients and their carers; provide a starting point for clinical practice guideline developers; provide summaries of previous research for funders wishing to support new research (1); and help editors judge the merits of publishing reports of new studies (2). Recent data suggest that at least 2,500 new systematic reviews reported in English are indexed in MEDLINE annually (3).

Unfortunately, there is considerable evidence that key information is often poorly reported in systematic reviews, thus diminishing their potential usefulness (36). As is true for all research, systematic reviews should be reported fully and transparently to allow readers to assess the strengths and weaknesses of the investigation (7). That rationale led to the development of the QUOROM (QUality Of Reporting Of Meta-analysis) Statement; those detailed reporting recommendations were published in 1999 (8). In this paper we describe the updating of that guidance. Our aim is to ensure clear presentation of what was planned, done, and found in a systematic review.

Terminology used to describe systematic reviews and meta-analyses has evolved over time and varies across different groups of researchers and authors (see Box 1). In this document we adopt the definitions used by the Cochrane Collaboration (9). A systematic review attempts to collate all empirical evidence that fits pre-specified eligibility criteria to answer a specific research question. It uses explicit, systematic methods that are selected to minimize bias, thus providing reliable findings from which conclusions can be drawn and decisions made. Meta-analysis is the use of statistical methods to summarize and combine the results of independent studies. Many systematic reviews contain meta-analyses, but not all.

The QUOROM Statement, developed in 1996 and published in 1999 (8), was conceived as a reporting guidance for authors reporting a meta-analysis of randomized trials. Since then, much has happened. First, knowledge about the conduct and reporting of systematic reviews has expanded considerably. For example, The Cochrane Library's Methodology Register (which includes reports of studies relevant to the methods for systematic reviews) now contains more than 11,000 entries (March 2009). Second, there have been many conceptual advances, such as “outcome-level” assessments of the risk of bias (1011), that apply to systematic reviews. Third, authors have increasingly used systematic reviews to summarize evidence other than that provided by randomized trials.

However, despite advances, the quality of the conduct and reporting of systematic reviews remains well short of ideal (36). All of these issues prompted the need for an update and expansion of the QUOROM Statement. Of note, recognizing that the updated statement now addresses the above conceptual and methodological issues and may also have broader applicability than the original QUOROM Statement, we changed the name of the reporting guidance to PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analyses).

The PRISMA Statement was developed by a group of 29 review authors, methodologists, clinicians, medical editors, and consumers (12). They attended a three-day meeting in 2005 and participated in extensive post-meeting electronic correspondence. A consensus process that was informed by evidence, whenever possible, was used to develop a 27-item checklist (Table 1; see also Table S1, for a downloadable template checklist for researchers to re-use) and a four-phase flow diagram (Figure 1; see also Figure S1, for a downloadable template document for researchers to re-use). Items deemed essential for transparent reporting of a systematic review were included in the checklist. The flow diagram originally proposed by QUOROM was also modified to show numbers of identified records, excluded articles, and included studies. After 11 revisions the group approved the checklist, flow diagram, and this explanatory paper.

Table Jump PlaceholderTable 1.  Checklist of Items to Include When Reporting a Systematic Review (With or Without Meta-Analysis)
Grahic Jump Location
Figure 1.
Flow of information through the different phases of a systematic review.
Grahic Jump Location

The PRISMA Statement itself provides further details regarding its background and development (12). This accompanying Explanation and Elaboration document explains the meaning and rationale for each checklist item. A few PRISMA Group participants volunteered to help draft specific items for this document, and four of these (DGA, AL, DM, and JT) met on several occasions to further refine the document, which was circulated and ultimately approved by the larger PRISMA Group.

PRISMA focuses on ways in which authors can ensure the transparent and complete reporting of systematic reviews and meta-analyses. It does not address directly or in a detailed manner the conduct of systematic reviews, for which other guides are available (1316).

We developed the PRISMA Statement and this explanatory document to help authors report a wide array of systematic reviews to assess the benefits and harms of a health care intervention. We consider most of the checklist items relevant when reporting systematic reviews of non-randomized studies assessing the benefits and harms of interventions. However, we recognize that authors who address questions relating to etiology, diagnosis, or prognosis, for example, and who review epidemiological or diagnostic accuracy studies may need to modify or incorporate additional items for their systematic reviews.

We modeled this Explanation and Elaboration document after those prepared for other reporting guidelines (1719). To maximize the benefit of this document, we encourage people to read it in conjunction with the PRISMA Statement (11).

We present each checklist item and follow it with a published exemplar of good reporting for that item. (We edited some examples by removing citations or Web addresses, or by spelling out abbreviations.) We then explain the pertinent issue, the rationale for including the item, and relevant evidence from the literature, whenever possible. No systematic search was carried out to identify exemplars and evidence. We also include seven Boxes that provide a more comprehensive explanation of certain thematic aspects of the methodology and conduct of systematic reviews.

Although we focus on a minimal list of items to consider when reporting a systematic review, we indicate places where additional information is desirable to improve transparency of the review process. We present the items numerically from 1 to 27; however, authors need not address items in this particular order in their reports. Rather, what is important is that the information for each item is given somewhere within the report.

Title and Abstract
Item 1: Title

Identify the report as a systematic review, meta-analysis, or both.

Examples

“Recurrence rates of video-assisted thoracoscopic versus open surgery in the prevention of recurrent pneumothoraces: a systematic review of randomised and non-randomised trials” (20).

“Mortality in randomized trials of antioxidant supplements for primary and secondary prevention: systematic review and meta-analysis” (21).

Explanation

Authors should identify their report as a systematic review or meta-analysis. Terms such as “review” or “overview” do not describe for readers whether the review was systematic or whether a meta-analysis was performed. A recent survey found that 50% of 300 authors did not mention the terms “systematic review” or “meta-analysis” in the title or abstract of their systematic review (3). Although sensitive search strategies have been developed to identify systematic reviews (22), inclusion of the terms systematic review or meta-analysis in the title may improve indexing and identification.

We advise authors to use informative titles that make key information easily accessible to readers. Ideally, a title reflecting the PICOS approach (participants, interventions, comparators, outcomes, and study design) (see Item 11 and Box 2) may help readers as it provides key information about the scope of the review. Specifying the design(s) of the studies included, as shown in the examples, may also help some readers and those searching databases.

Grahic Jump Location
Box 2.
Helping To Develop the Research Question(s): The PICOS Approach.
Grahic Jump Location

Some journals recommend “indicative titles” that indicate the topic matter of the review, while others require declarative titles that give the review's main conclusion. Busy practitioners may prefer to see the conclusion of the review in the title, but declarative titles can oversimplify or exaggerate findings. Thus, many journals and methodologists prefer indicative titles as used in the examples above.

Item 2: Structured Summary

Provide a structured summary including, as applicable: background; objectives; data sources; study eligibility criteria, participants, and interventions; study appraisal and synthesis methods; results; limitations; conclusions and implications of key findings; funding for the systematic review; and systematic review registration number.

Example

Context: The role and dose of oral vitamin D supplementation in nonvertebral fracture prevention have not been well established.

Objective: To estimate the effectiveness of vitamin D supplementation in preventing hip and nonvertebral fractures in older persons.

Data Sources: A systematic review of English and non-English articles using MEDLINE and the Cochrane Controlled Trials Register (15), and EMBASE (15). Additional studies were identified by contacting clinical experts and searching bibliographies and abstracts presented at the American Society for Bone and Mineral Research (14). Search terms included randomized controlled trial (RCT), controlled clinical trial, random allocation, double-blind method, cholecalciferol, ergocalciferol, 25-hydroxyvitamin D, fractures, humans, elderly, falls, and bone density.

Study Selection: Only double-blind RCTs of oral vitamin D supplementation (cholecalciferol, ergocalciferol) with or without calcium supplementation vs calcium supplementation or placebo in older persons (>60 years) that examined hip or nonvertebral fractures were included.

Data Extraction: Independent extraction of articles by 2 authors using predefined data fields, including study quality indicators.

Data Synthesis: All pooled analyses were based on random-effects models. Five RCTs for hip fracture (n = 9294) and 7 RCTs for nonvertebral fracture risk (n = 9820) met our inclusion criteria. All trials used cholecalciferol. Heterogeneity among studies for both hip and nonvertebral fracture prevention was observed, which disappeared after pooling RCTs with low-dose (400 IU/d) and higher-dose vitamin D (700-800 IU/d), separately. A vitamin D dose of 700 to 800 IU/d reduced the relative risk (RR) of hip fracture by 26% (3 RCTs with 5572 persons; pooled RR, 0.74; 95% confidence interval [CI], 0.61-0.88) and any nonvertebral fracture by 23% (5 RCTs with 6098 persons; pooled RR, 0.77; 95% CI, 0.68-0.87) vs calcium or placebo. No significant benefit was observed for RCTs with 400 IU/d vitamin D (2 RCTs with 3722 persons; pooled RR for hip fracture, 1.15; 95% CI, 0.88-1.50; and pooled RR for any nonvertebral fracture, 1.03; 95% CI, 0.86-1.24).

Conclusions: Oral vitamin D supplementation between 700 to 800 IU/d appears to reduce the risk of hip and any nonvertebral fractures in ambulatory or institutionalized elderly persons. An oral vitamin D dose of 400 IU/d is not sufficient for fracture prevention” (23).

Explanation

Abstracts provide key information that enables readers to understand the scope, processes, and findings of a review and to decide whether to read the full report. The abstract may be all that is readily available to a reader, for example, in a bibliographic database. The abstract should present a balanced and realistic assessment of the review's findings that mirrors, albeit briefly, the main text of the report.

We agree with others that the quality of reporting in abstracts presented at conferences and in journal publications needs improvement (2425). While we do not uniformly favor a specific format over another, we generally recommend structured abstracts. Structured abstracts provide readers with a series of headings pertaining to the purpose, conduct, findings, and conclusions of the systematic review being reported (2627). They give readers more complete information and facilitate finding information more easily than unstructured abstracts (2832).

A highly structured abstract of a systematic review could include the following headings: Context (or Background); Objective (or Purpose); Data Sources; Study Selection (or Eligibility Criteria); Study Appraisal and Synthesis Methods (or Data Extraction and Data Synthesis); Results; Limitations; and Conclusions (or Implications). Alternatively, a simpler structure could cover but collapse some of the above headings (e.g., label Study Selection and Study Appraisal as Review Methods) or omit some headings such as Background and Limitations.

In the highly structured abstract mentioned above, authors use the Background heading to set the context for readers and explain the importance of the review question. Under the Objectives heading, they ideally use elements of PICOS (see Box 2) to state the primary objective of the review. Under a Data Sources heading, they summarize sources that were searched, any language or publication type restrictions, and the start and end dates of searches. Study Selection statements then ideally describe who selected studies using what inclusion criteria. Data Extraction Methods statements describe appraisal methods during data abstraction and the methods used to integrate or summarize the data. The Data Synthesis section is where the main results of the review are reported. If the review includes meta-analyses, authors should provide numerical results with confidence intervals for the most important outcomes. Ideally, they should specify the amount of evidence in these analyses (numbers of studies and numbers of participants). Under a Limitations heading, authors might describe the most important weaknesses of included studies as well as limitations of the review process. Then authors should provide clear and balanced Conclusions that are closely linked to the objective and findings of the review. Additionally, it would be helpful if authors included some information about funding for the review. Finally, although protocol registration for systematic reviews is still not common practice, if authors have registered their review or received a registration number, we recommend providing the registration information at the end of the abstract.

Taking all the above considerations into account, the intrinsic tension between the goal of completeness of the abstract and its keeping into the space limit often set by journal editors is recognized as a major challenge.

Introduction
Item 3: Rationale

Describe the rationale for the review in the context of what is already known.

Example

“Reversing the trend of increasing weight for height in children has proven difficult. It is widely accepted that increasing energy expenditure and reducing energy intake form the theoretical basis for management. Therefore, interventions aiming to increase physical activity and improve diet are the foundation of efforts to prevent and treat childhood obesity. Such lifestyle interventions have been supported by recent systematic reviews, as well as by the Canadian Paediatric Society, the Royal College of Paediatrics and Child Health, and the American Academy of Pediatrics. However, these interventions are fraught with poor adherence. Thus, school-based interventions are theoretically appealing because adherence with interventions can be improved. Consequently, many local governments have enacted or are considering policies that mandate increased physical activity in schools, although the effect of such interventions on body composition has not been assessed” (33).

Explanation

Readers need to understand the rationale behind the study and what the systematic review may add to what is already known. Authors should tell readers whether their report is a new systematic review or an update of an existing one. If the review is an update, authors should state reasons for the update, including what has been added to the evidence base since the previous version of the review.

An ideal background or introduction that sets context for readers might include the following. First, authors might define the importance of the review question from different perspectives (e.g., public health, individual patient, or health policy). Second, authors might briefly mention the current state of knowledge and its limitations. As in the above example, information about the effects of several different interventions may be available that helps readers understand why potential relative benefits or harms of particular interventions need review. Third, authors might whet readers' appetites by clearly stating what the review aims to add. They also could discuss the extent to which the limitations of the existing evidence base may be overcome by the review.

Item 4: Objectives

Provide an explicit statement of questions being addressed with reference to participants, interventions, comparisons, outcomes, and study design (PICOS).

Example

“To examine whether topical or intraluminal antibiotics reduce catheter-related bloodstream infection, we reviewed randomized, controlled trials that assessed the efficacy of these antibiotics for primary prophylaxis against catheter-related bloodstream infection and mortality compared with no antibiotic therapy in adults undergoing hemodialysis” (34).

Explanation

The questions being addressed, and the rationale for them, are one of the most critical parts of a systematic review. They should be stated precisely and explicitly so that readers can understand quickly the review's scope and the potential applicability of the review to their interests (35). Framing questions so that they include the following five “PICOS” components may improve the explicitness of review questions: 1) the patient population or disease being addressed (P), 2) the interventions or exposure of interest (I), 3) the comparators (C), 4) the main outcome or endpoint of interest (O), and 5) the study designs chosen (S). For more detail regarding PICOS, see Box 2.

Good review questions may be narrowly focused or broad, depending on the overall objectives of the review. Sometimes broad questions might increase the applicability of the results and facilitate detection of bias, exploratory analyses, and sensitivity analyses (3536). Whether narrowly focused or broad, precisely stated review objectives are critical as they help define other components of the review process such as the eligibility criteria (Item 6) and the search for relevant literature (Items 7 and 8).

Methods
Item 5: Protocol and Registration

Indicate if a review protocol exists, if and where it can be accessed (e.g., Web address) and, if available, provide registration information including the registration number.

Example

“Methods of the analysis and inclusion criteria were specified in advance and documented in a protocol” (37).

Explanation

A protocol is important because it pre-specifies the objectives and methods of the systematic review. For instance, a protocol specifies outcomes of primary interest, how reviewers will extract information about those outcomes, and methods that reviewers might use to quantitatively summarize the outcome data (see Item 13). Having a protocol can help restrict the likelihood of biased post hoc decisions in review methods, such as selective outcome reporting. Several sources provide guidance about elements to include in the protocol for a systematic review (16, 3839). For meta-analyses of individual patient-level data, we advise authors to describe whether a protocol was explicitly designed and whether, when, and how participating collaborators endorsed it (4041).

Authors may modify protocols during the research, and readers should not automatically consider such modifications inappropriate. For example, legitimate modifications may extend the period of searches to include older or newer studies, broaden eligibility criteria that proved too narrow, or add analyses if the primary analyses suggest that additional ones are warranted. Authors should, however, describe the modifications and explain their rationale.

Although worthwhile protocol amendments are common, one must consider the effects that protocol modifications may have on the results of a systematic review, especially if the primary outcome is changed. Bias from selective outcome reporting in randomized trials has been well documented (4243). An examination of 47 Cochrane reviews revealed indirect evidence for possible selective reporting bias for systematic reviews. Almost all (n = 43) contained a major change, such as the addition or deletion of outcomes, between the protocol and the full publication (44). Whether (or to what extent) the changes reflected bias, however, was not clear. For example, it has been rather common not to describe outcomes that were not presented in any of the included studies.

Registration of a systematic review, typically with a protocol and registration number, is not yet common, but some opportunities exist (4546). Registration may possibly reduce the risk of multiple reviews addressing the same question (4548), reduce publication bias, and provide greater transparency when updating systematic reviews. Of note, a survey of systematic reviews indexed in MEDLINE in November 2004 found that reports of protocol use had increased to about 46% (3) from 8% noted in previous surveys (49). The improvement was due mostly to Cochrane reviews, which, by requirement, have a published protocol (3).

Item 6: Eligibility Criteria

Specify study characteristics (e.g., PICOS, length of follow-up) and report characteristics (e.g., years considered, language, publication status) used as criteria for eligibility, giving rationale.

Examples

Types of studies: “Randomised clinical trials studying the administration of hepatitis B vaccine to CRF [chronic renal failure] patients, with or without dialysis. No language, publication date, or publication status restrictions were imposed … ” (50).

Types of participants: “Participants of any age with CRF or receiving dialysis (haemodialysis or peritoneal dialysis) were considered. CRF was defined as serum creatinine greater than 200 µmol/L for a period of more than six months or individuals receiving dialysis (haemodialysis or peritoneal dialysis). … Renal transplant patients were excluded from this review as these individuals are immunosuppressed and are receiving immunosuppressant agents to prevent rejection of their transplanted organs, and they have essentially normal renal function … ” (50).

Types of intervention: “Trials comparing the beneficial and harmful effects of hepatitis B vaccines with adjuvant or cytokine co-interventions [and] trials comparing the beneficial and harmful effects of immunoglobulin prophylaxis. This review was limited to studies looking at active immunization. Hepatitis B vaccines (plasma or recombinant [yeast] derived) of all types, dose, and regimens versus placebo, control vaccine, or no vaccine … ” (50).

Types of outcome measures: “Primary outcome measures: Seroconversion, ie, proportion of patients with adequate anti-HBs response (>10 IU/L or Sample Ratio Units). Hepatitis B infections (as measured by hepatitis B core antigen [HBcAg] positivity or persistent HBsAg positivity), both acute and chronic. Acute (primary) HBV [hepatitis B virus] infections were defined as seroconversion to HBsAg positivity or development of IgM anti-HBc. Chronic HBV infections were defined as the persistence of HBsAg for more than six months or HBsAg positivity and liver biopsy compatible with a diagnosis or chronic hepatitis B. Secondary outcome measures: Adverse events of hepatitis B vaccinations … [and] …mortality” (50).

Explanation

Knowledge of the eligibility criteria is essential in appraising the validity, applicability, and comprehensiveness of a review. Thus, authors should unambiguously specify eligibility criteria used in the review. Carefully defined eligibility criteria inform various steps of the review methodology. They influence the development of the search strategy and serve to ensure that studies are selected in a systematic and unbiased manner.

A study may be described in multiple reports, and one report may describe multiple studies. Therefore, we separate eligibility criteria into the following two components: study characteristics and report characteristics. Both need to be reported. Study eligibility criteria are likely to include the populations, interventions, comparators, outcomes, and study designs of interest (PICOS; see Box 2), as well as other study-specific elements, such as specifying a minimum length of follow-up. Authors should state whether studies will be excluded because they do not include (or report) specific outcomes to help readers ascertain whether the systematic review may be biased as a consequence of selective reporting (4243).

Report eligibility criteria are likely to include language of publication, publication status (e.g., inclusion of unpublished material and abstracts), and year of publication. Inclusion or not of non-English language literature (5155), unpublished data, or older data can influence the effect estimates in meta-analyses (5659). Caution may need to be exercised in including all identified studies due to potential differences in the risk of bias such as, for example, selective reporting in abstracts (6062).

Item 7: Information Sources

Describe all information sources in the search (e.g., databases with dates of coverage, contact with study authors to identify additional studies) and date last searched.

Example

“Studies were identified by searching electronic databases, scanning reference lists of articles and consultation with experts in the field and drug companies. … No limits were applied for language and foreign papers were translated. This search was applied to Medline (1966-Present), CancerLit (1975-Present), and adapted for Embase (1980-Present), Science Citation Index Expanded (1981-Present) and Pre-Medline electronic databases. Cochrane and DARE (Database of Abstracts of Reviews of Effectiveness) databases were reviewed. … The last search was run on 19 June 2001. In addition, we handsearched contents pages of Journal of Clinical Oncology 2001, European Journal of Cancer 2001 and Bone 2001, together with abstracts printed in these journals 1999-2001. A limited update literature search was performed from 19 June 2001 to 31 December 2003” (63).

Explanation

The National Library of Medicine's MEDLINE database is one of the most comprehensive sources of health care information in the world. Like any database, however, its coverage is not complete and varies according to the field. Retrieval from any single database, even by an experienced searcher, may be imperfect, which is why detailed reporting is important within the systematic review.

At a minimum, for each database searched, authors should report the database, platform, or provider (e.g., Ovid, Dialog, PubMed) and the start and end dates for the search of each database. This information lets readers assess the currency of the review, which is important because the publication time-lag outdates the results of some reviews (64). This information should also make updating more efficient (65). Authors should also report who developed and conducted the search (66).

In addition to searching databases, authors should report the use of supplementary approaches to identify studies, such as hand searching of journals, checking reference lists, searching trials registries or regulatory agency Web sites (67), contacting manufacturers, or contacting authors. Authors should also report if they attempted to acquire any missing information (e.g., on study methods or results) from investigators or sponsors; it is useful to describe briefly who was contacted and what unpublished information was obtained.

Item 8: Search

Present the full electronic search strategy for at least one major database, including any limits used, such that it could be repeated.

Examples

In text: “We used the following search terms to search all trials registers and databases: immunoglobulin*; IVIG; sepsis; septic shock; septicaemia; and septicemia … ” (68).

In appendix: “Search strategy: MEDLINE (OVID)

01. immunoglobulins/

02. immunoglobulin$.tw.

03. ivig.tw.

04. 1 or 2 or 3

05. sepsis/

06. sepsis.tw.

07. septic shock/

08. septic shock.tw.

09. septicemia/

10. septicaemia.tw.

11. septicemia.tw.

12. 5 or 6 or 7 or 8 or 9 or 10 or 11

13. 4 and 12

14. randomized controlled trials/

15. randomized-controlled-trial.pt.

16. controlled-clinical-trial.pt.

17. random allocation/

18. double-blind method/

19. single-blind method/