Brian Hutton, PhD, MSc; Georgia Salanti, PhD; Deborah M. Caldwell, PhD, MA, BA; Anna Chaimani, PhD; Christopher H. Schmid, PhD; Chris Cameron, MSc; John P.A. Ioannidis, MD, DSc; Sharon Straus, MD, MSc; Kristian Thorlund, PhD; Jeroen P. Jansen, PhD; Cynthia Mulrow, MD, MSc; Ferrán Catalá-López, PhD, MPH, PharmD; Peter C. Gøtzsche, MD, MSc; Kay Dickersin, PhD, MA; Isabelle Boutron, MD, PhD; Douglas G. Altman, DSc; David Moher, PhD
Financial Support: By the Canadian Agency for Drugs and Technologies in Health and Pfizer Canada for the development of this work. Dr. Hutton is supported by a New Investigator Award from the Canadian Institutes of Health Research and the Drug Safety and Effectiveness Network. Dr. Caldwell is supported by a Medical Research Council Population Health Science Fellowship award (G0902118). Mr. Cameron is a recipient of a Vanier Canada Graduate Scholarship from the Canadian Institutes of Health Research (funding reference number CGV 121171) and is a trainee on the Canadian Institutes of Health Research Drug Safety and Effectiveness Network team grant (funding reference number 116573). The Meta-Research Innovation Center at Stanford (METRICS) is funded by a grant from the Laura and John Arnold Foundation.
Disclosures: Dr. Hutton reports honoraria from Amgen Canada. Dr. Thorlund reports that he is a cofounding partner and majority shareholder of Redwood Outcomes. Dr. Jansen reports that he is a cofounding partner and majority shareholder of Redwood Outcomes. Authors not named here have disclosed no conflicts of interest. Disclosures can be viewed at www.acponline.org/authors/icmje/ConflictOfInterestForms.do?msNum=M14-2385.
Editors' Disclosures: Christine Laine, MD, MPH, Editor in Chief, reports that she has no financial relationships or interests to disclose. Darren B. Taichman, MD, PhD, Executive Deputy Editor, reports that he has no financial relationships or interests to disclose. Cynthia D. Mulrow, MD, MSc, Senior Deputy Editor, reports that she has no relationships or interests to disclose. Deborah Cotton, MD, MPH, Deputy Editor, reports that she has no financial relationships or interest to disclose. Jaya K. Rao, MD, MHS, Deputy Editor, reports that she has stock holdings/options in Eli Lilly and Pfizer. Sankey V. Williams, MD, Deputy Editor, reports that he has no financial relationships or interests to disclose. Catharine B. Stack, PhD, MS, Deputy Editor for Statistics, reports that she has stock holdings in Pfizer.
Requests for Single Reprints: Brian Hutton, PhD, MSc, Ottawa Hospital Research Institute, Center for Practice Changing Research, The Ottawa Hospital–General Campus, 501 Smyth Road, PO Box 201B, Ottawa, Ontario K1H 8L6, Canada; e-mail, firstname.lastname@example.org.
Current Author Addresses:Drs. Hutton and Moher and Mr. Cameron: Ottawa Hospital Research Institute, Center for Practice Changing Research, The Ottawa Hospital–General Campus, 501 Smyth Road, PO Box 201B, Ottawa, Ontario K1H 8L6, Canada.
Drs. Salanti and Chaimani: Department of Hygiene and Epidemiology, University of Ioannina School of Medicine, University Campus Ioannina 45110, Ioannina, Greece.
Dr. Caldwell: School of Social and Community Medicine, Canynge Hall, 39 Whately Road, Bristol BS8 2PS, United Kingdom.
Dr. Schmid: Center for Evidence-Based Medicine, Brown University School of Public Health, Box G-S121-8, Providence, RI 02912.
Dr. Ioannidis: Stanford Prevention Research Center, Stanford University School of Medicine, Medical School Office Building, 1265 Welch Road, Mail Code 5411, Stanford, CA 94305-5411.
Dr. Straus: Li Ka Shing Knowledge Institute, St. Michael's Hospital, 30 Bond Street, Toronto, Ontario M5B 1W8, Canada.
Drs. Thorlund and Jansen: 1505 West Second Avenue, Suite 302, Vancouver, British Columbia V6H 3Y4, Canada.
Dr. Mulrow: American College of Physicians, 190 N. Independence Mall West, Philadelphia, PA 19106.
Dr. Catála-López: Division of Pharmacoepidemiology and Pharmacovigilance, Spanish Medicines and Healthcare Products Agency, Campezo 1, 28022 Madrid, Spain.
Dr. Gøtzsche: Nordic Cochrane Centre, Rigshospitalet, Department 7811, Blegdamsvej 9, DK-2100 Copenhagen, Denmark.
Dr. Dickersin: Johns Hopkins Bloomberg School of Public Health, 615 North Wolfe Street, Room E6152, Baltimore, MD 21205.
Dr. Boutron: Centre d'Epidémiologie Clinique, L'Université Paris Descartes Centre de recherche Epidémiologies et Statistique, INSERM U1153, Equipe: Méthodes en évaluation thérapeutique des maladies chroniques, Hópital Hôtel Dieu, Aile A2 1er étage, 1 Place du parvis Notre Dame, 75181 Paris, Cedex 4, France.
Dr. Altman: Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Botnar Research Centre, Windmill Road, Oxford OX3 7LD, United Kingdom.
Author Contributions:Conception and design: B. Hutton, G. Salanti, D. Moher, C.H. Schmid, A. Chaimani, D.M. Caldwell, K. Thorlund.
Analysis and interpretation of the data: B. Hutton, D. Moher, D.M. Caldwell, A. Chaimani, K. Thorlund, C.H. Schmid, S. Straus, P.C. Gøtzsche.
Drafting of the article: B. Hutton, G. Salanti, D.M. Caldwell, C.H. Schmid, K. Thorlund, D. Moher, C. Cameron, C. Mulrow, F. Catalá-López, P.C. Gøtzsche.
Critical revision of the article for important intellectual content: B. Hutton, G. Salanti, D.M. Caldwell, A. Chaimani, C.H. Schmid, C. Cameron, J.P.A. Ioannidis, S. Straus, K. Thorlund, J.P. Jansen, C. Mulrow, F. Catalá-López, P.C. Gøtzsche, K. Dickersin, I. Boutron, D.G. Altman, D. Moher.
Final approval of the article: B. Hutton, G. Salanti, D.M. Caldwell, A. Chaimani, C.H. Schmid, C. Cameron, J.P.A. Ioannidis, S. Straus, K. Thorlund, J.P. Jansen, C. Mulrow, F. Catalá-López, P.C. Gøtzsche, K. Dickersin, I. Boutron, D.G. Altman, D. Moher.
Statistical expertise: B. Hutton, G. Salanti, A. Chaimani, C.H. Schmid, D.M. Caldwell, K. Thorlund, C. Cameron, D.G. Altman.
Obtaining of funding: B. Hutton, D. Moher.
The PRISMA statement is a reporting guideline designed to improve the completeness of reporting of systematic reviews and meta-analyses. Authors have used this guideline worldwide to prepare their reviews for publication. In the past, these reports typically compared 2 treatment alternatives. With the evolution of systematic reviews that compare multiple treatments, some of them only indirectly, authors face novel challenges for conducting and reporting their reviews. This extension of the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-analyses) statement was developed specifically to improve the reporting of systematic reviews incorporating network meta-analyses.
A group of experts participated in a systematic review, Delphi survey, and face-to-face discussion and consensus meeting to establish new checklist items for this extension statement. Current PRISMA items were also clarified. A modified, 32-item PRISMA extension checklist was developed to address what the group considered to be immediately relevant to the reporting of network meta-analyses.
This document presents the extension and provides examples of good reporting, as well as elaborations regarding the rationale for new checklist items and the modification of previously existing items from the PRISMA statement. It also highlights educational information related to key considerations in the practice of network meta-analysis. The target audience includes authors and readers of network meta-analyses, as well as journal editors and peer reviewers.
Table.Checklist of Items to Include When Reporting a Systematic Review Involving a Network Meta-analysis
Overview of a network graph.
A network graph presenting the evidence base for a hypothetical review of 4 interventions is shown. Treatments are represented by nodes and head-to-head studies between treatments are represented by edges. The sizes of edges and nodes are used to visually depict the available numbers of studies comparing interventions and the numbers of patients studied with each treatment.
Terminology: Reviews With Networks of Multiple Treatments
Terms are discussed further in the Box.Top. Adjusted indirect treatment comparison of treatments B and C based on studies that used a common comparator, treatment A. Middle. A network of 8 treatments and a common comparator, with a mix of comparisons against the control treatment and a subset of all possible comparisons between active treatments. Bottom. A treatment network similar to that shown in the middle panel, but with study data available for an additional 4 comparisons in the network which form closed loops.
Graphical overview of the terminologies that are related to the study of treatment networks.
Different combined oral contraceptives and the risk of venous thrombosis: systematic review and network meta-analysis. (61)
Network meta-analysis on randomized trials focusing on the preventive effect of statins on contrast-induced nephropathy. (62)
Objective. To determine the comparative effectiveness and safety of current maintenance strategies in preventing exacerbations of asthma.
Design. Systematic reviewand network meta-analysisusing Bayesian statistics.
Data Sources. Cochrane systematic reviewson chronic asthma, complemented by an updated search when appropriate.
Eligibility Criteria. Trials of adults with asthma randomised to maintenance treatments of at least 24 weeks duration and that reported on asthma exacerbations in full text. Low dose inhaled corticosteroid treatment was the comparator strategy. The primary effectiveness outcome was the rate of severe exacerbations. The secondary outcome was the composite of moderate or severe exacerbations. The rate of withdrawal was analysed as a safety outcome.
Results. 64 trials with 59,622 patient years of follow-up comparing 15 strategies and placebo were included. For prevention of severe exacerbations, combined inhaled corticosteroids and long acting β-agonists as maintenance and reliever treatment and combined inhaled corticosteroids and long acting β-agonists in a fixed daily dose performed equally well and were ranked first for effectiveness. The rate ratios compared with low dose inhaled corticosteroids were 0.44 (95% CrI 0.29 to 0.66) and 0.51 (0.35 to 0.77), respectively. Other combined strategies were not superior to inhaled corticosteroids and all single drug treatments were inferior to single low dose inhaled corticosteroids. Safety was best for conventional best (guideline based) practice and combined maintenance and reliever therapy.
Conclusions. Strategies with combined inhaled corticosteroids and long acting β-agonists are most effective and safe in preventing severe exacerbations of asthma, although some heterogeneity was observed in this network meta-analysisof full text reports.
Probabilities and Rankings in Network Meta-analysis
The Assumption of Transitivity for Network Meta-analysis
Network Meta-analysis and Assessment of Consistency of Findings
Although progress has been achieved in the field and patients live longer, the relative merits of the many different chemotherapy and targeted treatment regimens are not well understood. Hundreds of trials have been conducted to compare treatments for advanced breast cancer, but because each has compared only two or a few treatments, it is difficult to integrate information on the relative efficacy of all tested regimens. This integration is important because different regimens vary both in cost and in toxicity. Therefore, we performed a comprehensive systematic review of chemotherapy and targeted treatment regimens in advanced breast cancer and evaluated through a multiple-treatments meta-analysis the relative merits of the many different regimens used to prolong survival in advanced breast cancer patients. (67)
Our analysis classified fluids as crystalloids (divided into balanced and unbalanced solutions) and colloids (divided into albumin, gelatin, and low- and high-molecular weight hydroxyethyl starch [HES] [threshold molecular weight, 150 000 kDa]). We considered fluid balanced if it contained an anion of a weak acid (buffer) and its chloride content was correspondingly less than in 0.9% sodium chloride. The relevant analyses were a 4-node NMA [network meta-analysis] (crystalloids vs. albumin vs. HES vs. gelatin), a 6-node NMA (crystalloids vs. albumin vs. HES vs. gelatin, with crystalloids divided into balanced or unbalanced and HES divided into low or high molecular weight), and a conventional direct frequentist fixed effects meta-analytic comparison of crystalloids versus colloids. (68)
We analyzed published and unpublished randomized trials performed in patients with pulmonary hypertension. At the level of drug classes, we examined whether head-to-head comparisons are between agents in the same class or between agents in different classes. At the level of companies, we examined whether trials involve only agents (as active comparators or backbones) owned by the same company, or include treatments by different companies. In the networks of drug comparisons, each drug is drawn by a node and randomized comparisons between drugs are shown by links between the nodes. When a drug is compared against the same agent in different dose or formulation, this is represented by an auto-loop. In the networks of companies, nodes stand for companies and auto-loops around these nodes represent trials involving agents of a single company. Links between different nodes characterize trials comparing agents that belong to different companies. ( 70)
Network Geometry and Considerations for Bias
For each pairwise comparison and each outcome at each time point, we used odds ratios (OR) with 95% confidence intervals (95% CIs) as a measure of the association between the treatment used and efficacy. As the outcomes are negative, ORs >1 correspond to beneficial treatment effects of the first treatment compared with the second treatment.
As a measure that reflects ranking and the uncertainty, we used the Surface Under the Cumulative RAnking curve (SUCRA) as described in Salanti 2011. This measure, expressed as percentage, showed the relative probability of an intervention being among the best options. (82)
The network meta-analysis was based on a bayesian random effects Poisson regression model, which preserves randomised treatment comparisons within trials. The model uses numbers of patients experiencing an event and accumulated patient years to estimate rate ratios. The specification of nodes in the network was based on the randomised intervention or in case of strategy trials, such as COURAGE [Clinical Outcomes Utilizing Revascularization and Aggressive Drug Evaluation] or FAME-2 [Fractional flow reserve versus Angiography for Multi-Vessel Evaluation], on the intervention received by the majority of patients in a trial arm. Analyses were performed using Markov-Chain Monte-Carlo methods. The prior distribution for treatment effects was minimally informative: a normal distribution with a mean of 1 and a 95% reference range from 0.01 to 100 on a rate ratio scale. The prior for the between trial variance τ2, which we assumed to be equal across comparisons, was based on empirical evidence derived from semi-objective outcomes of head to head comparisons: a log normal distribution with a geometric mean of τ2 of 0.04 and a 95% reference range from 0.001 to 1.58. Rate ratios were estimated from the median and corresponding 95% credibility intervals from the 2.5th and 97.5th centiles of the posterior distribution. Convergence was deemed to be achieved if plots of the Gelman-Rubin statistics indicated that widths of pooled runs and individual runs stabilised around the same value and their ratio was around 1. (83)
Consistency was mainly assessed by the comparison of the conventional network meta-analysis model, for which consistency is assumed, with a model that does not assume consistency (a series of pairwise meta-analyses analysed jointly). If the trade-off between model fit and complexity favoured the model with assumed consistency, this model was preferred. Moreover, we calculated the difference between direct and indirect evidence in all closed loops in the network; inconsistent loops were identified with a significant (95% CrI that excludes 0) disagreement between direct and indirect evidence. A loop of evidence is a collection of studies that links treatments to allow for indirect comparisons; the simplest loop is a triangle formed by three direct comparison studies with shared comparators. (88)
We considered how decisions to group glaucoma treatments could affect the transitivity assumption and interpretation of the analysis. (27) [See Appendix Figure 1.]
Example figures: alternative geometries of a network of interventions for glaucoma.
Example of alternative geometries of a treatment network for the treatment of glaucoma based on the splitting (A) versus lumping (B) of treatment regimens in the treatment network. A sensitivity analysis considering alternative geometries should be considered when lumping treatment nodes. Depending on quantity, results may be best in appendices. APRAC = apraclonidine; BETAX = betaxolol; BIMAT = bimatoprost;BRIM = brimonidine; BRIN = brinzolamide; CART = carteolol; DOR = dorzolamide; NO TRT = no treatment; NR = not reported; LATAN = latanoprost; LEVO = levobudolol; PL = placebo; TIMO = timolol; TRAV = travoprost.
We a priori had selected allocation concealment, assessor blinding, treatment fidelity and imputation of numbers of responders as potentially important effect modifiers to be examined in sensitivity analyses to limit the included studies to those at low risk of bias. We conducted additional meta-regression analyses using random effects network meta-regression models to examine potential effect moderators such as the mean age of participants, the type of rating scales (clinician-rated versus self-rated), publication status (published versus dissertation), and therapy format (individual vs group). (94)
Random effects network meta-analyses with informative priors for heterogeneity variances were conducted for the analyses. We also conducted fixed and random effects models with vague priors. (95)
Appendix Figure 2 shows a network graph comparing antipsychotic agents for prevention of schizophrenia relapse (12).
Example figure: presentation of network graph on antipsychotics for schizophrenia relapse.
The size of treatment nodes reflects the number of patients randomly assigned to each treatment. The thickness of edges represents the number of studies underlying each comparison.
A total of 2,545 pulmonary hypertension patients received active pulmonary hypertension medication. The studied agents were more commonly bosentan (n = 13 trials; patients receiving treatment = 633) and sildenafil (n = 13 trials; patients receiving treatment = 593). Placebo was used as the comparator arm in 38 studies (patients receiving placebo = 1,643). Of the patients that received placebo, 52 participants were part of crossover studies with sildenafil. The most frequently used comparisons were bosentan versus placebo (n = 11) and sildenafil versus placebo (n = 11). Studies that used placebo as the comparator arm (n = 38) were for the most part sponsored by the pharmaceutical company that owned the product (n = 28 studies [74%]). The only two published head-to-head comparisons of different medications (sildenafil against bosentan) were not sponsored by pharmaceutical companies, but by the British Heart Foundation and the Italian Health Authority. (70)
Example figure: network geometry of published and unpublished randomized studies on U.S. Food and Drug Administration–approved medications for pulmonary hypertension.
Each intervention is shown by a circular node, with the same color used to group interventions which belong to the same drug class. An auto-loop represents studies where different doses of the same medication have been compared. IV = intravenous; SC = subcutaneous.
The Appendix Table presents an example of one possible approach to provision of data on mortality observed with five different interventions for treatment of left ventricular dysfunction (medical resynchronisation, cardiac resynchronisation, implantable defibrillator, combined resynchronisation and defibrillator, and amiodarone) as described elsewhere (98).
Appendix Table. Example Table: Presentation of Outcome Data, by Included Study*
Two examples of reporting of comparative treatment efficacy from a review comparing efficacy of treatments for multiple sclerosis with regard to progression of disability are presented in Appendix Figures 4 and 5(82).
Example figure: league table presenting network meta-analysis estimateslower triangle) and direct estimates (upper triangle) of efficacy (disability progression over 36 months) of immunomodulators and immunosuppressants for multiple sclerosis.
Treatments are reported in order of relative ranking for efficacy. Comparisons between treatments should be read from left to right, and their odds ratio is in the cell in common between the column-defining treatment and the row-defining treatment. Odds ratios less than 1 favor the column-defining treatment for the network estimates and the row-defining treatment for the direct estimates. IFN = interferon.
Example figure: forest plot for efficacy (disability progression over 36 months) of immunomodulators and immunosuppressants for multiple sclerosis versus placebo.
Summary estimates are reported for only a subset of all possible pairwise comparisons, namely active interventions versus placebo. Treatments are ranked according to their surface under the cumulative ranking values. OR = odds ratio; CrI = credible interval; IFN =interferon.
Examples: tabular (top) and graphical ( bottom) reporting of treatment rankings regarding comparison of treatment-associated risks of grade 3 or 4 hematologic toxicities for resected pancreatic adenocarcinoma.
Rankings nearer 1 suggest greater risk. 5-FU = 5-fluorouracil.
The assumption of consistency was generally supported by a better trade-off between model fit and complexity when consistency was assumed than when it was not. Significant disagreement between direct and indirect estimates (inconsistency) was identified in only very few cases: for efficacy seven of 80 loops; for all-cause discontinuation three of 80 loops; for weight gain one of 62 loops; for extrapyramidal side-effects one of 56 loops; for prolactin increase three of 44 loops; for QTc prolongation two of 35 loops; and for sedation none of 49 loops were inconsistent (appendix pp 105-14). Data were double-checked and we could not identify any important variable that differed across comparisons in these loops. The number of included studies in the inconsistent loops was typically small, so the extent of inconsistency was not substantial enough to change the results. (88)
Differences in Approach to Fitting Network Meta-Analyses.
Standard adjusted dose vitamin K agonist (VKA) (odds ratio 0.11 (95% credible interval 0.04 to 0.27)), dabigatran, apixaban 5 mg, apixaban 2.5 mg, and rivaroxaban decreased the risk of recurrent venous thromboembolism, compared with ASA [acetylsalicylic acid]. Compared with low dose VKA, standard adjusted dose VKA reduced the risk of recurrent venous thromboembolism (0.25 (0.10 to 0.58)).
An appendix presents a detailed explanation for the potential discrepancy between ASA and placebo results. Results for most class level analyses also aligned with those reported previously in the treatment level analysis. Subgroup analyses, performed to account for heterogeneity due to study duration, yielded results that were more favourable for ASA than those obtained from the primary analysis. However, results for ASA were still less pronounced than those reported for other treatments (standard adjusted dose VKA, low intensity VKA, and dabigatran) that remained in the evidence network. Sensitivity analysis excluding ximelagatran from the analysis did not change the results reported. (95 )
Table 2 presents an investigation into potential sources of variation in people with diabetes in the network. Estimates of relative risk comparing sirolimus eluting stents with paclitaxel eluting stents depended to some extent on the quality of the trials, the length of followup, and the time of completion of patient recruitment (table 2), but 95% credibility intervals were wide and tests for interaction negative (P for interaction ≥0.16). The estimated relative risk of death when sirolimus eluting stents were compared with bare metal stents was greater when the specified duration of dual antiplatelet therapy was less than six months (2.37, 95% credibility interval 1.18 to 5.12) compared with six months or longer (0.89, 0.58 to 1.40, P for interaction 0.02), however. (106)
None of the regression coefficients of the meta-regression examining possible effect moderators turned out to be statistically significant [-0.024 (95% CI -0.056 to 0.006) for age, -0.899 (95% CI -1.843 to 0.024 for rating scale), -0.442 (95% CI -1.399 to 0.520) for publication status, and 0.004 (95% CI -0.798 to 0.762) for therapy format]. (94)
Our study has several limitations. The network could be expanded to old drugs such as perphenazine and sulpiride, which have had good results in effectiveness studies, but only a few relevant perphenazine trials have been done.
Reporting of side-effects is unsatisfactory in randomised controlled trials in patients with psychiatric disorders, and some side-effects were not recorded at all for some drugs. The meta-regression with percentage of withdrawals as a moderator could not rule out all potential bias associated with high attrition in schizophrenia trials.
Our findings cannot be generalised to young people with schizophrenia, patients with predominant negative symptoms, refractory patients, or stable patients, all of whom were excluded to enhance homogeneity as required by multiple-treatments meta-analysis. A funnel plot asymmetry was seen, which is not necessarily the expression of publication bias, but rather of higher efficacy in small trials than in larger ones, for various reasons. For example, sample size estimates for drugs with low efficacy might have needed higher numbers of participants to attain statistical significance than in trials with more effective drugs. However, accounting for trial size did not substantially change the rankings. Finally, because multiple-treatments meta-analysis requires reasonably homogeneous studies, we had to restrict ourselves to short-term trials. Because schizophrenia is often a chronic disorder, future multiple-treatments meta-analyses could focus on long-term trials, but these remain scarce. In any case, for clinicians to know to which drugs patients are most likely to respond within a reasonable duration such as 6 weeks is important. (88)
The In the Clinic® slide sets are owned and copyrighted by the American College of Physicians (ACP). All text, graphics, trademarks, and other intellectual property incorporated into the slide sets remain the sole and exclusive property of the ACP. The slide sets may be used only by the person who downloads or purchases them and only for the purpose of presenting them during not-for-profit educational activities. Users may incorporate the entire slide set or selected individual slides into their own teaching presentations but may not alter the content of the slides in any way or remove the ACP copyright notice. Users may make print copies for use as hand-outs for the audience the user is personally addressing but may not otherwise reproduce or distribute the slides by any means or media, including but not limited to sending them as e-mail attachments, posting them on Internet or Intranet sites, publishing them in meeting proceedings, or making them available for sale or distribution in any unauthorized form, without the express written permission of the ACP. Unauthorized use of the In the Clinic slide sets will constitute copyright infringement.
Long Ge, Jin-hui Tian, Lun Li, Ke-hu Yang
The First Clinical Medicine College of Lanzhou University, Evidence-Based Medicine Center of Lanzhou University,Key Laboratory of Evidence-based Medicine and Clinical Translational Research of Gansu P
June 14, 2015
The PRISMA Extension Statement for Statistical Analysis Reporting of Network Meta-Analysis is Needed
Hutton B and colleagues (1) have published the PRISMA extension statement for reporting of network meta-analyses (NMAs). We believe the PRISMA extension statement adding this very important items to improve the reporting of network meta-analyses. We all know that the validity of NMAs results highly depends on some key basic assumptions. Some previous reviews have focused on the reporting of published NMAs. Their results indicated that there were serious reporting flaws, especially regarding assessment of assumptions and reporting of statistical analysis applied (2). Tan SH et al. (3) established a guidance based on 19 published NMAs to guide reporting of statistical methods applied, but some key reporting items such as convergence assessment were missing. Therefore, specifically detailed checklists for reporting and conducting of statistical analysis are needed. Based on published literature, we suggest reporting for statistical analysis of Bayesian NMAs:Methods section: The detail of sample size calculationDirect comparison: Assessment of heterogeneity, model of pooling data,summary measure, assessment of publication bias, sensitivity analysis, other analysis, software applied.Network meta-analysis: Assessment of heterogeneity, adjustment for covariates, adjustment of multiple arms, code applied, selection of prior distribution, selection of fixed or random effect model, selection of consistency or inconsistency, assessment of inconsistency, assessment of convergence, summary measure (including treatment ranking), assessment of publication bias, sensitivity analysis (based on prior distribution or other), other analysis, software applied.Results section:The results of sample size calculationDirect comparison: Results of heterogeneity assessment, model applied, results of direct comparison, publication bias assessment, sensitivity analysis and other analysis.Network meta-analysis: Methods and results of heterogeneity, results of model fit tested, number of chains, the staring values for sampling, number of iterations per chain, number of iteration used for final results, results of convergence assessment, prior distributions used, results of indirect comparison, results of network meta-analysis, results of inconsistency assessment, results of ranking, publication bias assessment, sensitivity analysis, other analysis. The details for each item should also be described in published papers. However, those items are based on published literature. Delphi survey, and face-to-face discussion and consensus meeting should be established to develop another PRISMA extension statement for reporting of statistical analysis and assumptions of NMA (PRISMA-S). We strongly believe that it will play very important parts to improve the quality of NMAs. We declare that we have no conflicts of interest.Long GeThe First Clinical Medicine College of Lanzhou University, Lanzhou 730000, China;Evidence-Based Medicine Center of Lanzhou University, Lanzhou 730000, China.Jin-hui Tian, Lun Li, Ke-hu Yang*Evidence-Based Medicine Center of Lanzhou University, Lanzhou 730000, China;Key Laboratory of Evidence-based Medicine and Clinical Translational Research of Gansu Province, Lanzhou email@example.com(1)Hutton B, Salanti G, Caldwell DM, Chaimani A, Schmid CH, Cameron C, et al. The PRISMA Extension Statement for Reporting of Systematic Reviews Incorporating Network Meta-analyses of Health Care Interventions: Checklist and Explanations.Ann Intern Med. 2015;162(11):777-84.(2)Li L, Tian JH, Yang KH. Current situation of reporting statement for network meta-analysis.Chin J Evid Based Pediatr.2014;9(6):467-71.(3)Tan SH, Bujkiewicz S, Sutton A, Dequen P, Cooper N.Presentational approaches used in the UK for reporting evidence synthesis using indirect and mixed treatment comparisons.J Health Serv Res Policy. 2013;18(4):224-32.
Jin-hui Tian, PhD, Long Ge, MD, Lun Li, PhD
Lanzhou University; Key Laboratory of Evidence-based Medicine and Clinical Translational Research of Gansu Province
June 22, 2015
Searching for previous published and unpublished or ongoing systematic reviews/meta-analyses is very important.
The much-anticipated PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analyses) extension statement for reporting of network meta-analysis (NMA) has been published by Hutton B and colleagues (1). We have noted that two items including “information sources” (Item 7) and “search” (Item 8) remained the original PRISMA statement. However, we all know that the searching for the evidence in NMA is more important and more complex than traditional systematic reviews and pairwise meta-analyses (2). For example, the first step for NMA is a thorough and rigorous search for previous systematic reviews/meta-analyses, to ensure the research question has not been carried out previously (2). In addition, the reference lists of previous published systematic reviews/meta-analyses should be tracked to avoid missing important studies. Unfortunately, only 40% of published NMAs searched the reference lists of previous systematic reviews/meta-analyses (3). Moreover, it is very important to peer review the quality of the searches of previous systematic reviews/meta-analyses by a specialist librarian and to determine whether the conducting of NMA is based on previous systematic reviews/meta-analyses. Therefore, more details of evidence searching should be needed to guide NMA reviewers.
We declare that we have no conflicts of interest.
Jin-hui Tian, PhD
Evidence-Based Medicine Center of Lanzhou University, Lanzhou 730000, China.
Key Laboratory of Evidence-based Medicine and Clinical Translational Research of Gansu Province, Lanzhou 730000.
Long Ge,MD; Lun Li, PhD
The First Clinical Medicine College of Lanzhou University, Lanzhou 730000, China;
Evidence-Based Medicine Center of Lanzhou University, Lanzhou 730000, China;
Key Laboratory of Evidence-based Medicine and Clinical Translational Research of Gansu Province, Lanzhou 730000.
(1)Hutton B, Salanti G, Caldwell DM, Chaimani A, Schmid CH, Cameron C, et al. The PRISMA Extension Statement for Reporting of Systematic Reviews Incorporating Network Meta-analyses of Health Care Interventions: Checklist and Explanations.Ann Intern Med. 2015;162(11):777-84.
(2)Golger S, Wright K. Searching for evidence.In: Biondi-Zoccai G, editor. Network meta-analysis: evidence synthesis with mixed treatment comparison. New York: Nova Science, 2014. P. 63-76.
(3)Li L, Tian J, Tian H, Moher D, Liang F, Jiang T, et al. Network meta-analyses could be improved by searching more sources and by involving a librarian. J Clin Epidemiol. 2014; 67(9):1001-7.
Brian Hutton (1,2), Chris Cameron (1,3), David Moher (1,2)
1=Ottawa Hospital Research Institute; 2=University of Ottawa School of Epidemiology, Public Health and Preventive Medicine; 3=Cornerstone Research Group
July 23, 2015
PRISMA Considerations and Searching the Literature
We thank Drs Ge, Tian, Li and Yang(1;2) for their interest regarding the PRISMA extension for network meta-analysis.(3) They suggest there are some potential additional considerations regarding information sources and search strategies to consider. While we agree with the practices described Drs. Tian, Ge and Li describe regarding searching, we feel they are entirely applicable to traditional systematic reviews and are addressed in the PRISMA Statement’s explanation and elaboration article.(4) The main intent of the PRISMA extension statement for reporting of network meta-analysis is to focus on items that were not addressed in PRISMA and differ substantially from practices for traditional systematic reviews and meta-analyses. Regarding their first suggestion, we feel ensuring the need for a proposed review is equally important for traditional reviews, and in all cases searching for existing literature should be preceded by in-depth consideration of the clinical importance of the research question in a PICOS (Population-Intervention-Comparator(s)-Outcome(s)-Study design) framework. Regarding their second suggestion, we believe it has long been common for researchers undertaking reviews of multiple forms to inspect bibliographies of past reviews and included studies as a source for potentially relevant studies; the PRISMA Statement’s Explanation and Elaboration document addresses this issue in Item 7, suggesting ‘In addition to searching databases, authors should report the use of supplementary approaches to identify studies, such as hand searching of journals, checking reference lists, searching trials registries or regulatory agency Web sites, contacting manufacturers, or contacting authors.’(4) Lastly, we agree that the increased number of interventions in a network meta-analysis can heighten the challenge of completing the systematic search strategy for a review. However, this may also often be true of other reviews not involving a multi-treatment question; for example, reviews involving more than one indication of relevance, or reviews involving complex interventions. In these and other scenarios, we support the practice of peer review of literature searches to maximize their quality. This is addressed in Item 8 in the PRISMA Explanations and Elaborations Statement: ‘We encourage authors to state whether search strategies were peer reviewed as part of the systematic review process.’(4) Therefore, we support these practices noted by Tian, Ge, and Li. However we feel their importance and existing practice amongst researchers extend to many additional types of reviews, and that guidance from the PRISMA Statement remains highly relevant. The authors also suggest a potential need for a second guidance document addressing reporting for Bayesian network meta-analyses; we disagree with this perspective at this time. The examples and elaborations provided in our guidance address reporting considerations for the key items that were suggested, while we do not foresee a need for certain suggested components such as specification of starting values or the number of iterations used (we recommend provision of details for convergence assessment already). We are unclear as to the authors’ intended meaning of suggesting sample size, however we hypothesize this is a reference to statistical power in network meta-analyses. We agree this can be of interest, and may be especially so for outcomes with few events. Some research has been conducted in this area,(5) although additional research is needed to inform considerations for reporting guidance.We believe the current guidance provides a strong set of minimum reporting items for Frequentist and Bayesian NMAs, while authors are certainly encouraged to provide additional information of relevance to readers to support their reviews. As methodologies continue to evolve in this rapidly developing area, we will continue to gather materials for a possible future update of this extension statement which may include guidance that additional statistical considerations be reported.Brian Hutton, PhD; David Moher, PhDOttawa Hospital Research Institute, Ottawa, Canada;University of Ottawa School of Epidemiology, Public Health and Preventive Medicine, Ottawa, CanadaChris Cameron, PhDOttawa Hospital Research Institute, Ottawa, Canada;Cornerstone Research Group Inc., Burlington, Canada Reference List(1) Tian J, Ge L, and Li L. Searching for previous published and unpublished or ongoing systematic reviews/meta-analyses is very important (Commentary). Annals of Internal Medicine. 2015. (2) Ge L, Tian J, Li L, and Yang K. The PRISMA Extension Statement for Statistical Analysis Reporting of Network Meta-Analysis is Needed (Commentary). Annals of Internal Medicine. 14-6-2015. (3) Hutton B, Salanti G, Caldwell D, Schmid C, Chaimani A, Cameron C, Ioannidis J, and et al. The PRISMA Extension Statement for Reporting of Systematic Reviews Incorporating Network Meta-Analyses of Healthcare Interventions: Checklist and Explanations. Annals of Internal Medicine 162(11), 777-784. 2015. (4) Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gotzsche P, Ioannidis J, Clarke M, Devereau PJ, Kleijnen J, and Moher D. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. Annals of Internal Medicine 151(4), W65-W94. 2009. (5) Thorlund K and Mills E. Sample size and power considerations in network meta-analysis. Systematic Reviews 1(41. doi: 10.1186/2046-4053-1-41). 2012.
Hutton B, Salanti G, Caldwell DM, Chaimani A, Schmid CH, Cameron C, et al. The PRISMA Extension Statement for Reporting of Systematic Reviews Incorporating Network Meta-analyses of Health Care Interventions: Checklist and Explanations. Ann Intern Med. 2015;162:777–784. doi: 10.7326/M14-2385
Download citation file:
Published: Ann Intern Med. 2015;162(11):777-784.
Research and Reporting Methods.
Results provided by:
Copyright © 2019 American College of Physicians. All Rights Reserved.
Print ISSN: 0003-4819 | Online ISSN: 1539-3704
Conditions of Use