0

The full content of Annals is available to subscribers

Subscribe/Learn More  >
Research and Reporting Methods |

Conceptual and Technical Challenges in Network Meta-analysis

Andrea Cipriani, PhD; Julian P.T. Higgins, PhD; John R. Geddes, MD; and Georgia Salanti, PhD
[+] Article and Author Information

From University of Verona, Verona, Italy; University of Oxford, Oxford, United Kingdom; University of Bristol, Bristol, United Kingdom; University of York, North Yorkshire, United Kingdom; and University of Ioannina School of Medicine, Ioannina, Greece.

Acknowledgment: The authors thank Anna Chaimani for her help and support in drafting Figure 3 and Appendix Figure 1.

Grant Support: By the European Research Council (grant 260559 IMMA; Dr. Salanti).

Potential Conflicts of Interest: Disclosures can be viewed at www.acponline.org/authors/icmje/ConflictOfInterestForms.do?msNum=M12-3072.

Requests for Single Reprints: Andrea Cipriani, PhD, Department of Public Health and Community Medicine, University of Verona, Policlinico G.B. Rossi, Piazzale L.A. Scuro 10, 37134 Verona, Italy; e-mail, andrea.cipriani@psych.ox.ac.uk.

Current Author Addresses: Dr. Cipriani: Department of Public Health and Community Medicine, University of Verona, Policlinico G.B. Rossi, Piazzale L.A. Scuro 10, 37134 Verona, Italy.

Dr. Higgins: School of Social and Community Medicine, University of Bristol, Canynge Hall, 39 Whatley Road, Bristol BS8 2PS, United Kingdom.

Dr. Geddes: Department of Psychiatry, University of Oxford, Warneford Hospital, Oxford OX3 7JX, United Kingdom.

Dr. Salanti: Department of Hygiene and Epidemiology, University of Ioannina School of Medicine, University Campus Ioannina, 45110 Ioannina, Greece.

Author Contributions: Conception and design: A. Cipriani, J.R. Geddes, G. Salanti.

Analysis and interpretation of the data: A. Cipriani, J.R. Geddes, G. Salanti.

Drafting of the article: A. Cipriani, J.P.T. Higgins, J.R. Geddes, G. Salanti.

Critical revision of the article for important intellectual content: J.P.T. Higgins, J.R. Geddes, G. Salanti.

Final approval of the article: J.P.T. Higgins, J.R. Geddes, G. Salanti.

Statistical expertise: J.P.T. Higgins, G. Salanti.

Collection and assembly of data: A. Cipriani, G. Salanti.


Ann Intern Med. 2013;159(2):130-137. doi:10.7326/0003-4819-159-2-201307160-00008
Text Size: A A A

The increase in treatment options creates an urgent need for comparative effectiveness research. Randomized, controlled trials comparing several treatments are usually not feasible, so other methodological approaches are needed. Meta-analyses provide summary estimates of treatment effects by combining data from many studies. However, an important drawback is that standard meta-analyses can compare only 2 interventions at a time. A new meta-analytic technique, called network meta-analysis (or multiple treatments meta-analysis or mixed-treatment comparison), allows assessment of the relative effectiveness of several interventions, synthesizing evidence across a network of randomized trials. Despite the growing prevalence and influence of network meta-analysis in many fields of medicine, several issues need to be addressed when constructing one to avoid conclusions that are inaccurate, invalid, or not clearly justified. This article explores the scope and limitations of network meta-analysis and offers advice on dealing with heterogeneity, inconsistency, and potential sources of bias in the available evidence to increase awareness among physicians about some of the challenges in interpretation.

Figures

Grahic Jump Location
Appendix Figure 1.

Number of network meta-analyses published in the scientific literature and their citations since 1997.

We defined a network meta-analysis as any meta-analysis that used a form of valid indirect relative treatment estimates.

Grahic Jump Location
Grahic Jump Location
Figure 1.

Network of eligible comparisons for the multiple treatments meta-analysis for efficacy in acute mania.

The lines between treatment nodes indicate the comparisons made within randomized trials. The width of the lines is proportional to the number of trials comparing each pair of treatments, and the size of each node is proportional to the number of randomly assigned participants (sample size). If there is no line between 2 nodes, then no studies (that is, no direct evidence) compare the 2 drugs. Stata (StataCorp, College Station, Texas) and R routines (R Foundation for Statistical Computing, Vienna, Austria) are used to plot a network, and their results, as well as the routines and their help files, are available at www.mtm.uoi.gr. Data from reference 11.

Grahic Jump Location
Grahic Jump Location
Figure 2.

Examples of network structures.

Nodes represent a treatment or an intervention; lines show where direct comparisons exist from 1 or more RCTs. RCT = randomized, controlled trial. A. A single closed loop involves 3 interventions and can provide data to calculate direct comparisons and indirect comparisons (mixed evidence). B. A “star network” in which all interventions have a single mutual comparator. C. A well-connected network in which all interventions have been compared with each other in several trials. D. A complex network with many loops and groups that may have sparse connections.

Grahic Jump Location
Grahic Jump Location
Figure 3.

An inconsistency plot reporting the first 18 triangular closed loops in the network meta-analysis comparing antimanic drugs.

Each diamond represents the difference between direct and indirect estimates in terms of standardized mean difference for efficacy; the corresponding horizontal line represents its 95% CI. For loops with no inconsistency (in green), this difference is 0 and the CI has 1 negative and 1 positive limit (across 0). If the limits of the CI are greater or less than 0, there is statistically significant inconsistency (in red). Data from reference 11.

Grahic Jump Location
Grahic Jump Location
Appendix Figure 2.

Absolute rankograms for presenting probabilities and rankings in network meta-analysis.

Network meta-analyses enable estimation of the probability that each intervention is the best for each outcome. Probabilities for each treatment taking each possible rank can be plotted in absolute rankograms or cumulative rankograms. These absolute rankograms (modified from reference 41) rank for response rate (efficacy is the solid red line) and withdrawal rate (acceptability is the dotted blue line). Ranking indicates the probability to be the best treatment, the second best, the third best, and so on, among the group of 12 antidepressants under investigation. For example, drug A has a higher probability to be among the worst antidepressants in terms of efficacy and acceptability, although drug B is good.

Grahic Jump Location
Grahic Jump Location
Appendix Figure 3.

Cumulative rankograms for presenting probabilities and rankings in network meta-analysis.

Network meta-analyses enable estimation of the probability that each intervention is the best for each outcome. Probabilities for each treatment taking each possible rank can be plotted in absolute rankograms or cumulative rankograms. These cumulative rankograms (modified from reference 41) show the distribution of the probabilities of each treatment to be ranked at each of the possible 14 positions within the group of antimanic drugs under investigation. The larger the surface below the cumulative ranking curve (usually called SUCRA), the more probable the drug will be among the lowest ranks (that is, the more effective or acceptable the treatment). For example, drugs A and B have a higher probability to be among the best antimanic drugs (drug B is better than drug A), although drug C is likely among the worst. The SUCRA can be quantified and reported in tables to show its mean values together with the 95% CIs.

Grahic Jump Location

Tables

References

Letters

NOTE:
Citing articles are presented as examples only. In non-demo SCM6 implementation, integration with CrossRef’s "Cited By" API will populate this tab (http://www.crossref.org/citedby.html).

Comments

Submit a Comment
Submit a Comment

Summary for Patients

Clinical Slide Sets

Terms of Use

The In the Clinic® slide sets are owned and copyrighted by the American College of Physicians (ACP). All text, graphics, trademarks, and other intellectual property incorporated into the slide sets remain the sole and exclusive property of the ACP. The slide sets may be used only by the person who downloads or purchases them and only for the purpose of presenting them during not-for-profit educational activities. Users may incorporate the entire slide set or selected individual slides into their own teaching presentations but may not alter the content of the slides in any way or remove the ACP copyright notice. Users may make print copies for use as hand-outs for the audience the user is personally addressing but may not otherwise reproduce or distribute the slides by any means or media, including but not limited to sending them as e-mail attachments, posting them on Internet or Intranet sites, publishing them in meeting proceedings, or making them available for sale or distribution in any unauthorized form, without the express written permission of the ACP. Unauthorized use of the In the Clinic slide sets will constitute copyright infringement.

Toolkit

Buy Now

to gain full access to the content and tools.

Want to Subscribe?

Learn more about subscription options

Advertisement
Related Articles
Journal Club
Topic Collections
PubMed Articles
Forgot your password?
Enter your username and email address. We'll send you a reminder to the email address on record.
(Required)
(Required)