The full content of Annals is available to subscribers

Subscribe/Learn More  >
Academia and the Profession |

Systematic Reviews of Diagnostic Test Accuracy

Mariska M.G. Leeflang, PhD; Jonathan J. Deeks, PhD; Constantine Gatsonis, PhD; Patrick M.M. Bossuyt, PhD, on behalf of the Cochrane Diagnostic Test Accuracy Working Group
[+] Article, Author, and Disclosure Information

From the Dutch Cochrane Centre and Academic Medical Center, University of Amsterdam, Amsterdam, the Netherlands; Unit of Public Health, Epidemiology and Biostatistics, University of Birmingham, Birmingham, United Kingdom; and Center for Statistical Sciences, Brown University, Providence, Rhode Island.

Grant Support: By the UK National Institute for Health Research (grant RNC/018/0003), the National Cancer Institute (grant 2U01CA079778), and the Cochrane Collaboration.

Potential Financial Conflicts of Interest: None disclosed.

Requests for Single Reprints: Jonathan J. Deeks, PhD, Unit of Public Health, Epidemiology and Biostatistics, School of Health and Population Sciences, University of Birmingham, Edgbaston, Birmingham B15 2TT, United Kingdom; e-mail, j.deeks@bham.ac.uk.

Current Author Addresses: Drs. Leeflang and Bossuyt: Department of Clinical Epidemiology, Biostatistics and Bioinformatics, Academic Medical Center, University of Amsterdam, PO Box 22700, 1100 DE Amsterdam, the Netherlands.

Dr. Deeks: Unit of Public Health, Epidemiology and Biostatistics, School of Health and Population Sciences, University of Birmingham, Edgbaston, Birmingham B15 2TT, United Kingdom.

Dr. Gatsonis: Center for Statistical Sciences, Brown University, Box G-S121, 121 South Main Street, 7th Floor, Providence, RI 02912.

Ann Intern Med. 2008;149(12):889-897. doi:10.7326/0003-4819-149-12-200812160-00008
Text Size: A A A

More and more systematic reviews of diagnostic test accuracy studies are being published, but they can be methodologically challenging. In this paper, the authors present some of the recent developments in the methodology for conducting systematic reviews of diagnostic test accuracy studies. Restrictive electronic search filters are discouraged, as is the use of summary quality scores. Methods for meta-analysis should take into account the paired nature of the estimates and their dependence on threshold. Authors of these reviews are advised to use the hierarchical summary receiver-operating characteristic or the bivariate model for the data analysis. Challenges that remain are the poor reporting of original diagnostic test accuracy studies and difficulties with the interpretation of the results of diagnostic test accuracy research.


Grahic Jump Location
Figure 1.
Review authors' judgments about quality items in a systematic review of magnetic resonance imaging for multiple sclerosis.

Data from reference (31). Data are presented as the proportion of included studies. Criteria that are unclear or not met introduce a risk for bias. The authors considered the relative lack of an acceptable reference standard as the main weakness of the review.

Grahic Jump Location
Grahic Jump Location
Figure 2.
Summary receiver-operating characteristic (ROC) curve plots showing test accuracy of a tumor marker for bladder cancer from 8 studies included in a systematic review.

Data from reference (10). Each study is represented by a small box positioned at the estimated sensitivity and specificity. The height and width of each box are proportional to the numbers of patients with and without bladder cancer, respectively, in each study. Top. This panel shows the summary ROC curve that can be drawn through these values. The scatter of the points fit, to a degree, with the existence of a threshold-type relationship between sensitivity and specificity. The curve is an estimate of the underlying relationship between sensitivity and specificity for the test used across varying thresholds. Bottom. This panel shows the average sensitivity and specificity estimate of the study results (solid circle) and a 95% confidence region around it. Estimation of a summary point only makes sense when the included studies have used a common threshold. The curves, points, and confidence regions can be estimated by using either the hierarchical summary ROC curve model (36, 5456) or the bivariate random-effects model (53, 57).

Grahic Jump Location
Grahic Jump Location
Figure 3.
Paired forest plot of the sensitivity and specificity of a tumor marker for bladder cancer.

FN = false-negative; FP = false-positive; TN = true-negative; TP = true-positive. Data are from reference (10). Forest plots document the extracted data for each study (numbers of TP, FP, FN, and TN results) together with estimates of sensitivity and specificity accompanied by 95% CIs. The scatter of the estimates and CIs indicates that the variability in sensitivity and specificity is unlikely to be explained by chance only, but it is not possible to ascertain whether a threshold-type relationship is evident.

Grahic Jump Location
Grahic Jump Location
Figure 4.
Meta-analysis of the diagnostic test accuracy of 2 index tests for bladder cancer: cytology (black squares) and bladder tumor antigen (green diamonds).

Data are from reference (10). The meta-analysis is restricted to studies that made a direct comparison between the tests by using both tests in each patient and comparing them with the reference standard of invasive cystoscopy. Restriction of the meta-analysis to direct test comparisons reduces concerns of confounding and allows stronger inferences to be drawn from the comparison of tests. The dashed lines link together the cytology and bladder tumor antigen results from each study and give the impression that bladder tumor antigen is much more sensitive but less specific than cytology. Top. Summary receiver-operating characteristic curves fitted to the data indicate that the bladder tumor antigen curve dominates the cytology curve as specificity decreases. Thus, bladder tumor antigen has the potential to be a more sensitive test than cytology, but only at specificities below 90%. Bottom. Cytology has an average sensitivity of 0.43 and an average specificity of 0.94 (black circle); bladder tumor antigen has an average sensitivity of 0.78 and an average specificity of 0.74 (green circle). The nonoverlapping 95% confidence regions indicate that the differences between the tests are unlikely to have occurred by chance alone.

Grahic Jump Location




Citing articles are presented as examples only. In non-demo SCM6 implementation, integration with CrossRef’s "Cited By" API will populate this tab (http://www.crossref.org/citedby.html).


Submit a Comment/Letter
Submit a Comment/Letter

Summary for Patients

Clinical Slide Sets

Terms of Use

The In the Clinic® slide sets are owned and copyrighted by the American College of Physicians (ACP). All text, graphics, trademarks, and other intellectual property incorporated into the slide sets remain the sole and exclusive property of the ACP. The slide sets may be used only by the person who downloads or purchases them and only for the purpose of presenting them during not-for-profit educational activities. Users may incorporate the entire slide set or selected individual slides into their own teaching presentations but may not alter the content of the slides in any way or remove the ACP copyright notice. Users may make print copies for use as hand-outs for the audience the user is personally addressing but may not otherwise reproduce or distribute the slides by any means or media, including but not limited to sending them as e-mail attachments, posting them on Internet or Intranet sites, publishing them in meeting proceedings, or making them available for sale or distribution in any unauthorized form, without the express written permission of the ACP. Unauthorized use of the In the Clinic slide sets will constitute copyright infringement.


Buy Now for $32.00

to gain full access to the content and tools.

Want to Subscribe?

Learn more about subscription options

Related Articles
Topic Collections
PubMed Articles
Forgot your password?
Enter your username and email address. We'll send you a reminder to the email address on record.