Tiffani J. Bright, PhD; Anthony Wong, MTech; Ravi Dhurjati, PhD; Erin Bristow, BA; Lori Bastian, MD, MS; Remy R. Coeytaux, MD, PhD; Gregory Samsa, PhD; Vic Hasselblad, PhD; John W. Williams, MD, MHS; Michael D. Musty, BA; Liz Wing, MA; Amy S. Kendrick, RN, MSN; Gillian D. Sanders, PhD; David Lobach, MD, PhD
Disclaimer: The authors of this report are responsible for its content. Statements in the report should not be construed as endorsements by the Agency for Healthcare Research and Quality or the U.S. Department of Health and Human Services.
Acknowledgment: The authors thank Connie Schardt, MSLS, for help with the literature search and retrieval.
Grant Support: This project was funded under contract 290-2007-10066-I from the Agency for Healthcare Research and Quality, U.S. Department of Health and Human Services.
Potential Conflicts of Interest: Disclosures can be viewed at www.acponline.org/authors/icmje/ConflictOfInterestForms.do?msNum=M11-1215.
Requests for Single Reprints: Gillian D. Sanders, PhD, Evidence-based Practice Center, Director, Duke Clinical Research Institute, 2400 Pratt Street, Durham, NC 27705; e-mail, email@example.com.
Current Author Addresses: Dr. Bright: 16 Kenilworth Drive, Hampton, VA 23666.
Mr. Wong: 2618 Briar Trail, Apartment 202, Schaumburg, IL 60173.
Dr. Dhurjati: D330-1 Mayo (MMC 729), 420 Delaware Street SE, University of Minnesota, Minneapolis, MN 55455.
Ms. Bristow: 728 Irolo Street, Apartment D, Los Angeles, CA 90005.
Dr. Bastian: Health Services Research & Development, 152 Veterans Affairs Medical Center, 508 Fulton Street, Durham, NC 27705.
Drs. Coeytaux, Hasselblad, Williams, and Sanders; Mr. Musty; Ms. Wing; and Ms. Kendrick: Duke Evidence-based Practice Center, Duke Clinical Research Institute, 2400 Pratt Street, Durham, NC 27705.
Dr. Samsa: Department of Biostatistics and Bioinformatics, Duke University School of Medicine, Durham, NC 27705.
Dr. Lobach: Klesis LLC, 6 Harvey Place, Durham, NC 27705.
Author Contributions: Conception and design: T.J. Bright, A. Wong, J.W. Williams, G.D. Sanders, D. Lobach.
Analysis and interpretation of the data: T.J. Bright, A. Wong, R. Dhurjati, G. Samsa, V. Hasselblad, G.D. Sanders, D. Lobach.
Drafting of the article: T.J. Bright, A. Wong, E. Bristow, G.D. Sanders, D. Lobach.
Critical revision of the article for important intellectual content: T.J. Bright, G. Samsa, G.D. Sanders, D. Lobach.
Final approval of the article: T.J. Bright, G. Samsa, G.D. Sanders, D. Lobach.
Provision of study materials or patients: T.J. Bright, M.D. Musty.
Statistical expertise: G. Samsa, V. Hasselblad.
Obtaining of funding: G.D. Sanders, D. Lobach.
Administrative, technical, or logistic support: T.J. Bright, M.D. Musty, L. Wing, A.S. Kendrick, G.D. Sanders.
Collection and assembly of data: T.J. Bright, A. Wong, R. Dhurjati, E. Bristow, L. Bastian, R.R. Coeytaux, M.D. Musty.
Despite increasing emphasis on the role of clinical decision-support systems (CDSSs) for improving care and reducing costs, evidence to support widespread use is lacking.
To evaluate the effect of CDSSs on clinical outcomes, health care processes, workload and efficiency, patient satisfaction, cost, and provider use and implementation.
MEDLINE, CINAHL, PsycINFO, and Web of Science through January 2011.
Investigators independently screened reports to identify randomized trials published in English of electronic CDSSs that were implemented in clinical settings; used by providers to aid decision making at the point of care; and reported clinical, health care process, workload, relationship-centered, economic, or provider use outcomes.
Investigators extracted data about study design, participant characteristics, interventions, outcomes, and quality.
148 randomized, controlled trials were included. A total of 128 (86%) assessed health care process measures, 29 (20%) assessed clinical outcomes, and 22 (15%) measured costs. Both commercially and locally developed CDSSs improved health care process measures related to performing preventive services (n = 25; odds ratio [OR], 1.42 [95% CI, 1.27 to 1.58]), ordering clinical studies (n = 20; OR, 1.72 [CI, 1.47 to 2.00]), and prescribing therapies (n = 46; OR, 1.57 [CI, 1.35 to 1.82]). Few studies measured potential unintended consequences or adverse effects.
Studies were heterogeneous in interventions, populations, settings, and outcomes. Publication bias and selective reporting cannot be excluded.
Both commercially and locally developed CDSSs are effective at improving health care process measures across diverse settings, but evidence for clinical, economic, workload, and efficiency outcomes remains sparse. This review expands knowledge in the field by demonstrating the benefits of CDSSs outside of experienced academic centers.
Agency for Healthcare Research and Quality.
Summary of evidence search and selection.
CDSS = clinical decision-support system; KQ = key question; RCT = randomized, controlled trial.
Appendix Table 1.
Summary of Evidence, by Outcome
Appendix Table 2.
Examples of Clinical Decision-Support Interventions
Results of studies that examined whether recommended preventive care services were ordered.
Studies reporting the odds ratio of adhering to recommendations for ordering or completing preventive care services of CDSS vs. control groups. In the 25 studies comparing CDSS with control groups, the random-effects–combined odds ratio of adherence to preventive care recommendations was 1.42 (95% CI, 1.27 to 1.58). CDSS = clinical decision-support system.
Results of studies that examined whether recommended clinical studies were ordered.
Studies reporting the odds ratio of adhering to recommendations for ordering or completing recommended clinical studies of CDSS vs. control groups. In the 20 studies comparing CDSS with control groups, the random-effects–combined odds ratio of adherence to clinical study recommendations was 1.72 (95% CI, 1.47 to 2.00). CDSS = clinical decision-support system.
Results of studies that examined whether recommended treatments were ordered.
Studies reporting the odds ratio of adhering to recommendations for ordering or prescribing treatment of CDSS vs. control groups. In the 46 studies comparing CDSS with control groups, the random-effects–combined odds ratio of adherence to treatment recommendations was 1.57 (95% CI, 1.35 to 1.82). CDSS = clinical decision-support system.
The In the Clinic® slide sets are owned and copyrighted by the American College of Physicians (ACP). All text, graphics, trademarks, and other intellectual property incorporated into the slide sets remain the sole and exclusive property of the ACP. The slide sets may be used only by the person who downloads or purchases them and only for the purpose of presenting them during not-for-profit educational activities. Users may incorporate the entire slide set or selected individual slides into their own teaching presentations but may not alter the content of the slides in any way or remove the ACP copyright notice. Users may make print copies for use as hand-outs for the audience the user is personally addressing but may not otherwise reproduce or distribute the slides by any means or media, including but not limited to sending them as e-mail attachments, posting them on Internet or Intranet sites, publishing them in meeting proceedings, or making them available for sale or distribution in any unauthorized form, without the express written permission of the ACP. Unauthorized use of the In the Clinic slide sets will constitute copyright infringement.
Edward Hoffer MD, FACP
July 9, 2012
To the Editor
The article by Bright et al on Clinical Decision-Support Systems (CDSS) (1) was timely and interesting, but had an important omission. No mention was made of CDSS that focused on helping doctors make the correct diagnosis. Such systems have been broadly available for some 25 years, including QMR/Internist-I, ILIAD and DXplain. Currently, at least two general-purpose Diagnostic Decision Support systems, DXplain and ISABEL, are widely available. Reviews (2,3) have shown that these systems suggest important diseases the clinician had not previously considered, and a study (4) from the Mayo Clinic found that use of DXplain lowered length-of-stay and hospital costs.
1. Bright TJ et al “Effect of Clinical Decision-Support Systems” Ann Intern Med 2012;157:29-432.
2. Berner ES et al “Performance of Four Computer-Based Diagnostic Systems” New Engl J Med 1994; 330:1792-63.
3. Bond WF et al “Differential Diagnosis Generators: an Evaluation of Currently Available Computer Programs” J Gen Intern Med 2011;27:213-94.
4. Elkin PL et al “The Introduction of a Diagnostic Decision Support System (DXplain) into the Workflow of a Teaching Hospital Service can Decrease the Cost of Service for Diagnostically Challenging DRGs” Internat Jl Med Informatics 2010; 79:772-
Bright TJ, Wong A, Dhurjati R, et al. Effect of Clinical Decision-Support Systems: A Systematic Review. Ann Intern Med. 2012;157:29–43. doi: https://doi.org/10.7326/0003-4819-157-1-201207030-00450
Download citation file:
Published: Ann Intern Med. 2012;157(1):29-43.
Results provided by:
Copyright © 2019 American College of Physicians. All Rights Reserved.
Print ISSN: 0003-4819 | Online ISSN: 1539-3704
Conditions of Use