0
Reviews |

Effect of Clinical Decision-Support Systems: A Systematic Review FREE

Tiffani J. Bright, PhD; Anthony Wong, MTech; Ravi Dhurjati, PhD; Erin Bristow, BA; Lori Bastian, MD, MS; Remy R. Coeytaux, MD, PhD; Gregory Samsa, PhD; Vic Hasselblad, PhD; John W. Williams, MD, MHS; Michael D. Musty, BA; Liz Wing, MA; Amy S. Kendrick, RN, MSN; Gillian D. Sanders, PhD; and David Lobach, MD, PhD
[+] Article and Author Information

From Duke Evidence-based Practice Center, Duke Clinical Research Institute, Duke University School of Medicine, Durham, North Carolina, and School of Medicine, University of Utah, Salt Lake City, Utah.

Disclaimer: The authors of this report are responsible for its content. Statements in the report should not be construed as endorsements by the Agency for Healthcare Research and Quality or the U.S. Department of Health and Human Services.

Acknowledgment: The authors thank Connie Schardt, MSLS, for help with the literature search and retrieval.

Grant Support: This project was funded under contract 290-2007-10066-I from the Agency for Healthcare Research and Quality, U.S. Department of Health and Human Services.

Potential Conflicts of Interest: Disclosures can be viewed at www.acponline.org/authors/icmje/ConflictOfInterestForms.do?msNum=M11-1215.

Requests for Single Reprints: Gillian D. Sanders, PhD, Evidence-based Practice Center, Director, Duke Clinical Research Institute, 2400 Pratt Street, Durham, NC 27705; e-mail, gillian.sanders@duke.edu.

Current Author Addresses: Dr. Bright: 16 Kenilworth Drive, Hampton, VA 23666.

Mr. Wong: 2618 Briar Trail, Apartment 202, Schaumburg, IL 60173.

Dr. Dhurjati: D330-1 Mayo (MMC 729), 420 Delaware Street SE, University of Minnesota, Minneapolis, MN 55455.

Ms. Bristow: 728 Irolo Street, Apartment D, Los Angeles, CA 90005.

Dr. Bastian: Health Services Research & Development, 152 Veterans Affairs Medical Center, 508 Fulton Street, Durham, NC 27705.

Drs. Coeytaux, Hasselblad, Williams, and Sanders; Mr. Musty; Ms. Wing; and Ms. Kendrick: Duke Evidence-based Practice Center, Duke Clinical Research Institute, 2400 Pratt Street, Durham, NC 27705.

Dr. Samsa: Department of Biostatistics and Bioinformatics, Duke University School of Medicine, Durham, NC 27705.

Dr. Lobach: Klesis LLC, 6 Harvey Place, Durham, NC 27705.

Author Contributions: Conception and design: T.J. Bright, A. Wong, J.W. Williams, G.D. Sanders, D. Lobach.

Analysis and interpretation of the data: T.J. Bright, A. Wong, R. Dhurjati, G. Samsa, V. Hasselblad, G.D. Sanders, D. Lobach.

Drafting of the article: T.J. Bright, A. Wong, E. Bristow, G.D. Sanders, D. Lobach.

Critical revision of the article for important intellectual content: T.J. Bright, G. Samsa, G.D. Sanders, D. Lobach.

Final approval of the article: T.J. Bright, G. Samsa, G.D. Sanders, D. Lobach.

Provision of study materials or patients: T.J. Bright, M.D. Musty.

Statistical expertise: G. Samsa, V. Hasselblad.

Obtaining of funding: G.D. Sanders, D. Lobach.

Administrative, technical, or logistic support: T.J. Bright, M.D. Musty, L. Wing, A.S. Kendrick, G.D. Sanders.

Collection and assembly of data: T.J. Bright, A. Wong, R. Dhurjati, E. Bristow, L. Bastian, R.R. Coeytaux, M.D. Musty.


Ann Intern Med. 2012;157(1):29-43. doi:10.7326/0003-4819-157-1-201207030-00450
Text Size: A A A

Background: Despite increasing emphasis on the role of clinical decision-support systems (CDSSs) for improving care and reducing costs, evidence to support widespread use is lacking.

Purpose: To evaluate the effect of CDSSs on clinical outcomes, health care processes, workload and efficiency, patient satisfaction, cost, and provider use and implementation.

Data Sources: MEDLINE, CINAHL, PsycINFO, and Web of Science through January 2011.

Study Selection: Investigators independently screened reports to identify randomized trials published in English of electronic CDSSs that were implemented in clinical settings; used by providers to aid decision making at the point of care; and reported clinical, health care process, workload, relationship-centered, economic, or provider use outcomes.

Data Extraction: Investigators extracted data about study design, participant characteristics, interventions, outcomes, and quality.

Data Synthesis: 148 randomized, controlled trials were included. A total of 128 (86%) assessed health care process measures, 29 (20%) assessed clinical outcomes, and 22 (15%) measured costs. Both commercially and locally developed CDSSs improved health care process measures related to performing preventive services (n = 25; odds ratio [OR], 1.42 [95% CI, 1.27 to 1.58]), ordering clinical studies (n = 20; OR, 1.72 [CI, 1.47 to 2.00]), and prescribing therapies (n = 46; OR, 1.57 [CI, 1.35 to 1.82]). Few studies measured potential unintended consequences or adverse effects.

Limitations: Studies were heterogeneous in interventions, populations, settings, and outcomes. Publication bias and selective reporting cannot be excluded.

Conclusion: Both commercially and locally developed CDSSs are effective at improving health care process measures across diverse settings, but evidence for clinical, economic, workload, and efficiency outcomes remains sparse. This review expands knowledge in the field by demonstrating the benefits of CDSSs outside of experienced academic centers.

Primary Funding Source: Agency for Healthcare Research and Quality.


Despite increasing emphasis on clinical decision-support systems (CDSSs) in improving care and reducing costs, evidence supporting widespread use is limited. A CDSS is “any electronic system designed to aid directly in clinical decision making, in which characteristics of individual patients are used to generate patient-specific assessments or recommendations that are then presented to clinicians for consideration” (1). This review examines 3 types of decision-support interventions.

Classic CDSSs include alerts, reminders, order sets, drug-dose calculations that automatically remind the clinician of a specific action, or care summary dashboards that provide performance feedback on quality indicators. Information retrieval tools, such as an “infobutton” embedded in a clinical information system, are designed to aid clinicians in the search and retrieval of context-specific knowledge from information sources based on patient-specific information from a clinical information system. Knowledge resources, such as UpToDate, Epocrates, and MD Consult, consist of distilled primary literature that allows selection of content germane to a specific patient to facilitate decision making at the point of care or for a specific care situation.

Until recently, most studies of CDSSs came from 4 benchmark settings (Brigham and Women's Hospital/Partners HealthCare, the Department of Veterans Affairs, LDS Hospital/Intermountain Healthcare, and the Regenstrief Institute) (2). Although several reviews have examined the effects of CDSSs (1, 39), many questions remain about their impact. This systematic review adds to the literature by summarizing trials of CDSSs implemented in a clinical setting to aid decision making at the point of care or for a specific care situation.

We developed and followed a standard protocol for our review. Full details of our methods, search strategies, results, and conclusions are provided in a report commissioned by the Agency for Healthcare Research and Quality, available at www.effectivehealthcare.ahrq.gov.

Data Sources and Searches

We searched for studies done between January 1976 and January 2011 in MEDLINE accessed through PubMed, CINAHL, PsycINFO, and Web of Science.

Study Selection

We identified randomized trials of CDSSs implemented in a real clinical setting and used by health care providers to aid decision making at the point of care or for a specific care situation. Studies had to report at least one of the following types of outcomes: clinical (length of stay, morbidity, mortality, health-related quality of life, and adverse events), health care process (recommended preventive care, clinical study, or treatment ordered or completed), user workload and efficiency (user knowledge, number of patients seen, clinician workload, and efficiency), relationship-centered (patient satisfaction), economic (cost and cost-effectiveness), or use and implementation by a health care provider (acceptance, satisfaction, use, and implementation). We excluded studies that described nonelectronic CDSSs, included fewer than 50 participants, were not published in English, described closed-loop systems that did not involve a provider, evaluated systems that required mandatory compliance with the CDSS, or evaluated only the performance of the system as opposed to its effect on clinical practice.

Data Extraction and Quality Assessment

Data related to study setting and design, sample characteristics, intervention characteristics, comparators, and outcomes were extracted by 1 reviewer and confirmed by another. Two reviewers used a standardized approach to independently categorize the quality of individual studies as good, fair, or poor (10) and evaluated the overall strength of evidence for each outcome as high, moderate, low, or insufficient (11). Reviewers also identified issues related to study setting, interventions, and outcomes that limited applicability of evidence (10, 12).

Data Synthesis and Analysis

A priori–defined outcomes believed to be important in measuring the effect of CDSSs in improving clinical practice guided our synthesis process. Studies with a common outcome were grouped together to facilitate qualitative analysis. Quantitative analysis was done where 4 or more studies assessed the same outcome in the same manner, regardless of the specific CDSS intervention. Summary estimates were calculated by using the DerSimonian and Laird (13) random-effects model as implemented in Comprehensive Meta-analysis, version 2.2.055 (Biostat, Englewood, New Jersey).

Role of the Funding Source

Primary funding was provided by the Agency for Healthcare Research and Quality. The funding source formulated the initial study questions but otherwise had no role in the design, analysis, or interpretation of the data or in the decision to submit the manuscript for publication.

We screened 15 176 abstracts, evaluated 1407 full-text articles, and included 160 articles, representing 148 unique studies (Appendix Figure). Appendix Table 1 summarizes important characteristics of the studies and their quality rating. A total of 128 studies (86%) assessed health care process measures, 29 (20%) assessed clinical outcomes, and 22 (15%) measured costs. Many studies (n = 51) were performed in environments with established health information technology (IT); many were multisite studies involving multiple institutions (n = 46).

Grahic Jump Location
Appendix Figure.

Summary of evidence search and selection.

CDSS = clinical decision-support system; KQ = key question; RCT = randomized, controlled trial.

Grahic Jump Location
Table Jump PlaceholderAppendix Table 1.  

Study Characteristics

The Table summarizes the most important findings and the strength of supporting evidence for those findings, whereas Appendix Table 2 provides examples of interventions that assessed the outcomes of interest. Both commercially and locally developed CDSSs improved health care process measures related to performing preventive services, ordering clinical studies, and prescribing therapies. Few studies (n = 15) indicated conceptualization of potential unintended consequences or measured potential adverse effects of implementing decision-support tools.

Table Jump PlaceholderTable.  

Summary of Evidence, by Outcome

Table Jump PlaceholderAppendix Table 2.  

Examples of Clinical Decision-Support Interventions

Clinical Outcomes
Morbidity

Several studies assessed morbidity outcomes (1438), such as hospitalizations, Apgar scores, surgical site infections, cardiovascular events, colorectal cancer, deep venous thrombosis, and hypoglycemia events. Topics addressed included diagnosis (16, 21, 2930, 34), pharmacotherapy (15, 1822, 25, 28, 33), chronic disease management (14, 1920, 22, 2628, 3132, 36, 38), laboratory test ordering (19, 37), immunizations (19, 35), preventive care (17, 19, 2830, 37), and discharge planning (2324). Approximately 50% of the studies were performed in an academic setting.

Many studies evaluated locally developed interventions implemented in the ambulatory environment. Typical interventions were automatically delivered, system-initiated recommendations provided synchronously at the point of care to enable decision making during the health care provider–patient encounter. Three such interventions required a mandatory response (that is, required that the user respond to the given recommendation, whether that response was to accept or dismiss the recommendation or to modify the user's action) (1718, 37).

Comparators included usual care or no CDSS and the same CDSS with additional features. Limitations included short follow-up, low statistical power to detect important differences, and the potential for contamination of providers in control groups that improved because of knowledge of interventions.

Meta-analysis of these heterogeneous studies (n = 16) suggested that CDSSs improved morbidity outcomes (relative risk, 0.88 [95% CI, 0.80 to 0.96]). We rated this level of evidence as moderate. Most studies were good quality, and many of the interventions were evaluated in multiple institutions. However, the interventions were often paper-based or standalone systems implemented in academic or Veterans Affairs settings and were targeted toward a single condition.

Mortality

Seven studies (17, 2022, 33, 3940) reported mortality outcomes. Issues addressed included diagnosis (21), pharmacotherapy (2022, 33, 40), chronic disease management (20, 22), preventing deep venous thrombosis (17), and detecting and notifying clinicians of critical laboratory values (39). Most CDSSs were locally developed and integrated into a computerized physician order entry (CPOE) or electronic health record (EHR) system and had system-initiated recommendations delivered synchronously at the point of care that did not require a clinician response.

Interventions were evaluated against usual care or no CDSS, except for 2 studies (20, 22) that compared the same intervention with additional features. Limitations included small sample size, duration shorter than 1 year, and possible contamination of control providers.

Meta-analysis of these heterogeneous studies (n = 6) suggested no significant effect of CDSSs on mortality, although CIs were wide (odds ratio [OR], 0.79 [CI, 0.54 to 1.15]). Of note, 2 studies reported a significant reduction in mortality with use of CDSSs (20, 22). We rated the overall strength of evidence related to mortality outcomes as low. Most studies were conducted in a single academic or Veterans Affairs setting with a comprehensive, well-established health IT infrastructure.

Adverse Events

Five studies assessed the effectiveness of CDSSs in reducing or preventing adverse events (2324, 3942). Studies were mostly implemented in an academic, inpatient setting. Studies evaluated the effect of these interventions to improve the timing of warfarin therapy (41), improve discharge planning (2324), prevent adverse drug events (42), detect critical laboratory values (39), and detect potentially inappropriate or inadequate antimicrobial therapy (40).

Typical interventions were locally developed, were integrated into a CPOE or an EHR system, and automatically delivered system-imitated recommendations in real time to enable decision making during the provider–patient encounter. Only 1 study clearly required a mandatory response (39).

All of the CDSSs were evaluated against usual care or no CDSS. Limitations included evaluation at a single institution, evaluation periods less than 1 year, and potential improvement in physician performance because of their knowledge of the intervention.

Meta-analysis of these heterogeneous studies estimated a relative risk of 1.01 (CI, 0.90 to 1.14), and neither this summary nor any individual studies demonstrated a significant effect. We rated this level of evidence as low. Most studies were good quality, and 2 were conducted at multiple sites; however, these interventions primarily contained locally developed knowledge, and results may not be generalizable to nonteaching settings.

Health Care Process Measures
Recommended Preventive Care Service Ordered or Completed

Forty-three studies examined the effect of CDSSs on the rates of ordering or completing recommended preventive care services (17, 19, 2728, 35, 37, 4382). Most studies were conducted in the academic or ambulatory environment. Topics addressed included diagnosis (43, 47, 62, 71, 82), pharmacotherapy (19, 28, 49, 5354, 70, 77, 80), chronic disease management (19, 26, 28, 4344, 48, 51, 70, 72, 74, 76), laboratory test ordering (19, 37, 55, 60, 7071, 7475, 8081), preventive care (17, 19, 28, 37, 43, 45, 48, 5260, 6265, 6871, 7375, 7982), immunizations (19, 35, 4850, 52, 56, 6667, 7879, 81), and initiating discussions with patients (6061, 72).

Most interventions were locally developed, paper-based, or standalone systems and automatically delivered recommendations in real time to enable decision making during the health care provider–patient encounter. Only 7 of the interventions required a mandatory response (17, 37, 47, 49) or justification (6465, 6869, 75) for not adhering to the recommendation.

Comparators included usual care or no CDSS, direct comparison against the same CDSS with additional features, or comparison of the same CDSS for different conditions. Limitations included sparse data measuring patient or economic outcomes; few assessments of long-term outcomes of interventions; and the Hawthorne effect, which probably stimulated more comprehensive preventive care across groups.

In meta-analysis of these 25 heterogeneous studies (17, 2728, 35, 37, 4363), the effect of CDSSs on preventive care services was significant (OR, 1.42 [CI, 1.27 to 1.58]) (Figure 1). We rated this level of evidence as high. Approximately one half of the studies were good quality, one third were evaluated in multicenter trials, and one fourth addressed multiple clinical conditions. However, most CDSSs were locally developed, not integrated into a CPOE or an EHR, and evaluated in academic medical centers, all of which can affect the generalizability of these findings.

Grahic Jump Location
Figure 1.

Results of studies that examined whether recommended preventive care services were ordered.

Studies reporting the odds ratio of adhering to recommendations for ordering or completing preventive care services of CDSS vs. control groups. In the 25 studies comparing CDSS with control groups, the random-effects–combined odds ratio of adherence to preventive care recommendations was 1.42 (95% CI, 1.27 to 1.58). CDSS = clinical decision-support system.

Grahic Jump Location
Recommended Clinical Study Ordered or Completed

Twenty-nine studies evaluated the effect of CDSSs on ordering and completing recommended clinical studies (26, 3132, 83109). Many studies were conducted in the academic setting, and most were evaluated in the ambulatory environment. Topics addressed included diagnosis (85, 8991, 95, 98, 101103), pharmacotherapy (94, 98), chronic disease management (26, 3132, 8485, 103), laboratory test ordering (83, 87, 89, 9294, 9697, 99101, 104108), initiating discussions with patients (108109), and additional clinical tasks (86, 88, 90, 109).

Typical interventions were locally developed, were integrated CDSS recommendations in a CPOE or an EHR system, and automatically delivered system-initiated recommendations to enable decision making during the provider–patient encounter. Eight interventions required a mandatory response (90, 99100, 104, 106, 108) or justification (83, 105) for not adhering to the recommendation.

Comparators included usual care or no CDSS, direct comparison against the same CDSS with additional features, or comparison of the same CDSS for different conditions. Limitations of the evidence base included diverse metrics to assess adherence to ordering and completing a recommended action, studies not designed to evaluate the clinical or economic outcomes associated with the CDSS interventions, and limited evidence of the effect of CDSSs on a broad set of conditions.

Meta-analysis of these 20 heterogeneous studies (26, 3132, 8384, 8889, 9192, 9496, 98103, 105, 108109) found a significant effect of CDSSs on ordering or completing of clinical studies (OR, 1.72 [CI, 1.47 to 2.00]) (Figure 2). We rated this level of evidence as moderate. Most studies were good quality, approximately one third were implemented in multiple sites, and almost one fourth included a direct comparison of the effectiveness of the CDSS against the same intervention with additional features. However, we noted a strong suggestion of publication bias in these studies, and these results may not be generalizable to all settings because most studies were either evaluated in environments with a well-established health IT infrastructure or conducted outside of the United States.

Grahic Jump Location
Figure 2.

Results of studies that examined whether recommended clinical studies were ordered.

Studies reporting the odds ratio of adhering to recommendations for ordering or completing recommended clinical studies of CDSS vs. control groups. In the 20 studies comparing CDSS with control groups, the random-effects–combined odds ratio of adherence to clinical study recommendations was 1.72 (95% CI, 1.47 to 2.00). CDSS = clinical decision-support system.

Grahic Jump Location
Recommended Treatment Ordered or Prescribed

Sixty-seven studies evaluated the effect of CDSSs on ordering and prescribing therapy (14, 18, 2022, 2528, 33, 36, 3841, 4344, 5354, 70, 80, 82, 84, 88, 94, 98, 110154). Many studies were conducted in the academic setting, and most were evaluated in the ambulatory environment. Topics addressed included diagnosis (21, 43, 82, 98, 112, 123, 129, 138, 151152), pharmacotherapy (18, 2022, 25, 28, 33, 40, 5354, 70, 80, 94, 98, 110111, 114, 116117, 119, 122123, 125134, 136, 138150, 153154), laboratory test ordering (70, 80, 94, 110, 130), chronic disease management (14, 20, 22, 2628, 36, 38, 4344, 70, 84, 110, 112113, 115, 118, 120121, 123125, 132133, 135, 141), preventive care (28, 43, 5354, 70, 80, 82, 120, 148), and additional clinical tasks (39, 41, 82, 88, 110, 137, 151152).

Typical interventions were locally developed, were integrated into a CPOE or an EHR system, and automatically delivered system-initiated recommendations in real time to enable decision making during the provider–patient encounter. Eighteen CDSSs required a mandatory response (18, 39, 119, 122, 134, 139143, 147148, 151152) or justification (112113, 136137, 145) for not adhering to the recommendation. Limitations included inadequate follow-up periods to observe sustained results, sparse data demonstrating how changes in clinician ordering and prescribing led to improvements in clinical or economic outcomes, and study designs that did not capture the extent to which nonadherence with recommended therapy resulted in adverse events.

Meta-analysis of 46 heterogeneous studies (14, 18, 2022, 2528, 33, 36, 38, 40, 4344, 5354, 70, 82, 84, 94, 98, 110, 112117, 121124, 129130, 134136, 141144, 146, 148, 151153) showed that intervention providers with decision support were more likely to order the appropriate treatment or therapy (OR, 1.57 [CI, 1.35 to 1.82]) (Figure 3). We rated this level of evidence as high. Most studies were good quality, and most were evaluated in multisite trials. However, generalizability may be limited because most studies were implemented in the ambulatory environment, were evaluated in settings where clinicians were experienced EHR users or provided care in an established health IT infrastructure, and incorporated knowledge that was targeted toward specific conditions.

Grahic Jump Location
Figure 3.

Results of studies that examined whether recommended treatments were ordered.

Studies reporting the odds ratio of adhering to recommendations for ordering or prescribing treatment of CDSS vs. control groups. In the 46 studies comparing CDSS with control groups, the random-effects–combined odds ratio of adherence to treatment recommendations was 1.57 (95% CI, 1.35 to 1.82). CDSS = clinical decision-support system.

Grahic Jump Location
User Workload and Efficiency Outcomes

Evidence on the effect of CDSSs on clinician knowledge or improved confidence in managing patient care was insufficient (7172, 86, 155156). Seven studies examined the effect of CDSSs on efficiency (2324, 40, 106, 141, 155157). Limitations included contamination of clinicians in the control group that improved because of knowledge of the intervention, evaluation periods that were too brief to demonstrate an effect on efficiency, and small clinician sample sizes.

We rated the level of evidence as low. Most interventions contained locally developed knowledge, such as protocols or algorithms derived on the basis of local performance, quality, and outcome data not representative of other sites, and were evaluated in academic settings.

Economic Outcomes
Cost

Twenty-two studies reported costs (21, 2627, 3132, 36, 40, 43, 5354, 71, 83, 90, 106, 109, 113, 118, 130, 141, 158163). Objectives of the CDSSs included diagnosis (21, 43, 71, 90, 160), pharmacotherapy (21, 40, 5354, 130, 141), chronic disease management (2627, 3132, 36, 43, 113, 118, 141, 161162), laboratory test ordering (71, 83, 106, 130, 160, 163), preventive care (43, 5354, 71, 158159), initiating discussions with patients (109, 159), and additional clinical tasks (90, 109). One study reported reduced hospitalization expenses with CDSS use (3132), and 12 studies reported that use had a positive effect on costs compared with control groups and other non-CDSS groups (21, 27, 40, 5354, 83, 90, 106, 113, 130, 141, 159160).

Modest evidence from academic and community inpatient and ambulatory settings showed that locally and commercially developed CDSSs had lower treatment costs, total costs, and reduced costs compared with control groups and other non-CDSS intervention groups. Most studies were conducted in the academic ambulatory setting and evaluated locally developed, integrated CDSSs in CPOE or EHR systems that automatically delivered system-initiated recommendations synchronously at the point of care and did not require a mandatory clinician response.

Cost-Effectiveness

Six studies (5354, 5657, 78, 9596, 161162) examined the cost-effectiveness of CDSSs or their effect on cost-effectiveness of care. These demonstrated conflicting findings, with 3 studies suggesting that CDSSs were cost-effective (5354, 78, 95) and 3 reporting that CDSSs were not cost-effective (5657, 161162). Objectives included diagnosis (95), pharmacotherapy (5354), chronic disease management (161162), preventive care (5354, 5657), and immunizations (56, 78).

Use and Implementation Outcomes

Twenty-four studies assessed the effect of provider acceptance of CDSSs (19, 41, 55, 62, 75, 90, 105, 113, 119120, 136137, 145147, 149, 158159, 164170). Topics addressed included diagnosis (62, 90, 166), pharmacotherapy (19, 119, 136, 145147, 149, 164165), chronic disease management (19, 113, 120, 168170), laboratory test ordering (19, 55, 75, 105), preventive care (19, 55, 62, 75, 120, 147, 158159, 167), immunizations (19), initiating discussions with patients (159), and additional clinical tasks (41, 90, 137).

Comparators included usual care or no CDSS and direct comparison with the same CDSS with additional features. One half of the studies required a mandatory response (55, 90, 119, 147, 166) or justification (75, 105, 113, 120, 136137, 145) for not adhering to the recommendation; however, there was no significant effect on provider acceptance. Limitations included an inconsistent definition of provider acceptance; small sample sizes; and scarce data on clinical outcomes, such as morbidity, length of stay, or adverse events. We rated this level of evidence as low. Most of these studies were fair quality and were evaluated in academic medical settings with established health IT infrastructures and experienced EHR users, which may limit the generalizability of the findings.

Provider satisfaction with CDSSs was examined in 19 studies (14, 2325, 37, 43, 80, 86, 105, 109, 112, 119, 127128, 141, 151153, 155156, 158, 165). Topics addressed included diagnosis (43, 112, 151152), pharmacotherapy (25, 80, 119, 127128, 141, 153, 165), chronic disease management (14, 43, 112, 141), laboratory test ordering (37, 80, 105), preventive care (37, 43, 80), and initiating discussions with patients (109).

Comparators included usual care or no CDSS and direct comparison with the same CDSS with additional features. Seven CDSSs required a mandatory response (37, 119, 141, 151152, 155) or justification (105, 112) for not adhering to the recommendation. Limitations included the narrow assessment of the role of provider satisfaction with CDSSs on patient-specific outcomes and small sample sizes of clinicians.

Twelve studies demonstrated provider satisfaction with CDSSs (25, 37, 80, 109, 112, 119, 127128, 141, 155156, 158, 165); 4 showed a significant effect of satisfaction among intervention providers compared with control providers (109, 112, 155, 165). Provider dissatisfaction with CDSSs was also reported in 6 studies (14, 43, 86, 105, 151153). We rated this level of evidence as moderate. Most studies were good quality and evaluated CDSSs integrated into CPOE or EHR systems in multiple interventions outside of environments with an established and robust health IT. However, most CDSSs were locally developed and implemented in the ambulatory setting.

Seventeen studies examined provider use of CDSSs by using such metrics as the number of times the CDSS was accessed by the clinician or provided a recommendation to the clinician (51, 71, 80, 86, 107, 110, 117, 119, 123, 138, 142, 145, 156, 165, 168172). Objectives included diagnosis (71, 123, 138), pharmacotherapy (80, 110, 117, 119, 123, 138, 142, 145, 165), chronic disease management (51, 110, 123, 168172), laboratory test ordering (71, 80, 107, 110), preventive care (71, 80), and additional clinical tasks (86, 110, 156).

Comparators included usual care or no CDSS, direct comparison with the same CDSS with additional features, or comparison of the same CDSS for different conditions. Limitations included sparse data demonstrating how provider use translated into more appropriate patient care and small sample sizes of clinicians. We rated this level of evidence as low.

Among the 12 studies (80, 107, 117, 119, 123, 138, 145, 168172) that provided statistical data about provider use, 8 (80, 110, 119, 123, 145, 165, 168170) documented low use (<50% of the clinician's time or of patient visits) or that less than 50% of clinicians used the CDSS or received alerts to guide therapeutic action. Most of these studies were fair quality and evaluated locally developed interventions in multiple community and ambulatory settings. Additional results are available from the technical report at www.effectivehealthcare.ahrq.gov.

Our systematic review investigated the continuum of information support for clinical care, including traditional CDSSs, as well as information retrieval systems and knowledge resources developed for access at the point of care. Studies were primarily conducted outside of institutions with an established health IT infrastructure. Most interventions targeted specific medical conditions and were evaluated in single settings.

Clinical decision support had a favorable effect on prescribing treatments, facilitating preventive care services, and ordering clinical studies across diverse venues and systems. This finding contrasts with that of another review (2), which showed that most reports of successful CDSS implementation were based on locally developed systems at 4 sites.

Evidence demonstrating positive effects of CDSSs on clinical and economic outcomes remains surprisingly sparse, although this could be because of the relative difficulty of implementing randomized, controlled trials in real clinical settings, as well as the logistics of measuring the direct clinical effect of CDSSs. Evidence was also limited in showing an effect of CDSSs on clinical workload and efficiency. Furthermore, available evidence is insufficient to draw conclusions about the potential negative effect of implementing decision-support tools, which is necessary to truly fulfill the goal of evaluating these interventions and to better address implementation challenges (173).

Our findings are important in light of the increasing political interest and financial investment of the U.S. government in resources for health IT. Meaningful use of CDSSs needs to be objectively informed about the role that they can and should play in reshaping health care delivery. Further understanding is increasingly important to optimally define their role in the context of meaningful use for EHRs.

Our systematic review has several limitations. The heterogeneity of the studies limited general observations about CDSSs. We minimized this limitation in our meta-analyses by including studies that assessed the same outcome in the same manner. Although this investigation was a comprehensive review of randomized, controlled trials that provided the best evidence on CDSS effectiveness, these studies may provide less information about issues related to CDSS implementation, effect on workflow, and factors affecting usability.

Most studies (76%) evaluated the effectiveness of the intervention by using usual care rather than a direct comparator, which may contribute to more positive results. Finally, we acknowledge the possibility of selective reporting or publication bias. However, a recent review by Buntin and colleagues (173) found that 62% of included studies on health IT reported positive effects where the technology was associated with improvement in 1 or more aspects of care (173). Formal assessment by using funnel plots found no consistent bias for most outcomes, except for ordering or completing of clinical studies, where there was a strong suggestion of publication bias.

Significant research is still required to promote widespread use of CDSSs and to augment their clinical effectiveness. Future studies should investigate how to expand CDSS content to accommodate multiple comorbid conditions simultaneously and to determine which members of the care team should receive clinical decision support, what effect CDSSs have on clinical and economic outcomes, and how CDSSs can be most effectively integrated into workflow and deployed across diverse settings. Further work is also needed to understand how CDSSs can aid in the transformation of care delivery models, such as accountable care organizations and patient-centered medical homes; how to incorporate CDSSs into workflow tools, such as medical registries and provider–provider messaging capabilities; and how to integrate CDSSs with workflow-oriented quality improvement programs.

Promoting extensive use of CDSSs will require a better definition of the clinical decision-support infrastructure. Such infrastructure could include consistent underlying frameworks for describing CDSSs, such as the “Clinical Decision Support Five Rights” (174), to aid in the aggregation and synthesis of results; development and evaluation of models for porting CDSSs across settings; and improved identification of characteristics of the environment and workflow into which a CDSS is deployed, as well as characteristics of the intended users.

In summary, evidence demonstrated the efficacy of CDSSs on health care process outcomes across diverse settings by using both commercially and locally developed systems, but data showing an effect on clinical and economic outcomes were sparse. Broad penetration of clinical decision-support tools will require aggressively seeking a better understanding of what the right information is and when and how it should be delivered to the right person, and a critical examination of the unintended consequences of CDSS implementation.

Kawamoto K, Houlihan CA, Balas EA, Lobach DF.  Improving clinical practice using clinical decision support systems: a systematic review of trials to identify features critical to success. BMJ. 2005; 330:765.
PubMed
CrossRef
 
Chaudhry B, Wang J, Wu S, Maglione M, Mojica W, Roth E, et al..  Systematic review: impact of health information technology on quality, efficiency, and costs of medical care. Ann Intern Med. 2006; 144:742-52.
PubMed
 
Bryan C, Boren SA.  The use and effectiveness of electronic clinical decision support tools in the ambulatory/primary care setting: a systematic review of the literature. Inform Prim Care. 2008; 16:79-91.
PubMed
 
Garg AX, Adhikari NK, McDonald H, Rosas-Arellano MP, Devereaux PJ, Beyene J, et al..  Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: a systematic review. JAMA. 2005; 293:1223-38.
PubMed
CrossRef
 
Grimshaw J, Freemantle N, Wallace S, Russell I, Hurwitz B, Watt I, et al..  Developing and implementing clinical practice guidelines. Qual Health Care. 1995; 4:55-64.
PubMed
 
Hunt DL, Haynes RB, Hanna SE, Smith K.  Effects of computer-based clinical decision support systems on physician performance and patient outcomes: a systematic review. JAMA. 1998; 280:1339-46.
PubMed
 
Sintchenko V, Magrabi F, Tipper S.  Are we measuring the right end-points? Variables that affect the impact of computerised decision support on patient outcomes: a systematic review. Med Inform Internet Med. 2007; 32:225-40.
PubMed
 
Shojania KG, Jennings A, Mayhew A, Ramsay C, Eccles M, Grimshaw J.  Effect of point-of-care computer reminders on physician behaviour: a systematic review. Can Med Assoc J. 2010; 182:216-25.
CrossRef
 
Kaushal R, Shojania KG, Bates DW.  Effects of computerized physician order entry and clinical decision support systems on medication safety: a systematic review. Arch Intern Med. 2003; 163:1409-16.
PubMed
CrossRef
 
Agency for Healthcare Research and Quality.  Methods Guide for Effectiveness and Comparative Effectiveness Reviews. Rockville, MD: Agency for Healthcare Research and Quality; 2011. Accessed at www.effectivehealthcare.ahrq.gov/index.cfm/search-for-guides-reviews-and-reports/?pageaction=displayproduct&productid=318 on 24 January 2012.
 
Owens DK, Lohr KN, Atkins D, Treadwell JR, Reston JT, Bass EB, et al..  AHRQ series paper 5: grading the strength of a body of evidence when comparing medical interventions—agency for healthcare research and quality and the effective health-care program. J Clin Epidemiol. 2010; 63:513-23.
PubMed
CrossRef
 
Atkins D, Chang SM, Gartlehner G, Buckley DI, Whitlock EP, Berliner E, et al..  Assessing applicability when comparing medical interventions: AHRQ and the Effective Health Care Program. J Clin Epidemiol. 2011; 64:1198-207.
PubMed
CrossRef
 
DerSimonian R, Laird N.  Meta-analysis in clinical trials. Control Clin Trials. 1986; 7:177-88.
PubMed
CrossRef
 
McCowan C, Neville RG, Ricketts IW, Warner FC, Hoskins G, Thomas GE.  Lessons from a randomized controlled trial designed to evaluate computer decision support software to improve the management of asthma. Med Inform Internet Med. 2001; 26:191-201.
PubMed
 
Cavalcanti AB, Silva E, Pereira AJ, Caldeira-Filho M, Almeida FP, Westphal GA, et al..  A randomized controlled trial comparing a computer-assisted insulin infusion protocol with a strict and a conventional protocol for glucose control in critically ill patients. J Crit Care. 2009; 24:371-8.
PubMed
CrossRef
 
Kline JA, Zeitouni RA, Hernandez-Nino J, Jones AE.  Randomized trial of computerized quantitative pretest probability in low-risk chest pain patients: effect on safety and resource use. Ann Emerg Med. 2009; 53:727-35.e1.
PubMed
CrossRef
 
Kucher N, Koo S, Quiroz R, Cooper JM, Paterno MD, Soukonnikov B, et al..  Electronic alerts to prevent venous thromboembolism among hospitalized patients. N Engl J Med. 2005; 352:969-77.
PubMed
CrossRef
 
Zanetti G, Flanagan HL Jr, Cohn LH, Giardina R, Platt R.  Improvement of intraoperative antibiotic prophylaxis in prolonged cardiac surgery by automated alerts in the operating room. Infect Control Hosp Epidemiol. 2003; 24:13-6.
PubMed
CrossRef
 
McDonald CJ, Hui SL, Smith DM, Tierney WM, Cohen SJ, Weinberger M, et al..  Reminders to physicians from an introspective computer medical record. A two-year randomized trial. Ann Intern Med. 1984; 100:130-8.
PubMed
 
Roumie CL, Elasy TA, Greevy R, Griffin MR, Liu X, Stone WJ, et al..  Improving blood pressure control through provider education, provider alerts, and patient education: a cluster randomized trial. Ann Intern Med. 2006; 145:165-75.
PubMed
 
Paul M, Andreassen S, Tacconelli E, Nielsen AD, Almanasreh N, Frank U, et al., TREAT Study Group.  Improving empirical antibiotic treatment using TREAT, a computerized decision support system: cluster randomized trial. J Antimicrob Chemother. 2006; 58:1238-45.
PubMed
 
Ansari M, Shlipak MG, Heidenreich PA, Van Ostaeyen D, Pohl EC, Browner WS, et al..  Improving guideline adherence: a randomized trial evaluating strategies to increase beta-blocker use in heart failure. Circulation. 2003; 107:2799-804.
PubMed
CrossRef
 
Graumlich JF, Novotny NL, Nace GS, Aldag JC.  Patient and physician perceptions after software-assisted hospital discharge: cluster randomized trial. J Hosp Med. 2009; 4:356-63.
PubMed
CrossRef
 
Graumlich JF, Novotny NL, Stephen Nace G, Kaushal H, Ibrahim-Ali W, Theivanayagam S, et al..  Patient readmissions, emergency visits, and adverse events after software-assisted discharge from hospital: cluster randomized trial. J Hosp Med. 2009; 4:11-9.
PubMed
CrossRef
 
Heidenreich PA, Gholami P, Sahay A, Massie B, Goldstein MK.  Clinical reminders attached to echocardiography reports of patients with reduced left ventricular ejection fraction increase use of beta-blockers: a randomized trial. Circulation. 2007; 115:2829-34.
PubMed
CrossRef
 
Tierney WM, Overhage JM, Murray MD, Harris LE, Zhou XH, Eckert GJ, et al..  Can computer-generated evidence-based care suggestions enhance evidence-based management of asthma and chronic obstructive pulmonary disease? A randomized, controlled trial. Health Serv Res. 2005; 40:477-97.
PubMed
CrossRef
 
Tierney WM, Overhage JM, Murray MD, Harris LE, Zhou XH, Eckert GJ, et al..  Effects of computerized guidelines for managing heart disease in primary care. J Gen Intern Med. 2003; 18:967-76.
PubMed
CrossRef
 
Gilutz H, Novack L, Shvartzman P, Zelingher J, Bonneh DY, Henkin Y, et al..  Computerized community cholesterol control (4C): meeting the challenge of secondary prevention. Isr Med Assoc J. 2009; 11:23-9.
PubMed
 
Holt TA, Thorogood M, Griffiths F, Munday S.  Protocol for the ‘e-Nudge trial’: a randomised controlled trial of electronic feedback to reduce the cardiovascular risk of individuals in general practice [ISRCTN64828380]. Trials. 2006; 7:11.
PubMed
CrossRef
 
Holt TA, Thorogood M, Griffiths F, Munday S, Friede T, Stables D.  Automated electronic reminders to facilitate primary cardiovascular disease prevention: randomised controlled trial. Br J Gen Pract. 2010; 60:137-43.
PubMed
CrossRef
 
Khan S, Maclean CD, Littenberg B.  The effect of the Vermont Diabetes Information System on inpatient and emergency room use: results from a randomized trial. Health Outcomes Res Med. 2010; 1:61-e66.
PubMed
 
Maclean CD, Gagnon M, Callas P, Littenberg B.  The Vermont diabetes information system: a cluster randomized trial of a population based decision support system. J Gen Intern Med. 2009; 24:1303-10.
PubMed
CrossRef
 
Brier ME, Gaweda AE, Dailey A, Aronoff GR, Jacobs AA.  Randomized trial of model predictive control for improved anemia management. Clin J Am Soc Nephrol. 2010; 5:814-20.
PubMed
CrossRef
 
Hamilton E, Platt R, Gauthier R, McNamara H, Miner L, Rothenberg S, et al..  The effect of computer-assisted evaluation of labor on cesarean rates. J Healthc Qual. 2004; 26:37-44.
PubMed
CrossRef
 
McDonald CJ, Hui SL, Tierney WM.  Effects of computer reminders for influenza vaccination on morbidity during influenza epidemics. MD Comput. 1992; 9:304-12.
PubMed
 
Murray MD, Harris LE, Overhage JM, Zhou XH, Eckert GJ, Smith FE, et al..  Failure of computerized treatment suggestions to improve health outcomes of outpatients with uncomplicated hypertension: results of a randomized controlled trial. Pharmacotherapy. 2004; 24:324-37.
PubMed
CrossRef
 
Sequist TD, Zaslavsky AM, Marshall R, Fletcher RH, Ayanian JZ.  Patient and physician reminders to promote colorectal cancer screening: a randomized controlled trial. Arch Intern Med. 2009; 169:364-71.
PubMed
 
Subramanian U, Fihn SD, Weinberger M, Plue L, Smith FE, Udris EM, et al..  A controlled trial of including symptom data in computer-based care suggestions for managing patients with chronic heart failure. Am J Med. 2004; 116:375-84.
PubMed
CrossRef
 
Kuperman GJ, Teich JM, Tanasijevic MJ, Ma'Luf N, Rittenberg E, Jha A, et al..  Improving response to critical laboratory results with automation: results of a randomized controlled trial. J Am Med Inform Assoc. 1999; 6:512-22.
PubMed
CrossRef
 
McGregor JC, Weekes E, Forrest GN, Standiford HC, Perencevich EN, Furuno JP, et al..  Impact of a computerized clinical decision support system on reducing inappropriate antimicrobial use: a randomized controlled trial. J Am Med Inform Assoc. 2006; 13:378-84.
PubMed
CrossRef
 
Fihn SD, McDonell MB, Vermes D, Henikoff JG, Martin DC, Callahan CM, et al..  A computerized intervention to improve timing of outpatient follow-up: a multicenter randomized trial in patients treated with warfarin. National Consortium of Anticoagulation Clinics. J Gen Intern Med. 1994; 9:131-9.
PubMed
CrossRef
 
Gurwitz JH, Field TS, Rochon P, Judge J, Harrold LR, Bell CM, et al..  Effect of computerized provider order entry with clinical decision support on adverse drug events in the long-term care setting. J Am Geriatr Soc. 2008; 56:2225-33.
PubMed
CrossRef
 
Apkon M, Mattera JA, Lin Z, Herrin J, Bradley EH, Carbone M, et al..  A randomized outpatient trial of a decision-support information technology tool. Arch Intern Med. 2005; 165:2388-94.
PubMed
CrossRef
 
Bertoni AG, Bonds DE, Chen H, Hogan P, Crago L, Rosenberger E, et al..  Impact of a multifaceted intervention on cholesterol management in primary care practices: guideline adherence for heart health randomized trial. Arch Intern Med. 2009; 169:678-86.
PubMed
CrossRef
 
Burack RC, Gimotty PA, George J, McBride S, Moncrease A, Simon MS, et al..  How reminders given to patients and physicians affected pap smear use in a health maintenance organization: results of a randomized controlled trial. Cancer. 1998; 82:2391-400.
PubMed
CrossRef
 
Burack RC, Gimotty PA, Simon M, Moncrease A, Dews P.  The effect of adding Pap smear information to a mammography reminder system in an HMO: results of randomized controlled trial. Prev Med. 2003; 36:547-54.
PubMed
CrossRef
 
Cannon DS, Allen SN.  A comparison of the effects of computer and manual reminders on compliance with a mental health clinical practice guideline. J Am Med Inform Assoc. 2000; 7:196-203.
PubMed
CrossRef
 
Demakis JG, Beauchamp C, Cull WL, Denwood R, Eisen SA, Lofgren R, et al..  Improving residents' compliance with standards of ambulatory care: results from the VA Cooperative Study on Computerized Reminders. JAMA. 2000; 284:1411-6.
PubMed
CrossRef
 
Dexter PR, Perkins S, Overhage JM, Maharry K, Kohler RB, McDonald CJ.  A computerized reminder system to increase the use of preventive care for hospitalized patients. N Engl J Med. 2001; 345:965-70.
PubMed
CrossRef
 
Dexter PR, Perkins SM, Maharry KS, Jones K, McDonald CJ.  Inpatient computer-based standing orders vs physician reminders to increase influenza and pneumococcal vaccination rates: a randomized trial. JAMA. 2004; 292:2366-71.
PubMed
CrossRef
 
Eccles M, McColl E, Steen N, Rousseau N, Grimshaw J, Parkin D, et al..  Effect of computerised evidence based guidelines on management of asthma and angina in adults in primary care: cluster randomised controlled trial. BMJ. 2002; 325:941.
PubMed
CrossRef
 
Frank O, Litt J, Beilby J.  Opportunistic electronic reminders. Improving performance of preventive care in general practice. Aust Fam Physician. 2004; 33:87-90.
PubMed
 
Fretheim A, Aaserud M, Oxman AD.  Rational prescribing in primary care (RaPP): economic evaluation of an intervention to improve professional practice. PLoS Med. 2006; 3:216.
PubMed
CrossRef
 
Fretheim A, Oxman AD, Håvelsrud K, Treweek S, Kristoffersen DT, Bjørndal A.  Rational prescribing in primary care (RaPP): a cluster randomized trial of a tailored intervention. PLoS Med. 2006; 3:134.
PubMed
CrossRef
 
Litzelman DK, Dittus RS, Miller ME, Tierney WM.  Requiring physicians to respond to computerized reminders improves their compliance with preventive care protocols. J Gen Intern Med. 1993; 8:311-7.
PubMed
CrossRef
 
McDowell I, Newell C, Rosser W.  Comparison of three methods of recalling patients for influenza vaccination. CMAJ. 1986; 135:991-7.
PubMed
 
McDowell I, Newell C, Rosser W.  Computerized reminders to encourage cervical screening in family practice. J Fam Pract. 1989; 28:420-4.
PubMed
 
Overhage JM, Tierney WM, McDonald CJ.  Computer reminders to implement preventive care guidelines for hospitalized patients. Arch Intern Med. 1996; 156:1551-6.
PubMed
CrossRef
 
Price M.  Can hand-held computers improve adherence to guidelines? A (Palm) Pilot study of family doctors in British Columbia. Can Fam Physician. 2005; 51:1506-7.
PubMed
 
Taylor V, Thompson B, Lessler D, Yasui Y, Montano D, Johnson KM, et al..  A clinic-based mammography intervention targeting inner-city women. J Gen Intern Med. 1999; 14:104-11.
PubMed
CrossRef
 
Unrod M, Smith M, Spring B, DePue J, Redd W, Winkel G.  Randomized controlled trial of a computer-based, tailored intervention to increase smoking cessation counseling by primary care physicians. J Gen Intern Med. 2007; 22:478-84.
PubMed
CrossRef
 
Dykes PC, Carroll DL, Hurley A, Lipsitz S, Benoit A, Chang F, et al..  Fall prevention in acute care hospitals: a randomized trial. JAMA. 2010; 304:1912-8.
PubMed
CrossRef
 
Chambers CV, Balaban DJ, Carlson BL, Ungemack JA, Grasberger DM.  Microcomputer-generated reminders. Improving the compliance of primary care physicians with mammography screening guidelines. J Fam Pract. 1989; 29:273-80.
PubMed
 
Burack RC, Gimotty PA.  Promoting screening mammography in inner-city settings. The sustained effectiveness of computerized reminders in a randomized controlled trial. Med Care. 1997; 35:921-31.
PubMed
CrossRef
 
Burack RC, Gimotty PA, George J, Stengle W, Warbasse L, Moncrease A.  Promoting screening mammography in inner-city settings: a randomized controlled trial of computerized reminders as a component of a program to facilitate mammography. Med Care. 1994; 32:609-24.
PubMed
CrossRef
 
Fiks AG, Hunter KF, Localio AR, Grundmeier RW, Bryant-Stephens T, Luberti AA, et al..  Impact of electronic health record-based alerts on influenza vaccination for children with asthma. Pediatrics. 2009; 124:159-69.
PubMed
CrossRef
 
Flanagan JR, Doebbeling BN, Dawson J, Beekmann S.  Randomized study of online vaccine reminders in adult primary care. Proc AMIA Symp. 1999; 755-9.
PubMed
 
Fordham D, McPhee SJ, Bird JA, Rodnick JE, Detmer WM.  The Cancer Prevention Reminder System. MD Comput. 1990; 7:289-95.
PubMed
 
McPhee SJ, Bird JA, Jenkins CN, Fordham D.  Promoting cancer screening. A randomized, controlled trial of three interventions. Arch Intern Med. 1989; 149:1866-72.
PubMed
CrossRef
 
Gill JM, Chen YX, Glutting JJ, Diamond JJ, Lieberman MI.  Impact of decision support in electronic medical records on lipid management in primary care. Popul Health Manag. 2009; 12:221-6.
PubMed
CrossRef
 
Hobbs FD, Delaney BC, Carson A, Kenkre JE.  A prospective controlled trial of computerized decision support for lipid management in primary care. Fam Pract. 1996; 13:133-7.
PubMed
 
Holbrook A, Thabane L, Keshavjee K, Dolovich L, Bernstein B, Chan D, et al., COMPETE II Investigators.  Individualized electronic decision support and reminders to improve diabetes care in the community: COMPETE II randomized trial. CMAJ. 2009; 181:37-44.
PubMed
 
Kenealy T, Arroll B, Petrie KJ.  Patients and computers as reminders to screen for diabetes in family practice. Randomized-controlled trial. J Gen Intern Med. 2005; 20:916-21.
PubMed
CrossRef
 
Lobach DF, Hammond WE.  Development and evaluation of a Computer-Assisted Management Protocol (CAMP): improved compliance with care guidelines for diabetes mellitus. Proc Annu Symp Comput Appl Med Care. 1994; 787-91.
PubMed
 
Ornstein SM, Garr DR, Jenkins RG, Rust PF, Arnon A.  Computer-generated physician and patient reminders. Tools to improve population adherence to selected preventive services. J Fam Pract. 1991; 32:82-90.
PubMed
 
Peterson KA, Radosevich DM, O'Connor PJ, Nyman JA, Prineas RJ, Smith SA, et al..  Improving diabetes care in practice: findings from the TRANSLATE trial. Diabetes Care. 2008; 31:2238-43.
PubMed
CrossRef
 
Reeve JF, Tenni PC, Peterson GM.  An electronic prompt in dispensing software to promote clinical interventions by community pharmacists: a randomized controlled trial. Br J Clin Pharmacol. 2008; 65:377-85.
PubMed
CrossRef
 
Rosser WW, Hutchison BG, McDowell I, Newell C.  Use of reminders to increase compliance with tetanus booster vaccination. CMAJ. 1992; 146:911-7.
PubMed
 
Rosser WW, McDowell I, Newell C.  Use of reminders for preventive procedures in family medicine. CMAJ. 1991; 145:807-14.
PubMed
 
Sequist TD, Gandhi TK, Karson AS, Fiskio JM, Bugbee D, Sperling M, et al..  A randomized trial of electronic clinical reminders to improve quality of care for diabetes and coronary artery disease. J Am Med Inform Assoc. 2005; 12:431-7.
PubMed
CrossRef
 
Tierney WM, Hui SL, McDonald CJ.  Delayed feedback of physician performance versus immediate reminders to perform preventive care. Effects on physician compliance. Med Care. 1986; 24:659-66.
PubMed
CrossRef
 
van Wyk JT, van Wijk MA, Sturkenboom MC, Mosseveld M, Moorman PW, van der Lei J.  Electronic alerts versus on-demand decision support to improve dyslipidemia treatment: a cluster randomized controlled trial. Circulation. 2008; 117:371-8.
PubMed
CrossRef
 
Bates DW, Kuperman GJ, Rittenberg E, Teich JM, Fiskio J, Ma'luf N, et al..  A randomized trial of a computer-based intervention to reduce utilization of redundant laboratory tests. Am J Med. 1999; 106:144-50.
PubMed
CrossRef
 
Bell LM, Grundmeier R, Localio R, Zorc J, Fiks AG, Zhang X, et al..  Electronic health record-based decision support to improve asthma care: a cluster-randomized trial. Pediatrics. 2010; 125:770-7.
PubMed
CrossRef
 
Downs M, Turner S, Bryans M, Wilcock J, Keady J, Levin E, et al..  Effectiveness of educational interventions in improving detection and management of dementia in primary care: cluster randomised controlled study. BMJ. 2006; 332:692-6.
PubMed
CrossRef
 
Emery J, Morris H, Goodchild R, Fanshawe T, Prevost AT, Bobrow M, et al..  The GRAIDS Trial: a cluster randomised controlled trial of computer decision support for the management of familial cancer risk in primary care. Br J Cancer. 2007; 97:486-93.
PubMed
CrossRef
 
Feldstein AC, Smith DH, Perrin N, Yang X, Rix M, Raebel MA, et al..  Improved therapeutic monitoring with several interventions: a randomized trial. Arch Intern Med. 2006; 166:1848-54.
PubMed
 
Flottorp S, Oxman AD, Håvelsrud K, Treweek S, Herrin J.  Cluster randomised controlled trial of tailored interventions to improve the management of urinary tract infections in women and sore throat. BMJ. 2002; 325:367.
PubMed
CrossRef
 
Greiver M, Drummond N, White D, Weshler J, Moineddin R, North Toronto Primary Care Research Network (Nortren).  Angina on the Palm: randomized controlled pilot trial of Palm PDA software for referrals for cardiac testing. Can Fam Physician. 2005; 51:382-3.
PubMed
 
Harpole LH, Khorasani R, Fiskio J, Kuperman GJ, Bates DW.  Automated evidence-based critiquing of orders for abdominal radiographs: impact on utilization and appropriateness. J Am Med Inform Assoc. 1997; 4:511-21.
PubMed
CrossRef
 
Lee NJ, Chen ES, Currie LM, Donovan M, Hall EK, Jia H, et al..  The effect of a mobile clinical decision support system on the diagnosis of obesity and overweight in acute and primary care encounters. ANS Adv Nurs Sci. 2009; 32:211-21.
PubMed
 
Lo HG, Matheny ME, Seger DL, Bates DW, Gandhi TK.  Impact of non-interruptive medication laboratory monitoring alerts in ambulatory care. J Am Med Inform Assoc. 2009; 16:66-71.
PubMed
CrossRef
 
Matheny ME, Sequist TD, Seger AC, Fiskio JM, Sperling M, Bugbee D, et al..  A randomized trial of electronic clinical reminders to improve medication laboratory monitoring. J Am Med Inform Assoc. 2008; 15:424-9.
PubMed
CrossRef
 
McDonald CJ.  Use of a computer to detect and respond to clinical events: its effect on clinician behavior. Ann Intern Med. 1976; 84:162-7.
PubMed
 
McDowell I, Newell C, Rosser W.  A randomized trial of computerized reminders for blood pressure screening in primary care. Med Care. 1989; 27:297-305.
PubMed
CrossRef
 
Palen TE, Raebel M, Lyons E, Magid DM.  Evaluation of laboratory monitoring alerts within a computerized physician order entry system for medication orders. Am J Manag Care. 2006; 12:389-95.
PubMed
 
Palen TE, Price DW, Snyder AJ, Shetterly SM.  Computerized alert reduced D-dimer testing in the elderly. Am J Manag Care. 2010; 16:267-75.
PubMed
 
Player MS, Gill JM, Mainous AG 3rd, Everett CJ, Koopman RJ, Diamond JJ, et al..  An electronic medical record-based intervention to improve quality of care for gastro-esophageal reflux disease (GERD) and atypical presentations of GERD. Qual Prim Care. 2010; 18:223-9.
PubMed
 
Raebel MA, Lyons EE, Chester EA, Bodily MA, Kelleher JA, Long CL, et al..  Improving laboratory monitoring at initiation of drug therapy in ambulatory care: a randomized trial. Arch Intern Med. 2005; 165:2395-401.
PubMed
CrossRef
 
Raebel MA, Chester EA, Newsom EE, Lyons EE, Kelleher JA, Long C, et al..  Randomized trial to improve laboratory safety monitoring of ongoing drug therapy in ambulatory patients. Pharmacotherapy. 2006; 26:619-26.
PubMed
CrossRef
 
Roukema J, Steyerberg EW, van der Lei J, Moll HA.  Randomized trial of a clinical decision support system: impact on the management of children with fever without apparent source. J Am Med Inform Assoc. 2008; 15:107-13.
PubMed
 
Roy PM, Durieux P, Gillaizeau F, Legall C, Armand-Perroux A, Martino L, et al..  A computerized handheld decision-support system to improve pulmonary embolism diagnosis: a randomized trial. Ann Intern Med. 2009; 151:677-86.
PubMed
 
Schriefer SP, Landis SE, Turbow DJ, Patch SC.  Effect of a computerized body mass index prompt on diagnosis and treatment of adult obesity. Fam Med. 2009; 41:502-7.
PubMed
 
Stiell IG, Clement CM, Grimshaw J, Brison RJ, Rowe BH, Schull MJ, et al..  Implementation of the Canadian C-Spine Rule: prospective 12 centre cluster randomised trial. BMJ. 2009; 339:b4146.
PubMed
CrossRef
 
Sundaram V, Lazzeroni LC, Douglass LR, Sanders GD, Tempio P, Owens DK.  A randomized trial of computer-based reminders and audit and feedback to improve HIV screening in a primary care setting. Int J STD AIDS. 2009; 20:527-33.
PubMed
 
Tierney WM, McDonald CJ, Martin DK, Rogers MP.  Computerized display of past test results. Effect on outpatient testing. Ann Intern Med. 1987; 107:569-74.
PubMed
 
van Wijk MA, van der Lei J, Mosseveld M, Bohnen AM, van Bemmel JH.  Assessment of decision support for blood test ordering in primary care. a randomized trial. Ann Intern Med. 2001; 134:274-81.
PubMed
 
Walker J, Fairley CK, Walker SM, Gurrin LC, Gunn JM, Pirotta MV, et al..  Computer reminders for Chlamydia screening in general practice: a randomized controlled trial. Sex Transm Dis. 2010; 37:445-50.
PubMed
 
Wilson BJ, Torrance N, Mollison J, Watson MS, Douglas A, Miedzybrodzka Z, et al..  Cluster randomized trial of a multifaceted primary care decision-support intervention for inherited breast cancer risk. Fam Pract. 2006; 23:537-44.
PubMed
CrossRef
 
Bourgeois FC, Linder J, Johnson SA, Co JP, Fiskio J, Ferris TG.  Impact of a computerized template on antibiotic prescribing for acute respiratory infections in children and adolescents. Clin Pediatr (Phila). 2010; 49:976-83.
PubMed
CrossRef
 
Christakis DA, Zimmerman FJ, Wright JA, Garrison MM, Rivara FP, Davis RL.  A randomized controlled trial of point-of-care evidence to improve the antibiotic prescribing practices for otitis media in children. Pediatrics. 2001; 107:15.
PubMed
CrossRef
 
Co JP, Johnson SA, Poon EG, Fiskio J, Rao SR, Van Cleave J, et al..  Electronic health record decision support and quality of care for children with ADHD. Pediatrics. 2010; 126:239-46.
PubMed
CrossRef
 
Cobos A, Vilaseca J, Asenjo C, Pedro-Botet J, Sánchez E, Val A, et al..  Cost effectiveness of a clinical decision support system based on the recommendations of the European Society of Cardiology and other societies for the management of hypercholesterolemia: report of a cluster-randomized trial. Disease Management & Health Outcomes. 2005; 13:421-32.
CrossRef
 
Davis RL, Wright J, Chalmers F, Levenson L, Brown JC, Lozano P, et al..  A cluster randomized clinical trial to improve prescribing patterns in ambulatory pediatrics. PLoS Clin Trials. 2007; 2:25.
PubMed
CrossRef
 
Feldstein A, Elmer PJ, Smith DH, Herson M, Orwoll E, Chen C, et al..  Electronic medical record reminder improves osteoporosis management after a fracture: a randomized, controlled trial. J Am Geriatr Soc. 2006; 54:450-7.
PubMed
CrossRef
 
Field TS, Rochon P, Lee M, Gavendo L, Baril JL, Gurwitz JH.  Computerized clinical decision support during medication ordering for long-term care residents with renal insufficiency. J Am Med Inform Assoc. 2009; 16:480-5.
PubMed
CrossRef
 
Filippi A, Sabatini A, Badioli L, Samani F, Mazzaglia G, Catapano A, et al..  Effects of an automated electronic reminder in changing the antiplatelet drug-prescribing behavior among Italian general practitioners in diabetic patients: an intervention trial. Diabetes Care. 2003; 26:1497-500.
PubMed
CrossRef
 
Fitzmaurice DA, Hobbs FD, Murray ET, Holder RL, Allan TF, Rose PE.  Oral anticoagulation management in primary care with the use of computerized decision support and near-patient testing: a randomized, controlled trial. Arch Intern Med. 2000; 160:2343-8.
PubMed
CrossRef
 
Fortuna RJ, Zhang F, Ross-Degnan D, Campion FX, Finkelstein JA, Kotch JB, et al..  Reducing the prescribing of heavily marketed medications: a randomized controlled trial. J Gen Intern Med. 2009; 24:897-903.
PubMed
CrossRef
 
Goud R, de Keizer NF, ter Riet G, Wyatt JC, Hasman A, Hellemans IM, et al..  Effect of guideline based computerised decision support on decision making of multidisciplinary teams: cluster randomised trial in cardiac rehabilitation. BMJ. 2009; 338:b1440.
PubMed
CrossRef
 
Hicks LS, Sequist TD, Ayanian JZ, Shaykevich S, Fairchild DG, Orav EJ, et al..  Impact of computerized decision support on blood pressure management and control: a randomized controlled trial. J Gen Intern Med. 2008; 23:429-41.
PubMed
 
Krall MA, Traunweiser K, Towery W.  Effectiveness of an electronic medical record clinical quality alert prepared by off-line data analysis. Stud Health Technol Inform. 2004; 107:135-9.
PubMed
 
Linder JA, Rigotti NA, Schneider LI, Kelley JH, Brawarsky P, Haas JS.  An electronic health record-based intervention to improve tobacco treatment in primary care: a cluster-randomized controlled trial. Arch Intern Med. 2009; 169:781-7.
PubMed
CrossRef
 
Locatelli F, Covic A, Macdougall IC, Scherhag A, Wiecek A, ORAMA Study Group.  Effect of computer-assisted European Best Practice Guideline implementation on adherence and target attainment: ORAMA results. J Nephrol. 2009; 22:662-74.
PubMed
 
Manotti C, Moia M, Palareti G, Pengo V, Ria L, Dettori AG.  Effect of computer-aided management on the quality of treatment in anticoagulated patients: a prospective, randomized, multicenter trial of APROAT (Automated PRogram for Oral Anticoagulant Treatment). Haematologica. 2001; 86:1060-70.
PubMed
 
Marco F, Sedano C, Bermúdez A, López-Duarte M, Fernández-Fontecha E, Zubizarreta A.  A prospective controlled study of a computer-assisted acenocoumarol dosage program. Pathophysiol Haemost Thromb. 2003; 33:59-63.
PubMed
CrossRef
 
Martens JD, van der Aa A, Panis B, van der Weijden T, Winkens RA, Severens JL.  Design and evaluation of a computer reminder system to improve prescribing behaviour of GPs. Stud Health Technol Inform. 2006; 124:617-23.
PubMed
 
Martens JD, van der Weijden T, Severens JL, de Clercq PA, de Bruijn DP, Kester AD, et al..  The effect of computer reminders on GPs' prescribing behaviour: a cluster-randomised trial. Int J Med Inform. 2007; 76:Suppl 3S403-16.
PubMed
CrossRef
 
Montgomery AA, Fahey T, Peters TJ, MacIntosh C, Sharp DJ.  Evaluation of computer based clinical decision support system and risk chart for management of hypertension in primary care: randomised controlled trial. BMJ. 2000; 320:686-90.
PubMed
CrossRef
 
Overhage JM, Tierney WM, Zhou XH, McDonald CJ.  A randomized trial of “corollary orders” to prevent errors of omission. J Am Med Inform Assoc. 1997; 4:364-75.
PubMed
CrossRef
 
Peterson JF, Rosenbaum BP, Waitman LR, Habermann R, Powers J, Harrell D, et al..  Physicians' response to guided geriatric dosing: initial results from a randomized trial. Stud Health Technol Inform. 2007; 129:1037-40.
PubMed
 
Phillips LS, Ziemer DC, Doyle JP, Barnes CS, Kolm P, Branch WT, et al..  An endocrinologist-supported intervention aimed at providers improves diabetes management in a primary care site: improving primary care of African Americans with diabetes (IPCAAD) 7. Diabetes Care. 2005; 28:2352-60.
PubMed
CrossRef
 
Ziemer DC, Doyle JP, Barnes CS, Branch WT Jr, Cook CB, El-Kebbi IM, et al..  An intervention to overcome clinical inertia and improve diabetes mellitus control in a primary care setting: Improving Primary Care of African Americans with Diabetes (IPCAAD) 8. Arch Intern Med. 2006; 166:507-13.
PubMed
 
Raebel MA, Charles J, Dugan J, Carroll NM, Korner EJ, Brand DW, et al..  Randomized trial to improve prescribing safety in ambulatory elderly patients. J Am Geriatr Soc. 2007; 55:977-85.
PubMed
CrossRef
 
Rood E, Bosman RJ, van der Spoel JI, Taylor P, Zandstra DF.  Use of a computerized guideline for glucose regulation in the intensive care unit improved both guideline adherence and glucose regulation. J Am Med Inform Assoc. 2005; 12:172-80.
PubMed
 
Rossi RA, Every NR.  A computerized intervention to decrease the use of calcium channel blockers in hypertension. J Gen Intern Med. 1997; 12:672-8.
PubMed
CrossRef
 
Rothschild JM, McGurk S, Honour M, Lu L, McClendon AA, Srivastava P, et al..  Assessment of education and computerized decision support interventions for improving transfusion practice. Transfusion. 2007; 47:228-39.
PubMed
CrossRef
 
Samore MH, Bateman K, Alder SC, Hannah E, Donnelly S, Stoddard GJ, et al..  Clinical decision support and appropriateness of antimicrobial prescribing: a randomized trial. JAMA. 2005; 294:2305-14.
PubMed
CrossRef
 
Shojania KG, Yokoe D, Platt R, Fiskio J, Ma'luf N, Bates DW.  Reducing vancomycin use utilizing a computer guideline: results of a randomized controlled trial. J Am Med Inform Assoc. 1998; 5:554-62.
PubMed
CrossRef
 
Simon SR, Smith DH, Feldstein AC, Perrin N, Yang X, Zhou Y, et al..  Computerized prescribing alerts and group academic detailing to reduce the use of potentially inappropriate medications in older people. J Am Geriatr Soc. 2006; 54:963-8.
PubMed
CrossRef
 
Smith SA, Shah ND, Bryant SC, Christianson TJ, Bjornsen SS, Giesler PD, et al., Evidens Research Group.  Chronic care model and shared care in diabetes: randomized trial of an electronic decision support system. Mayo Clin Proc. 2008; 83:747-57.
PubMed
 
Strom BL, Schinnar R, Aberra F, Bilker W, Hennessy S, Leonard CE, et al..  Unintended effects of a computerized physician order entry nearly hard-stop alert to prevent a drug interaction: a randomized controlled trial. Arch Intern Med. 2010; 170:1578-83.
PubMed
CrossRef
 
Strom BL, Schinnar R, Bilker W, Hennessy S, Leonard CE, Pifer E.  Randomized clinical trial of a customized electronic alert requiring an affirmative response compared to a control group receiving a commercial passive CPOE alert: NSAID—warfarin co-prescribing as a test case. J Am Med Inform Assoc. 2010; 17:411-5.
PubMed
CrossRef
 
Tamblyn R, Huang A, Perreault R, Jacques A, Roy D, Hanley J, et al..  The medical office of the 21st century (MOXXI): effectiveness of computerized decision-making support in reducing inappropriate prescribing in primary care. CMAJ. 2003; 169:549-56.
PubMed
 
Tamblyn R, Huang A, Taylor L, Kawasumi Y, Bartlett G, Grad R, et al..  A randomized trial of the effectiveness of on-demand versus computer-triggered drug decision support in primary care. J Am Med Inform Assoc. 2008; 15:430-8.
PubMed
CrossRef
 
Tamblyn R, Reidel K, Huang A, Taylor L, Winslade N, Bartlett G, et al..  Increasing the detection and response to adherence problems with cardiovascular medication in primary care through computerized drug management systems: a randomized controlled trial. Med Decis Making. 2010; 30:176-88.
PubMed
CrossRef
 
Terrell KM, Perkins AJ, Dexter PR, Hui SL, Callahan CM, Miller DK.  Computerized decision support to reduce potentially inappropriate prescribing to older emergency department patients: a randomized, controlled trial. J Am Geriatr Soc. 2009; 57:1388-94.
PubMed
CrossRef
 
Terrell KM, Perkins AJ, Hui SL, Callahan CM, Dexter PR, Miller DK.  Computerized decision support for medication dosing in renal insufficiency: a randomized, controlled trial. Ann Emerg Med. 2010; 56:623-9.
PubMed
CrossRef
 
Vadher BD, Patterson DL, Leaning M.  Comparison of oral anticoagulant control by a nurse-practitioner using a computer decision-support system with that by clinicians. Clin Lab Haematol. 1997; 19:203-7.
PubMed
CrossRef
 
Vadher B, Patterson DL, Leaning M.  Evaluation of a decision support system for initiation and control of oral anticoagulation in a randomised trial. BMJ. 1997; 314:1252-6.
PubMed
CrossRef
 
Vissers MC, Biert J, van der Linden CJ, Hasman A.  Effects of a supportive protocol processing system (ProtoVIEW) on clinical behaviour of residents in the accident and emergency department. Comput Methods Programs Biomed. 1996; 49:177-84.
PubMed
CrossRef
 
Vissers MC, Hasman A, van der Linden CJ.  Protocol processing system (ProtoVIEW) to support residents at the emergency ward. Comput Methods Programs Biomed. 1995; 48:53-8.
PubMed
CrossRef
 
Weir CJ, Lees KR, MacWalter RS, Muir KW, Wallesch CW, McLelland EV, et al., PRISM Study Group.  Cluster-randomized, controlled trial of computer-based decision support for selecting long-term anti-thrombotic therapy after acute ischaemic stroke. QJM. 2003; 96:143-53.
PubMed
CrossRef
 
White KS, Lindsay A, Pryor TA, Brown WF, Walsh K.  Application of a computerized medical decision-making process to the problem of digoxin intoxication. J Am Coll Cardiol. 1984; 4:571-6.
PubMed
CrossRef
 
Alper BS, White DS, Ge B.  Physicians answer more clinical questions and change clinical decisions more often with synthesized evidence: a randomized trial in primary care. Ann Fam Med. 2005; 3:507-13.
PubMed
CrossRef
 
Del Fiol G, Haug PJ, Cimino JJ, Narus SP, Norlin C, Mitchell JA.  Effectiveness of topic-specific infobuttons: a randomized controlled trial. J Am Med Inform Assoc. 2008; 15:752-9.
PubMed
CrossRef
 
Etchells E, Adhikari NK, Cheung C, Fowler R, Kiss A, Quan S, et al..  Real-time clinical alerting: effect of an automated paging system on response time to critical laboratory values—a randomised controlled trial. Qual Saf Health Care. 2010; 19:99-102.
PubMed
 
Bird JA, McPhee SJ, Jenkins C, Fordham D.  Three strategies to promote cancer screening. How feasible is wide-scale implementation? Med Care. 1990; 28:1005-12.
PubMed
CrossRef
 
Frame PS, Zimmer JG, Werth PL, Hall WJ, Eberly SW.  Computer-based vs manual health maintenance tracking. A controlled trial. Arch Fam Med. 1994; 3:581-8.
PubMed
CrossRef
 
Tierney WM, McDonald CJ, Hui SL, Martin DK.  Computer predictions of abnormal test results. Effects on outpatient testing. JAMA. 1988; 259:1194-8.
PubMed
CrossRef
 
Cleveringa FG, Gorter KJ, van den Donk M, Rutten GE.  Combined task delegation, computerized decision support, and feedback improve cardiovascular risk for type 2 diabetic patients: a cluster randomized trial in primary care. Diabetes Care. 2008; 31:2273-5.
PubMed
CrossRef
 
Cleveringa FG, Welsing PM, van den Donk M, Gorter KJ, Niessen LW, Rutten GE, et al..  Cost-effectiveness of the diabetes care protocol, a multifaceted computerized decision support diabetes management intervention that reduces cardiovascular risk. Diabetes Care. 2010; 33:258-63.
PubMed
CrossRef
 
Smith DH, Feldstein AC, Perrin NA, Yang X, Rix MM, Raebel MA, et al..  Improving laboratory monitoring of medications: an economic analysis alongside a clinical trial. Am J Manag Care. 2009; 15:281-9.
PubMed
 
Judge J, Field TS, DeFlorio M, Laprino J, Auger J, Rochon P, et al..  Prescribers' responses to alerts during medication ordering in the long term care setting. J Am Med Inform Assoc. 2006; 13:385-90.
PubMed
CrossRef
 
Maviglia SM, Yoon CS, Bates DW, Kuperman G.  KnowledgeLink: impact of context-sensitive information retrieval on clinicians' information needs. J Am Med Inform Assoc. 2006; 13:67-73.
PubMed
CrossRef
 
Rollman BL, Hanusa BH, Gilbert T, Lowe HJ, Kapoor WN, Schulberg HC.  The electronic medical record. A randomized trial of its impact on primary care physicians' initial management of major depression [corrected]. Arch Intern Med. 2001; 161:189-97.
PubMed
CrossRef
 
McLaughlin D, Hayes JR, Kelleher K.  Office-based interventions for recognizing abnormal pediatric blood pressures. Clin Pediatr (Phila). 2010; 49:355-62.
CrossRef
 
Hetlevik I, Holmen J, Krüger O.  Implementing clinical guidelines in the treatment of hypertension in general practice. Evaluation of patient outcome related to implementation of a computer-based clinical decision support system. Scand J Prim Health Care. 1999; 17:35-40.
PubMed
 
Hetlevik I, Holmen J, Krüger O, Kristensen P, Iversen H.  Implementing clinical guidelines in the treatment of hypertension in general practice. Blood Press. 1998; 7:270-6.
PubMed
 
Hetlevik I, Holmen J, Krüger O, Kristensen P, Iversen H, Furuseth K.  Implementing clinical guidelines in the treatment of diabetes mellitus in general practice. Evaluation of effort, process, and patient outcome related to implementation of a computer-based decision support system. Int J Technol Assess Health Care. 2000; 16:210-27.
PubMed
CrossRef
 
Bosworth HB, Olsen MK, Dudley T, Orr M, Goldstein MK, Datta SK, et al..  Patient education and provider decision support to control blood pressure in primary care: a cluster randomized trial. Am Heart J. 2009; 157:450-6.
PubMed
CrossRef
 
Bosworth HB, Olsen MK, Goldstein MK, Orr M, Dudley T, McCant F, et al..  The veterans' study to improve the control of hypertension (V-STITCH): design and methodology. Contemp Clin Trials. 2005; 26:155-68.
PubMed
CrossRef
 
Buntin MB, Burke MF, Hoaglin MC, Blumenthal D.  The benefits of health information technology: a review of the recent literature shows predominantly positive results. Health Aff (Millwood). 2011; 30:464-71.
PubMed
CrossRef
 
Osheroff JA, Healthcare Information and Management Systems Society.  Improving Outcomes with Clinical Decision Support: An Implementer's Guide. 2nd ed. Chicago: Healthcare Information and Management Systems Society; 2005.
 

This article was published at www.annals.org on 24 April 2012.

Figures

Grahic Jump Location
Appendix Figure.

Summary of evidence search and selection.

CDSS = clinical decision-support system; KQ = key question; RCT = randomized, controlled trial.

Grahic Jump Location
Grahic Jump Location
Figure 1.

Results of studies that examined whether recommended preventive care services were ordered.

Studies reporting the odds ratio of adhering to recommendations for ordering or completing preventive care services of CDSS vs. control groups. In the 25 studies comparing CDSS with control groups, the random-effects–combined odds ratio of adherence to preventive care recommendations was 1.42 (95% CI, 1.27 to 1.58). CDSS = clinical decision-support system.

Grahic Jump Location
Grahic Jump Location
Figure 2.

Results of studies that examined whether recommended clinical studies were ordered.

Studies reporting the odds ratio of adhering to recommendations for ordering or completing recommended clinical studies of CDSS vs. control groups. In the 20 studies comparing CDSS with control groups, the random-effects–combined odds ratio of adherence to clinical study recommendations was 1.72 (95% CI, 1.47 to 2.00). CDSS = clinical decision-support system.

Grahic Jump Location
Grahic Jump Location
Figure 3.

Results of studies that examined whether recommended treatments were ordered.

Studies reporting the odds ratio of adhering to recommendations for ordering or prescribing treatment of CDSS vs. control groups. In the 46 studies comparing CDSS with control groups, the random-effects–combined odds ratio of adherence to treatment recommendations was 1.57 (95% CI, 1.35 to 1.82). CDSS = clinical decision-support system.

Grahic Jump Location

Tables

Table Jump PlaceholderAppendix Table 1.  

Study Characteristics

Table Jump PlaceholderTable.  

Summary of Evidence, by Outcome

Table Jump PlaceholderAppendix Table 2.  

Examples of Clinical Decision-Support Interventions

References

Kawamoto K, Houlihan CA, Balas EA, Lobach DF.  Improving clinical practice using clinical decision support systems: a systematic review of trials to identify features critical to success. BMJ. 2005; 330:765.
PubMed
CrossRef
 
Chaudhry B, Wang J, Wu S, Maglione M, Mojica W, Roth E, et al..  Systematic review: impact of health information technology on quality, efficiency, and costs of medical care. Ann Intern Med. 2006; 144:742-52.
PubMed
 
Bryan C, Boren SA.  The use and effectiveness of electronic clinical decision support tools in the ambulatory/primary care setting: a systematic review of the literature. Inform Prim Care. 2008; 16:79-91.
PubMed
 
Garg AX, Adhikari NK, McDonald H, Rosas-Arellano MP, Devereaux PJ, Beyene J, et al..  Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: a systematic review. JAMA. 2005; 293:1223-38.
PubMed
CrossRef
 
Grimshaw J, Freemantle N, Wallace S, Russell I, Hurwitz B, Watt I, et al..  Developing and implementing clinical practice guidelines. Qual Health Care. 1995; 4:55-64.
PubMed
 
Hunt DL, Haynes RB, Hanna SE, Smith K.  Effects of computer-based clinical decision support systems on physician performance and patient outcomes: a systematic review. JAMA. 1998; 280:1339-46.
PubMed
 
Sintchenko V, Magrabi F, Tipper S.  Are we measuring the right end-points? Variables that affect the impact of computerised decision support on patient outcomes: a systematic review. Med Inform Internet Med. 2007; 32:225-40.
PubMed
 
Shojania KG, Jennings A, Mayhew A, Ramsay C, Eccles M, Grimshaw J.  Effect of point-of-care computer reminders on physician behaviour: a systematic review. Can Med Assoc J. 2010; 182:216-25.
CrossRef
 
Kaushal R, Shojania KG, Bates DW.  Effects of computerized physician order entry and clinical decision support systems on medication safety: a systematic review. Arch Intern Med. 2003; 163:1409-16.
PubMed
CrossRef
 
Agency for Healthcare Research and Quality.  Methods Guide for Effectiveness and Comparative Effectiveness Reviews. Rockville, MD: Agency for Healthcare Research and Quality; 2011. Accessed at www.effectivehealthcare.ahrq.gov/index.cfm/search-for-guides-reviews-and-reports/?pageaction=displayproduct&productid=318 on 24 January 2012.
 
Owens DK, Lohr KN, Atkins D, Treadwell JR, Reston JT, Bass EB, et al..  AHRQ series paper 5: grading the strength of a body of evidence when comparing medical interventions—agency for healthcare research and quality and the effective health-care program. J Clin Epidemiol. 2010; 63:513-23.
PubMed
CrossRef
 
Atkins D, Chang SM, Gartlehner G, Buckley DI, Whitlock EP, Berliner E, et al..  Assessing applicability when comparing medical interventions: AHRQ and the Effective Health Care Program. J Clin Epidemiol. 2011; 64:1198-207.
PubMed
CrossRef
 
DerSimonian R, Laird N.  Meta-analysis in clinical trials. Control Clin Trials. 1986; 7:177-88.
PubMed
CrossRef
 
McCowan C, Neville RG, Ricketts IW, Warner FC, Hoskins G, Thomas GE.  Lessons from a randomized controlled trial designed to evaluate computer decision support software to improve the management of asthma. Med Inform Internet Med. 2001; 26:191-201.
PubMed
 
Cavalcanti AB, Silva E, Pereira AJ, Caldeira-Filho M, Almeida FP, Westphal GA, et al..  A randomized controlled trial comparing a computer-assisted insulin infusion protocol with a strict and a conventional protocol for glucose control in critically ill patients. J Crit Care. 2009; 24:371-8.
PubMed
CrossRef
 
Kline JA, Zeitouni RA, Hernandez-Nino J, Jones AE.  Randomized trial of computerized quantitative pretest probability in low-risk chest pain patients: effect on safety and resource use. Ann Emerg Med. 2009; 53:727-35.e1.
PubMed
CrossRef
 
Kucher N, Koo S, Quiroz R, Cooper JM, Paterno MD, Soukonnikov B, et al..  Electronic alerts to prevent venous thromboembolism among hospitalized patients. N Engl J Med. 2005; 352:969-77.
PubMed
CrossRef
 
Zanetti G, Flanagan HL Jr, Cohn LH, Giardina R, Platt R.  Improvement of intraoperative antibiotic prophylaxis in prolonged cardiac surgery by automated alerts in the operating room. Infect Control Hosp Epidemiol. 2003; 24:13-6.
PubMed
CrossRef
 
McDonald CJ, Hui SL, Smith DM, Tierney WM, Cohen SJ, Weinberger M, et al..  Reminders to physicians from an introspective computer medical record. A two-year randomized trial. Ann Intern Med. 1984; 100:130-8.
PubMed
 
Roumie CL, Elasy TA, Greevy R, Griffin MR, Liu X, Stone WJ, et al..  Improving blood pressure control through provider education, provider alerts, and patient education: a cluster randomized trial. Ann Intern Med. 2006; 145:165-75.
PubMed
 
Paul M, Andreassen S, Tacconelli E, Nielsen AD, Almanasreh N, Frank U, et al., TREAT Study Group.  Improving empirical antibiotic treatment using TREAT, a computerized decision support system: cluster randomized trial. J Antimicrob Chemother. 2006; 58:1238-45.
PubMed
 
Ansari M, Shlipak MG, Heidenreich PA, Van Ostaeyen D, Pohl EC, Browner WS, et al..  Improving guideline adherence: a randomized trial evaluating strategies to increase beta-blocker use in heart failure. Circulation. 2003; 107:2799-804.
PubMed
CrossRef
 
Graumlich JF, Novotny NL, Nace GS, Aldag JC.  Patient and physician perceptions after software-assisted hospital discharge: cluster randomized trial. J Hosp Med. 2009; 4:356-63.
PubMed
CrossRef
 
Graumlich JF, Novotny NL, Stephen Nace G, Kaushal H, Ibrahim-Ali W, Theivanayagam S, et al..  Patient readmissions, emergency visits, and adverse events after software-assisted discharge from hospital: cluster randomized trial. J Hosp Med. 2009; 4:11-9.
PubMed
CrossRef
 
Heidenreich PA, Gholami P, Sahay A, Massie B, Goldstein MK.  Clinical reminders attached to echocardiography reports of patients with reduced left ventricular ejection fraction increase use of beta-blockers: a randomized trial. Circulation. 2007; 115:2829-34.
PubMed
CrossRef
 
Tierney WM, Overhage JM, Murray MD, Harris LE, Zhou XH, Eckert GJ, et al..  Can computer-generated evidence-based care suggestions enhance evidence-based management of asthma and chronic obstructive pulmonary disease? A randomized, controlled trial. Health Serv Res. 2005; 40:477-97.
PubMed
CrossRef
 
Tierney WM, Overhage JM, Murray MD, Harris LE, Zhou XH, Eckert GJ, et al..  Effects of computerized guidelines for managing heart disease in primary care. J Gen Intern Med. 2003; 18:967-76.
PubMed
CrossRef
 
Gilutz H, Novack L, Shvartzman P, Zelingher J, Bonneh DY, Henkin Y, et al..  Computerized community cholesterol control (4C): meeting the challenge of secondary prevention. Isr Med Assoc J. 2009; 11:23-9.
PubMed
 
Holt TA, Thorogood M, Griffiths F, Munday S.  Protocol for the ‘e-Nudge trial’: a randomised controlled trial of electronic feedback to reduce the cardiovascular risk of individuals in general practice [ISRCTN64828380]. Trials. 2006; 7:11.
PubMed
CrossRef
 
Holt TA, Thorogood M, Griffiths F, Munday S, Friede T, Stables D.  Automated electronic reminders to facilitate primary cardiovascular disease prevention: randomised controlled trial. Br J Gen Pract. 2010; 60:137-43.
PubMed
CrossRef
 
Khan S, Maclean CD, Littenberg B.  The effect of the Vermont Diabetes Information System on inpatient and emergency room use: results from a randomized trial. Health Outcomes Res Med. 2010; 1:61-e66.
PubMed
 
Maclean CD, Gagnon M, Callas P, Littenberg B.  The Vermont diabetes information system: a cluster randomized trial of a population based decision support system. J Gen Intern Med. 2009; 24:1303-10.
PubMed
CrossRef
 
Brier ME, Gaweda AE, Dailey A, Aronoff GR, Jacobs AA.  Randomized trial of model predictive control for improved anemia management. Clin J Am Soc Nephrol. 2010; 5:814-20.
PubMed
CrossRef
 
Hamilton E, Platt R, Gauthier R, McNamara H, Miner L, Rothenberg S, et al..  The effect of computer-assisted evaluation of labor on cesarean rates. J Healthc Qual. 2004; 26:37-44.
PubMed
CrossRef
 
McDonald CJ, Hui SL, Tierney WM.  Effects of computer reminders for influenza vaccination on morbidity during influenza epidemics. MD Comput. 1992; 9:304-12.
PubMed
 
Murray MD, Harris LE, Overhage JM, Zhou XH, Eckert GJ, Smith FE, et al..  Failure of computerized treatment suggestions to improve health outcomes of outpatients with uncomplicated hypertension: results of a randomized controlled trial. Pharmacotherapy. 2004; 24:324-37.
PubMed
CrossRef
 
Sequist TD, Zaslavsky AM, Marshall R, Fletcher RH, Ayanian JZ.  Patient and physician reminders to promote colorectal cancer screening: a randomized controlled trial. Arch Intern Med. 2009; 169:364-71.
PubMed
 
Subramanian U, Fihn SD, Weinberger M, Plue L, Smith FE, Udris EM, et al..  A controlled trial of including symptom data in computer-based care suggestions for managing patients with chronic heart failure. Am J Med. 2004; 116:375-84.
PubMed
CrossRef
 
Kuperman GJ, Teich JM, Tanasijevic MJ, Ma'Luf N, Rittenberg E, Jha A, et al..  Improving response to critical laboratory results with automation: results of a randomized controlled trial. J Am Med Inform Assoc. 1999; 6:512-22.
PubMed
CrossRef
 
McGregor JC, Weekes E, Forrest GN, Standiford HC, Perencevich EN, Furuno JP, et al..  Impact of a computerized clinical decision support system on reducing inappropriate antimicrobial use: a randomized controlled trial. J Am Med Inform Assoc. 2006; 13:378-84.
PubMed
CrossRef
 
Fihn SD, McDonell MB, Vermes D, Henikoff JG, Martin DC, Callahan CM, et al..  A computerized intervention to improve timing of outpatient follow-up: a multicenter randomized trial in patients treated with warfarin. National Consortium of Anticoagulation Clinics. J Gen Intern Med. 1994; 9:131-9.
PubMed
CrossRef
 
Gurwitz JH, Field TS, Rochon P, Judge J, Harrold LR, Bell CM, et al..  Effect of computerized provider order entry with clinical decision support on adverse drug events in the long-term care setting. J Am Geriatr Soc. 2008; 56:2225-33.
PubMed
CrossRef
 
Apkon M, Mattera JA, Lin Z, Herrin J, Bradley EH, Carbone M, et al..  A randomized outpatient trial of a decision-support information technology tool. Arch Intern Med. 2005; 165:2388-94.
PubMed
CrossRef
 
Bertoni AG, Bonds DE, Chen H, Hogan P, Crago L, Rosenberger E, et al..  Impact of a multifaceted intervention on cholesterol management in primary care practices: guideline adherence for heart health randomized trial. Arch Intern Med. 2009; 169:678-86.
PubMed
CrossRef
 
Burack RC, Gimotty PA, George J, McBride S, Moncrease A, Simon MS, et al..  How reminders given to patients and physicians affected pap smear use in a health maintenance organization: results of a randomized controlled trial. Cancer. 1998; 82:2391-400.
PubMed
CrossRef
 
Burack RC, Gimotty PA, Simon M, Moncrease A, Dews P.  The effect of adding Pap smear information to a mammography reminder system in an HMO: results of randomized controlled trial. Prev Med. 2003; 36:547-54.
PubMed
CrossRef
 
Cannon DS, Allen SN.  A comparison of the effects of computer and manual reminders on compliance with a mental health clinical practice guideline. J Am Med Inform Assoc. 2000; 7:196-203.
PubMed
CrossRef
 
Demakis JG, Beauchamp C, Cull WL, Denwood R, Eisen SA, Lofgren R, et al..  Improving residents' compliance with standards of ambulatory care: results from the VA Cooperative Study on Computerized Reminders. JAMA. 2000; 284:1411-6.
PubMed
CrossRef
 
Dexter PR, Perkins S, Overhage JM, Maharry K, Kohler RB, McDonald CJ.  A computerized reminder system to increase the use of preventive care for hospitalized patients. N Engl J Med. 2001; 345:965-70.
PubMed
CrossRef
 
Dexter PR, Perkins SM, Maharry KS, Jones K, McDonald CJ.  Inpatient computer-based standing orders vs physician reminders to increase influenza and pneumococcal vaccination rates: a randomized trial. JAMA. 2004; 292:2366-71.
PubMed
CrossRef
 
Eccles M, McColl E, Steen N, Rousseau N, Grimshaw J, Parkin D, et al..  Effect of computerised evidence based guidelines on management of asthma and angina in adults in primary care: cluster randomised controlled trial. BMJ. 2002; 325:941.
PubMed
CrossRef
 
Frank O, Litt J, Beilby J.  Opportunistic electronic reminders. Improving performance of preventive care in general practice. Aust Fam Physician. 2004; 33:87-90.
PubMed
 
Fretheim A, Aaserud M, Oxman AD.  Rational prescribing in primary care (RaPP): economic evaluation of an intervention to improve professional practice. PLoS Med. 2006; 3:216.
PubMed
CrossRef
 
Fretheim A, Oxman AD, Håvelsrud K, Treweek S, Kristoffersen DT, Bjørndal A.  Rational prescribing in primary care (RaPP): a cluster randomized trial of a tailored intervention. PLoS Med. 2006; 3:134.
PubMed
CrossRef
 
Litzelman DK, Dittus RS, Miller ME, Tierney WM.  Requiring physicians to respond to computerized reminders improves their compliance with preventive care protocols. J Gen Intern Med. 1993; 8:311-7.
PubMed
CrossRef
 
McDowell I, Newell C, Rosser W.  Comparison of three methods of recalling patients for influenza vaccination. CMAJ. 1986; 135:991-7.
PubMed
 
McDowell I, Newell C, Rosser W.  Computerized reminders to encourage cervical screening in family practice. J Fam Pract. 1989; 28:420-4.
PubMed
 
Overhage JM, Tierney WM, McDonald CJ.  Computer reminders to implement preventive care guidelines for hospitalized patients. Arch Intern Med. 1996; 156:1551-6.
PubMed
CrossRef
 
Price M.  Can hand-held computers improve adherence to guidelines? A (Palm) Pilot study of family doctors in British Columbia. Can Fam Physician. 2005; 51:1506-7.
PubMed
 
Taylor V, Thompson B, Lessler D, Yasui Y, Montano D, Johnson KM, et al..  A clinic-based mammography intervention targeting inner-city women. J Gen Intern Med. 1999; 14:104-11.
PubMed
CrossRef
 
Unrod M, Smith M, Spring B, DePue J, Redd W, Winkel G.  Randomized controlled trial of a computer-based, tailored intervention to increase smoking cessation counseling by primary care physicians. J Gen Intern Med. 2007; 22:478-84.
PubMed
CrossRef
 
Dykes PC, Carroll DL, Hurley A, Lipsitz S, Benoit A, Chang F, et al..  Fall prevention in acute care hospitals: a randomized trial. JAMA. 2010; 304:1912-8.
PubMed
CrossRef
 
Chambers CV, Balaban DJ, Carlson BL, Ungemack JA, Grasberger DM.  Microcomputer-generated reminders. Improving the compliance of primary care physicians with mammography screening guidelines. J Fam Pract. 1989; 29:273-80.
PubMed
 
Burack RC, Gimotty PA.  Promoting screening mammography in inner-city settings. The sustained effectiveness of computerized reminders in a randomized controlled trial. Med Care. 1997; 35:921-31.
PubMed
CrossRef
 
Burack RC, Gimotty PA, George J, Stengle W, Warbasse L, Moncrease A.  Promoting screening mammography in inner-city settings: a randomized controlled trial of computerized reminders as a component of a program to facilitate mammography. Med Care. 1994; 32:609-24.
PubMed
CrossRef
 
Fiks AG, Hunter KF, Localio AR, Grundmeier RW, Bryant-Stephens T, Luberti AA, et al..  Impact of electronic health record-based alerts on influenza vaccination for children with asthma. Pediatrics. 2009; 124:159-69.
PubMed
CrossRef
 
Flanagan JR, Doebbeling BN, Dawson J, Beekmann S.  Randomized study of online vaccine reminders in adult primary care. Proc AMIA Symp. 1999; 755-9.
PubMed
 
Fordham D, McPhee SJ, Bird JA, Rodnick JE, Detmer WM.  The Cancer Prevention Reminder System. MD Comput. 1990; 7:289-95.
PubMed
 
McPhee SJ, Bird JA, Jenkins CN, Fordham D.  Promoting cancer screening. A randomized, controlled trial of three interventions. Arch Intern Med. 1989; 149:1866-72.
PubMed
CrossRef
 
Gill JM, Chen YX, Glutting JJ, Diamond JJ, Lieberman MI.  Impact of decision support in electronic medical records on lipid management in primary care. Popul Health Manag. 2009; 12:221-6.
PubMed
CrossRef
 
Hobbs FD, Delaney BC, Carson A, Kenkre JE.  A prospective controlled trial of computerized decision support for lipid management in primary care. Fam Pract. 1996; 13:133-7.
PubMed
 
Holbrook A, Thabane L, Keshavjee K, Dolovich L, Bernstein B, Chan D, et al., COMPETE II Investigators.  Individualized electronic decision support and reminders to improve diabetes care in the community: COMPETE II randomized trial. CMAJ. 2009; 181:37-44.
PubMed
 
Kenealy T, Arroll B, Petrie KJ.  Patients and computers as reminders to screen for diabetes in family practice. Randomized-controlled trial. J Gen Intern Med. 2005; 20:916-21.
PubMed
CrossRef
 
Lobach DF, Hammond WE.  Development and evaluation of a Computer-Assisted Management Protocol (CAMP): improved compliance with care guidelines for diabetes mellitus. Proc Annu Symp Comput Appl Med Care. 1994; 787-91.
PubMed
 
Ornstein SM, Garr DR, Jenkins RG, Rust PF, Arnon A.  Computer-generated physician and patient reminders. Tools to improve population adherence to selected preventive services. J Fam Pract. 1991; 32:82-90.
PubMed
 
Peterson KA, Radosevich DM, O'Connor PJ, Nyman JA, Prineas RJ, Smith SA, et al..  Improving diabetes care in practice: findings from the TRANSLATE trial. Diabetes Care. 2008; 31:2238-43.
PubMed
CrossRef
 
Reeve JF, Tenni PC, Peterson GM.  An electronic prompt in dispensing software to promote clinical interventions by community pharmacists: a randomized controlled trial. Br J Clin Pharmacol. 2008; 65:377-85.
PubMed
CrossRef
 
Rosser WW, Hutchison BG, McDowell I, Newell C.  Use of reminders to increase compliance with tetanus booster vaccination. CMAJ. 1992; 146:911-7.
PubMed
 
Rosser WW, McDowell I, Newell C.  Use of reminders for preventive procedures in family medicine. CMAJ. 1991; 145:807-14.
PubMed
 
Sequist TD, Gandhi TK, Karson AS, Fiskio JM, Bugbee D, Sperling M, et al..  A randomized trial of electronic clinical reminders to improve quality of care for diabetes and coronary artery disease. J Am Med Inform Assoc. 2005; 12:431-7.
PubMed
CrossRef
 
Tierney WM, Hui SL, McDonald CJ.  Delayed feedback of physician performance versus immediate reminders to perform preventive care. Effects on physician compliance. Med Care. 1986; 24:659-66.
PubMed
CrossRef
 
van Wyk JT, van Wijk MA, Sturkenboom MC, Mosseveld M, Moorman PW, van der Lei J.  Electronic alerts versus on-demand decision support to improve dyslipidemia treatment: a cluster randomized controlled trial. Circulation. 2008; 117:371-8.
PubMed
CrossRef
 
Bates DW, Kuperman GJ, Rittenberg E, Teich JM, Fiskio J, Ma'luf N, et al..  A randomized trial of a computer-based intervention to reduce utilization of redundant laboratory tests. Am J Med. 1999; 106:144-50.
PubMed
CrossRef
 
Bell LM, Grundmeier R, Localio R, Zorc J, Fiks AG, Zhang X, et al..  Electronic health record-based decision support to improve asthma care: a cluster-randomized trial. Pediatrics. 2010; 125:770-7.
PubMed
CrossRef
 
Downs M, Turner S, Bryans M, Wilcock J, Keady J, Levin E, et al..  Effectiveness of educational interventions in improving detection and management of dementia in primary care: cluster randomised controlled study. BMJ. 2006; 332:692-6.
PubMed
CrossRef
 
Emery J, Morris H, Goodchild R, Fanshawe T, Prevost AT, Bobrow M, et al..  The GRAIDS Trial: a cluster randomised controlled trial of computer decision support for the management of familial cancer risk in primary care. Br J Cancer. 2007; 97:486-93.
PubMed
CrossRef
 
Feldstein AC, Smith DH, Perrin N, Yang X, Rix M, Raebel MA, et al..  Improved therapeutic monitoring with several interventions: a randomized trial. Arch Intern Med. 2006; 166:1848-54.
PubMed
 
Flottorp S, Oxman AD, Håvelsrud K, Treweek S, Herrin J.  Cluster randomised controlled trial of tailored interventions to improve the management of urinary tract infections in women and sore throat. BMJ. 2002; 325:367.
PubMed
CrossRef
 
Greiver M, Drummond N, White D, Weshler J, Moineddin R, North Toronto Primary Care Research Network (Nortren).  Angina on the Palm: randomized controlled pilot trial of Palm PDA software for referrals for cardiac testing. Can Fam Physician. 2005; 51:382-3.
PubMed
 
Harpole LH, Khorasani R, Fiskio J, Kuperman GJ, Bates DW.  Automated evidence-based critiquing of orders for abdominal radiographs: impact on utilization and appropriateness. J Am Med Inform Assoc. 1997; 4:511-21.
PubMed
CrossRef
 
Lee NJ, Chen ES, Currie LM, Donovan M, Hall EK, Jia H, et al..  The effect of a mobile clinical decision support system on the diagnosis of obesity and overweight in acute and primary care encounters. ANS Adv Nurs Sci. 2009; 32:211-21.
PubMed
 
Lo HG, Matheny ME, Seger DL, Bates DW, Gandhi TK.  Impact of non-interruptive medication laboratory monitoring alerts in ambulatory care. J Am Med Inform Assoc. 2009; 16:66-71.
PubMed
CrossRef
 
Matheny ME, Sequist TD, Seger AC, Fiskio JM, Sperling M, Bugbee D, et al..  A randomized trial of electronic clinical reminders to improve medication laboratory monitoring. J Am Med Inform Assoc. 2008; 15:424-9.
PubMed
CrossRef
 
McDonald CJ.  Use of a computer to detect and respond to clinical events: its effect on clinician behavior. Ann Intern Med. 1976; 84:162-7.
PubMed
 
McDowell I, Newell C, Rosser W.  A randomized trial of computerized reminders for blood pressure screening in primary care. Med Care. 1989; 27:297-305.
PubMed
CrossRef
 
Palen TE, Raebel M, Lyons E, Magid DM.  Evaluation of laboratory monitoring alerts within a computerized physician order entry system for medication orders. Am J Manag Care. 2006; 12:389-95.
PubMed
 
Palen TE, Price DW, Snyder AJ, Shetterly SM.  Computerized alert reduced D-dimer testing in the elderly. Am J Manag Care. 2010; 16:267-75.
PubMed
 
Player MS, Gill JM, Mainous AG 3rd, Everett CJ, Koopman RJ, Diamond JJ, et al..  An electronic medical record-based intervention to improve quality of care for gastro-esophageal reflux disease (GERD) and atypical presentations of GERD. Qual Prim Care. 2010; 18:223-9.
PubMed
 
Raebel MA, Lyons EE, Chester EA, Bodily MA, Kelleher JA, Long CL, et al..  Improving laboratory monitoring at initiation of drug therapy in ambulatory care: a randomized trial. Arch Intern Med. 2005; 165:2395-401.
PubMed
CrossRef
 
Raebel MA, Chester EA, Newsom EE, Lyons EE, Kelleher JA, Long C, et al..  Randomized trial to improve laboratory safety monitoring of ongoing drug therapy in ambulatory patients. Pharmacotherapy. 2006; 26:619-26.
PubMed
CrossRef
 
Roukema J, Steyerberg EW, van der Lei J, Moll HA.  Randomized trial of a clinical decision support system: impact on the management of children with fever without apparent source. J Am Med Inform Assoc. 2008; 15:107-13.
PubMed
 
Roy PM, Durieux P, Gillaizeau F, Legall C, Armand-Perroux A, Martino L, et al..  A computerized handheld decision-support system to improve pulmonary embolism diagnosis: a randomized trial. Ann Intern Med. 2009; 151:677-86.
PubMed
 
Schriefer SP, Landis SE, Turbow DJ, Patch SC.  Effect of a computerized body mass index prompt on diagnosis and treatment of adult obesity. Fam Med. 2009; 41:502-7.
PubMed
 
Stiell IG, Clement CM, Grimshaw J, Brison RJ, Rowe BH, Schull MJ, et al..  Implementation of the Canadian C-Spine Rule: prospective 12 centre cluster randomised trial. BMJ. 2009; 339:b4146.
PubMed
CrossRef
 
Sundaram V, Lazzeroni LC, Douglass LR, Sanders GD, Tempio P, Owens DK.  A randomized trial of computer-based reminders and audit and feedback to improve HIV screening in a primary care setting. Int J STD AIDS. 2009; 20:527-33.
PubMed
 
Tierney WM, McDonald CJ, Martin DK, Rogers MP.  Computerized display of past test results. Effect on outpatient testing. Ann Intern Med. 1987; 107:569-74.
PubMed
 
van Wijk MA, van der Lei J, Mosseveld M, Bohnen AM, van Bemmel JH.  Assessment of decision support for blood test ordering in primary care. a randomized trial. Ann Intern Med. 2001; 134:274-81.
PubMed
 
Walker J, Fairley CK, Walker SM, Gurrin LC, Gunn JM, Pirotta MV, et al..  Computer reminders for Chlamydia screening in general practice: a randomized controlled trial. Sex Transm Dis. 2010; 37:445-50.
PubMed
 
Wilson BJ, Torrance N, Mollison J, Watson MS, Douglas A, Miedzybrodzka Z, et al..  Cluster randomized trial of a multifaceted primary care decision-support intervention for inherited breast cancer risk. Fam Pract. 2006; 23:537-44.
PubMed
CrossRef
 
Bourgeois FC, Linder J, Johnson SA, Co JP, Fiskio J, Ferris TG.  Impact of a computerized template on antibiotic prescribing for acute respiratory infections in children and adolescents. Clin Pediatr (Phila). 2010; 49:976-83.
PubMed
CrossRef
 
Christakis DA, Zimmerman FJ, Wright JA, Garrison MM, Rivara FP, Davis RL.  A randomized controlled trial of point-of-care evidence to improve the antibiotic prescribing practices for otitis media in children. Pediatrics. 2001; 107:15.
PubMed
CrossRef
 
Co JP, Johnson SA, Poon EG, Fiskio J, Rao SR, Van Cleave J, et al..  Electronic health record decision support and quality of care for children with ADHD. Pediatrics. 2010; 126:239-46.
PubMed
CrossRef
 
Cobos A, Vilaseca J, Asenjo C, Pedro-Botet J, Sánchez E, Val A, et al..  Cost effectiveness of a clinical decision support system based on the recommendations of the European Society of Cardiology and other societies for the management of hypercholesterolemia: report of a cluster-randomized trial. Disease Management & Health Outcomes. 2005; 13:421-32.
CrossRef
 
Davis RL, Wright J, Chalmers F, Levenson L, Brown JC, Lozano P, et al..  A cluster randomized clinical trial to improve prescribing patterns in ambulatory pediatrics. PLoS Clin Trials. 2007; 2:25.
PubMed
CrossRef
 
Feldstein A, Elmer PJ, Smith DH, Herson M, Orwoll E, Chen C, et al..  Electronic medical record reminder improves osteoporosis management after a fracture: a randomized, controlled trial. J Am Geriatr Soc. 2006; 54:450-7.
PubMed
CrossRef
 
Field TS, Rochon P, Lee M, Gavendo L, Baril JL, Gurwitz JH.  Computerized clinical decision support during medication ordering for long-term care residents with renal insufficiency. J Am Med Inform Assoc. 2009; 16:480-5.
PubMed
CrossRef
 
Filippi A, Sabatini A, Badioli L, Samani F, Mazzaglia G, Catapano A, et al..  Effects of an automated electronic reminder in changing the antiplatelet drug-prescribing behavior among Italian general practitioners in diabetic patients: an intervention trial. Diabetes Care. 2003; 26:1497-500.
PubMed
CrossRef
 
Fitzmaurice DA, Hobbs FD, Murray ET, Holder RL, Allan TF, Rose PE.  Oral anticoagulation management in primary care with the use of computerized decision support and near-patient testing: a randomized, controlled trial. Arch Intern Med. 2000; 160:2343-8.
PubMed
CrossRef
 
Fortuna RJ, Zhang F, Ross-Degnan D, Campion FX, Finkelstein JA, Kotch JB, et al..  Reducing the prescribing of heavily marketed medications: a randomized controlled trial. J Gen Intern Med. 2009; 24:897-903.
PubMed
CrossRef
 
Goud R, de Keizer NF, ter Riet G, Wyatt JC, Hasman A, Hellemans IM, et al..  Effect of guideline based computerised decision support on decision making of multidisciplinary teams: cluster randomised trial in cardiac rehabilitation. BMJ. 2009; 338:b1440.
PubMed
CrossRef
 
Hicks LS, Sequist TD, Ayanian JZ, Shaykevich S, Fairchild DG, Orav EJ, et al..  Impact of computerized decision support on blood pressure management and control: a randomized controlled trial. J Gen Intern Med. 2008; 23:429-41.
PubMed
 
Krall MA, Traunweiser K, Towery W.  Effectiveness of an electronic medical record clinical quality alert prepared by off-line data analysis. Stud Health Technol Inform. 2004; 107:135-9.
PubMed
 
Linder JA, Rigotti NA, Schneider LI, Kelley JH, Brawarsky P, Haas JS.  An electronic health record-based intervention to improve tobacco treatment in primary care: a cluster-randomized controlled trial. Arch Intern Med. 2009; 169:781-7.
PubMed
CrossRef
 
Locatelli F, Covic A, Macdougall IC, Scherhag A, Wiecek A, ORAMA Study Group.  Effect of computer-assisted European Best Practice Guideline implementation on adherence and target attainment: ORAMA results. J Nephrol. 2009; 22:662-74.
PubMed
 
Manotti C, Moia M, Palareti G, Pengo V, Ria L, Dettori AG.  Effect of computer-aided management on the quality of treatment in anticoagulated patients: a prospective, randomized, multicenter trial of APROAT (Automated PRogram for Oral Anticoagulant Treatment). Haematologica. 2001; 86:1060-70.
PubMed
 
Marco F, Sedano C, Bermúdez A, López-Duarte M, Fernández-Fontecha E, Zubizarreta A.  A prospective controlled study of a computer-assisted acenocoumarol dosage program. Pathophysiol Haemost Thromb. 2003; 33:59-63.
PubMed
CrossRef
 
Martens JD, van der Aa A, Panis B, van der Weijden T, Winkens RA, Severens JL.  Design and evaluation of a computer reminder system to improve prescribing behaviour of GPs. Stud Health Technol Inform. 2006; 124:617-23.
PubMed
 
Martens JD, van der Weijden T, Severens JL, de Clercq PA, de Bruijn DP, Kester AD, et al..  The effect of computer reminders on GPs' prescribing behaviour: a cluster-randomised trial. Int J Med Inform. 2007; 76:Suppl 3S403-16.
PubMed
CrossRef
 
Montgomery AA, Fahey T, Peters TJ, MacIntosh C, Sharp DJ.  Evaluation of computer based clinical decision support system and risk chart for management of hypertension in primary care: randomised controlled trial. BMJ. 2000; 320:686-90.
PubMed
CrossRef
 
Overhage JM, Tierney WM, Zhou XH, McDonald CJ.  A randomized trial of “corollary orders” to prevent errors of omission. J Am Med Inform Assoc. 1997; 4:364-75.
PubMed
CrossRef
 
Peterson JF, Rosenbaum BP, Waitman LR, Habermann R, Powers J, Harrell D, et al..  Physicians' response to guided geriatric dosing: initial results from a randomized trial. Stud Health Technol Inform. 2007; 129:1037-40.
PubMed
 
Phillips LS, Ziemer DC, Doyle JP, Barnes CS, Kolm P, Branch WT, et al..  An endocrinologist-supported intervention aimed at providers improves diabetes management in a primary care site: improving primary care of African Americans with diabetes (IPCAAD) 7. Diabetes Care. 2005; 28:2352-60.
PubMed
CrossRef
 
Ziemer DC, Doyle JP, Barnes CS, Branch WT Jr, Cook CB, El-Kebbi IM, et al..  An intervention to overcome clinical inertia and improve diabetes mellitus control in a primary care setting: Improving Primary Care of African Americans with Diabetes (IPCAAD) 8. Arch Intern Med. 2006; 166:507-13.
PubMed
 
Raebel MA, Charles J, Dugan J, Carroll NM, Korner EJ, Brand DW, et al..  Randomized trial to improve prescribing safety in ambulatory elderly patients. J Am Geriatr Soc. 2007; 55:977-85.
PubMed
CrossRef
 
Rood E, Bosman RJ, van der Spoel JI, Taylor P, Zandstra DF.  Use of a computerized guideline for glucose regulation in the intensive care unit improved both guideline adherence and glucose regulation. J Am Med Inform Assoc. 2005; 12:172-80.
PubMed
 
Rossi RA, Every NR.  A computerized intervention to decrease the use of calcium channel blockers in hypertension. J Gen Intern Med. 1997; 12:672-8.
PubMed
CrossRef
 
Rothschild JM, McGurk S, Honour M, Lu L, McClendon AA, Srivastava P, et al..  Assessment of education and computerized decision support interventions for improving transfusion practice. Transfusion. 2007; 47:228-39.
PubMed
CrossRef
 
Samore MH, Bateman K, Alder SC, Hannah E, Donnelly S, Stoddard GJ, et al..  Clinical decision support and appropriateness of antimicrobial prescribing: a randomized trial. JAMA. 2005; 294:2305-14.
PubMed
CrossRef
 
Shojania KG, Yokoe D, Platt R, Fiskio J, Ma'luf N, Bates DW.  Reducing vancomycin use utilizing a computer guideline: results of a randomized controlled trial. J Am Med Inform Assoc. 1998; 5:554-62.
PubMed
CrossRef
 
Simon SR, Smith DH, Feldstein AC, Perrin N, Yang X, Zhou Y, et al..  Computerized prescribing alerts and group academic detailing to reduce the use of potentially inappropriate medications in older people. J Am Geriatr Soc. 2006; 54:963-8.
PubMed
CrossRef
 
Smith SA, Shah ND, Bryant SC, Christianson TJ, Bjornsen SS, Giesler PD, et al., Evidens Research Group.  Chronic care model and shared care in diabetes: randomized trial of an electronic decision support system. Mayo Clin Proc. 2008; 83:747-57.
PubMed
 
Strom BL, Schinnar R, Aberra F, Bilker W, Hennessy S, Leonard CE, et al..  Unintended effects of a computerized physician order entry nearly hard-stop alert to prevent a drug interaction: a randomized controlled trial. Arch Intern Med. 2010; 170:1578-83.
PubMed
CrossRef
 
Strom BL, Schinnar R, Bilker W, Hennessy S, Leonard CE, Pifer E.  Randomized clinical trial of a customized electronic alert requiring an affirmative response compared to a control group receiving a commercial passive CPOE alert: NSAID—warfarin co-prescribing as a test case. J Am Med Inform Assoc. 2010; 17:411-5.
PubMed
CrossRef
 
Tamblyn R, Huang A, Perreault R, Jacques A, Roy D, Hanley J, et al..  The medical office of the 21st century (MOXXI): effectiveness of computerized decision-making support in reducing inappropriate prescribing in primary care. CMAJ. 2003; 169:549-56.
PubMed
 
Tamblyn R, Huang A, Taylor L, Kawasumi Y, Bartlett G, Grad R, et al..  A randomized trial of the effectiveness of on-demand versus computer-triggered drug decision support in primary care. J Am Med Inform Assoc. 2008; 15:430-8.
PubMed
CrossRef
 
Tamblyn R, Reidel K, Huang A, Taylor L, Winslade N, Bartlett G, et al..  Increasing the detection and response to adherence problems with cardiovascular medication in primary care through computerized drug management systems: a randomized controlled trial. Med Decis Making. 2010; 30:176-88.
PubMed
CrossRef
 
Terrell KM, Perkins AJ, Dexter PR, Hui SL, Callahan CM, Miller DK.  Computerized decision support to reduce potentially inappropriate prescribing to older emergency department patients: a randomized, controlled trial. J Am Geriatr Soc. 2009; 57:1388-94.
PubMed
CrossRef
 
Terrell KM, Perkins AJ, Hui SL, Callahan CM, Dexter PR, Miller DK.  Computerized decision support for medication dosing in renal insufficiency: a randomized, controlled trial. Ann Emerg Med. 2010; 56:623-9.
PubMed
CrossRef
 
Vadher BD, Patterson DL, Leaning M.  Comparison of oral anticoagulant control by a nurse-practitioner using a computer decision-support system with that by clinicians. Clin Lab Haematol. 1997; 19:203-7.
PubMed
CrossRef
 
Vadher B, Patterson DL, Leaning M.  Evaluation of a decision support system for initiation and control of oral anticoagulation in a randomised trial. BMJ. 1997; 314:1252-6.
PubMed
CrossRef
 
Vissers MC, Biert J, van der Linden CJ, Hasman A.  Effects of a supportive protocol processing system (ProtoVIEW) on clinical behaviour of residents in the accident and emergency department. Comput Methods Programs Biomed. 1996; 49:177-84.
PubMed
CrossRef
 
Vissers MC, Hasman A, van der Linden CJ.  Protocol processing system (ProtoVIEW) to support residents at the emergency ward. Comput Methods Programs Biomed. 1995; 48:53-8.
PubMed
CrossRef
 
Weir CJ, Lees KR, MacWalter RS, Muir KW, Wallesch CW, McLelland EV, et al., PRISM Study Group.  Cluster-randomized, controlled trial of computer-based decision support for selecting long-term anti-thrombotic therapy after acute ischaemic stroke. QJM. 2003; 96:143-53.
PubMed
CrossRef
 
White KS, Lindsay A, Pryor TA, Brown WF, Walsh K.  Application of a computerized medical decision-making process to the problem of digoxin intoxication. J Am Coll Cardiol. 1984; 4:571-6.
PubMed
CrossRef
 
Alper BS, White DS, Ge B.  Physicians answer more clinical questions and change clinical decisions more often with synthesized evidence: a randomized trial in primary care. Ann Fam Med. 2005; 3:507-13.
PubMed
CrossRef
 
Del Fiol G, Haug PJ, Cimino JJ, Narus SP, Norlin C, Mitchell JA.  Effectiveness of topic-specific infobuttons: a randomized controlled trial. J Am Med Inform Assoc. 2008; 15:752-9.
PubMed
CrossRef
 
Etchells E, Adhikari NK, Cheung C, Fowler R, Kiss A, Quan S, et al..  Real-time clinical alerting: effect of an automated paging system on response time to critical laboratory values—a randomised controlled trial. Qual Saf Health Care. 2010; 19:99-102.
PubMed
 
Bird JA, McPhee SJ, Jenkins C, Fordham D.  Three strategies to promote cancer screening. How feasible is wide-scale implementation? Med Care. 1990; 28:1005-12.
PubMed
CrossRef
 
Frame PS, Zimmer JG, Werth PL, Hall WJ, Eberly SW.  Computer-based vs manual health maintenance tracking. A controlled trial. Arch Fam Med. 1994; 3:581-8.
PubMed
CrossRef
 
Tierney WM, McDonald CJ, Hui SL, Martin DK.  Computer predictions of abnormal test results. Effects on outpatient testing. JAMA. 1988; 259:1194-8.
PubMed
CrossRef
 
Cleveringa FG, Gorter KJ, van den Donk M, Rutten GE.  Combined task delegation, computerized decision support, and feedback improve cardiovascular risk for type 2 diabetic patients: a cluster randomized trial in primary care. Diabetes Care. 2008; 31:2273-5.
PubMed
CrossRef
 
Cleveringa FG, Welsing PM, van den Donk M, Gorter KJ, Niessen LW, Rutten GE, et al..  Cost-effectiveness of the diabetes care protocol, a multifaceted computerized decision support diabetes management intervention that reduces cardiovascular risk. Diabetes Care. 2010; 33:258-63.
PubMed
CrossRef
 
Smith DH, Feldstein AC, Perrin NA, Yang X, Rix MM, Raebel MA, et al..  Improving laboratory monitoring of medications: an economic analysis alongside a clinical trial. Am J Manag Care. 2009; 15:281-9.
PubMed
 
Judge J, Field TS, DeFlorio M, Laprino J, Auger J, Rochon P, et al..  Prescribers' responses to alerts during medication ordering in the long term care setting. J Am Med Inform Assoc. 2006; 13:385-90.
PubMed
CrossRef
 
Maviglia SM, Yoon CS, Bates DW, Kuperman G.  KnowledgeLink: impact of context-sensitive information retrieval on clinicians' information needs. J Am Med Inform Assoc. 2006; 13:67-73.
PubMed
CrossRef
 
Rollman BL, Hanusa BH, Gilbert T, Lowe HJ, Kapoor WN, Schulberg HC.  The electronic medical record. A randomized trial of its impact on primary care physicians' initial management of major depression [corrected]. Arch Intern Med. 2001; 161:189-97.
PubMed
CrossRef
 
McLaughlin D, Hayes JR, Kelleher K.  Office-based interventions for recognizing abnormal pediatric blood pressures. Clin Pediatr (Phila). 2010; 49:355-62.
CrossRef
 
Hetlevik I, Holmen J, Krüger O.  Implementing clinical guidelines in the treatment of hypertension in general practice. Evaluation of patient outcome related to implementation of a computer-based clinical decision support system. Scand J Prim Health Care. 1999; 17:35-40.
PubMed
 
Hetlevik I, Holmen J, Krüger O, Kristensen P, Iversen H.  Implementing clinical guidelines in the treatment of hypertension in general practice. Blood Press. 1998; 7:270-6.
PubMed
 
Hetlevik I, Holmen J, Krüger O, Kristensen P, Iversen H, Furuseth K.  Implementing clinical guidelines in the treatment of diabetes mellitus in general practice. Evaluation of effort, process, and patient outcome related to implementation of a computer-based decision support system. Int J Technol Assess Health Care. 2000; 16:210-27.
PubMed
CrossRef
 
Bosworth HB, Olsen MK, Dudley T, Orr M, Goldstein MK, Datta SK, et al..  Patient education and provider decision support to control blood pressure in primary care: a cluster randomized trial. Am Heart J. 2009; 157:450-6.
PubMed
CrossRef
 
Bosworth HB, Olsen MK, Goldstein MK, Orr M, Dudley T, McCant F, et al..  The veterans' study to improve the control of hypertension (V-STITCH): design and methodology. Contemp Clin Trials. 2005; 26:155-68.
PubMed
CrossRef
 
Buntin MB, Burke MF, Hoaglin MC, Blumenthal D.  The benefits of health information technology: a review of the recent literature shows predominantly positive results. Health Aff (Millwood). 2011; 30:464-71.
PubMed
CrossRef
 
Osheroff JA, Healthcare Information and Management Systems Society.  Improving Outcomes with Clinical Decision Support: An Implementer's Guide. 2nd ed. Chicago: Healthcare Information and Management Systems Society; 2005.
 

Letters

NOTE:
Citing articles are presented as examples only. In non-demo SCM6 implementation, integration with CrossRef’s "Cited By" API will populate this tab (http://www.crossref.org/citedby.html).

Comments

Submit a Comment
To the Editor
Posted on July 9, 2012
Edward Hoffer MD, FACP
MD, FACP
Conflict of Interest: None Declared

The article by Bright et al on Clinical Decision-Support Systems (CDSS) (1) was timely and interesting, but had an important omission. No mention was made of CDSS that focused on helping doctors make the correct diagnosis. Such systems have been broadly available for some 25 years, including QMR/Internist-I, ILIAD and DXplain. Currently, at least two general-purpose Diagnostic Decision Support systems, DXplain and ISABEL, are widely available. Reviews (2,3) have shown that these systems suggest important diseases the clinician had not previously considered, and a study (4) from the Mayo Clinic found that use of DXplain lowered length-of-stay and hospital costs.

1. Bright TJ et al “Effect of Clinical Decision-Support Systems” Ann Intern Med 2012;157:29-432.

2. Berner ES et al “Performance of Four Computer-Based Diagnostic Systems” New Engl J Med 1994; 330:1792-63.

3. Bond WF et al “Differential Diagnosis Generators: an Evaluation of Currently Available Computer Programs” J Gen Intern Med 2011;27:213-94.

4. Elkin PL et al “The Introduction of a Diagnostic Decision Support System (DXplain) into the Workflow of a Teaching Hospital Service can Decrease the Cost of Service for Diagnostically Challenging DRGs” Internat Jl Med Informatics 2010; 79:772-

Submit a Comment

Summary for Patients

Clinical Slide Sets

Terms of Use

The In the Clinic® slide sets are owned and copyrighted by the American College of Physicians (ACP). All text, graphics, trademarks, and other intellectual property incorporated into the slide sets remain the sole and exclusive property of the ACP. The slide sets may be used only by the person who downloads or purchases them and only for the purpose of presenting them during not-for-profit educational activities. Users may incorporate the entire slide set or selected individual slides into their own teaching presentations but may not alter the content of the slides in any way or remove the ACP copyright notice. Users may make print copies for use as hand-outs for the audience the user is personally addressing but may not otherwise reproduce or distribute the slides by any means or media, including but not limited to sending them as e-mail attachments, posting them on Internet or Intranet sites, publishing them in meeting proceedings, or making them available for sale or distribution in any unauthorized form, without the express written permission of the ACP. Unauthorized use of the In the Clinic slide sets will constitute copyright infringement.

Toolkit

Want to Subscribe?

Learn more about subscription options

Advertisement
Related Articles
Related Point of Care
Topic Collections