0
Reviews |

IMPROVING PATIENT CARE

Health Information Technology: An Updated Systematic Review With a Focus on Meaningful Use FREE

Spencer S. Jones, PhD; Robert S. Rudin, PhD; Tanja Perry, BHM; and Paul G. Shekelle, MD, PhD
[+] Article and Author Information

From RAND Corporation and Southern California Evidence-based Practice Center, Santa Monica, California; Harvard Medical School and Brigham and Women's Hospital, Boston, Massachusetts; Greater Los Angeles Veterans Affairs Healthcare System, Los Angeles, California; and Vanguard Health Systems, Nashville, Tennessee.

Acknowledgment: The authors thank Aneesa Motala and Roberta Shanman for their research assistance and assistance with the literature searches and retrieval. They also thank the members of the Technical Expert Panel: David W. Bates, MD, MSc; George Hripsak, MD, MS; Philip J. Aponte, MD; Louise Liang, MD; and Paul Tang, MD, MS.

Grant Support: By the Office of the National Coordinator.

Potential Conflicts of Interest: Disclosures can be viewed at www.acponline.org/authors/icmje/ConflictOfInterestForms.do?msNum=M13-1531.

Requests for Single Reprints: Spencer S. Jones, PhD, Southern California Evidence-Based Practice Center, RAND Health, 1776 Main Street, Santa Monica, CA 90401; e-mail: spencer.jones@vanguardhealth.com.

Current Author Addresses: Drs. Jones and Shekelle and Ms. Perry: Southern California Evidence-Based Practice Center, RAND Health, 1776 Main Street, Santa Monica, CA 90401.

Dr. Rudin: RAND Corporation, 20 Park Plaza, Suite 920, Boston, MA 02116.

Author Contributions: Conception and design: S.S. Jones, R.S. Rudin, P.G. Shekelle.

Analysis and interpretation of the data: S.S. Jones, R.S. Rudin, P.G. Shekelle.

Drafting of the article: S.S. Jones, R.S. Rudin, T. Perry, P.G. Shekelle.

Critical revision of the article for important intellectual content: S.S. Jones, R.S. Rudin, P.G. Shekelle.

Final approval of the article: S.S. Jones, R.S. Rudin, T. Perry, P.G. Shekelle.

Provision of study materials or patients: S.S. Jones, P.G. Shekelle.

Statistical expertise: S.S. Jones.

Obtaining of funding: S.S. Jones, P.G. Shekelle.

Administrative, technical, or logistic support: T. Perry.

Collection and assembly of data: S.S. Jones, P.G. Shekelle.


Ann Intern Med. 2014;160(1):48-54. doi:10.7326/M13-1531
Text Size: A A A

Background: Incentives offered by the U.S. government have spurred marked increases in use of health information technology (IT).

Purpose: To update previous reviews and examine recent evidence that relates health IT functionalities prescribed in meaningful use regulations to key aspects of health care.

Data Sources: English-language articles in PubMed from January 2010 to August 2013.

Study Selection: 236 studies, including pre–post and time-series designs and clinical trials that related the use of health IT to quality, safety, or efficiency.

Data Extraction: Two independent reviewers extracted data on functionality, study outcomes, and context.

Data Synthesis: Fifty-seven percent of the 236 studies evaluated clinical decision support and computerized provider order entry, whereas other meaningful use functionalities were rarely evaluated. Fifty-six percent of studies reported uniformly positive results, and an additional 21% reported mixed-positive effects. Reporting of context and implementation details was poor, and 61% of studies did not report any contextual details beyond basic information.

Limitation: Potential for publication bias, and evaluated health IT systems and outcomes were heterogeneous and incompletely described.

Conclusion: Strong evidence supports the use of clinical decision support and computerized provider order entry. However, insufficient reporting of implementation and context of use makes it impossible to determine why some health IT implementations are successful and others are not. The most important improvement that can be made in health IT evaluations is increased reporting of the effects of implementation and context.

Primary Funding Source: Office of the National Coordinator.


In the United States, adoption of health information technology (IT) has been accelerated by the meaningful use incentive program, which provides financial incentives to individual health care providers and organizations that demonstrate that they use “certified” health IT to meet a set of several use criteria specified by the Centers for Medicare & Medicaid Services (12). This increase in use has been accompanied by a concomitant increase in the number of published evaluations of health IT. Because of the rapidly expanding evidence base, the Office of the National Coordinator requested a systematic update of the literature (3). The objective of this review is to update previous reviews (46) and examine recent evidence that relates health IT functionalities prescribed in meaningful use regulations to health care quality (including process, health, and patient and provider satisfaction outcomes), safety (including medication safety and other manifestations of patient safety), and efficiency (including costs, utilization, timeliness, and time burden of health care).

Although we did not develop a formal protocol for this update, we based it on the search strategy, inclusion and exclusion criteria, data collection, and synthesis methods from previous reviews on health IT (46).

Data Sources and Search Strategy

A 5-person technical expert panel, which included academic, health care delivery, and policy experts in health IT, guided the review process. Literature searches were based on the strategy initially used by Chaudhry and colleagues (4) and updated by Goldzweig (5) and Buntin (6) and their respective colleagues. This strategy uses broad-based search terms for the English-language literature indexed in PubMed. Our initial search covered the period of January 2010 to November 2011. We used a computer-aided screening method (7) to update that search to November 2012, and then updated searches again to August 2013 (Tables 1 and 2 of the Supplement). Our expert panel reviewed the search results and suggested additional articles that may have been missed.

Study Selection

Two expert reviewers used a Web-based system, DistillerSR (8), to independently select studies. Following the methods of our previous reviews, we considered “hypothesis-testing” studies of health IT effects and “descriptive quantitative” studies for inclusion. We classified articles as “hypothesis-testing” if the investigators compared data between groups or across periods and used statistical tests to assess differences. Hypothesis-testing studies were further classified by study design (such as randomized, controlled trials [RCTs]). To be included, a study needed to evaluate a health IT functionality encompassed by the meaningful use regulations. The meaningful use requirements specify 25 criteria for health IT functionality and use (such as “Use CPOE for medication orders”), of which providers must meet a portion to receive incentive payments (Tables 3 and 4 of the Supplement).

Data Synthesis and Analysis

Using a structured form, we abstracted information about the following: study design; clinical setting, health care conditions, and aspects of care assessed; research sites; health IT type (commercial or “homegrown”); meaningful use functionality evaluated; and context and implementation details. We adapted criteria developed to assess health IT applications in patient safety to classify articles according to reported context (4 domains) and implementation details (7 specific components) (9). We used a modified version of the outcome result classification framework (positive, mixed-positive, neutral, or negative), originally used by Buntin and colleagues (6), to assess outcomes. Our adaptations made the classification framework more conservative than the original framework, thus increasing the likelihood that an article's findings would be classified as mixed-positive, neutral, or negative (Table 5 of the Supplement). The functionality evaluated, context domains, implementation components, and article outcomes were classified by dual-review, and conflicts were resolved by consensus.

Role of the Funding Source

This project was done under contract to the Office of the National Coordinator. Representatives of the Office of the National Coordinator were briefed on study findings and reviewed a draft manuscript but were not involved in the analysis or decision to submit the manuscript for publication.

We deemed 2482 of 12 678 titles identified in the searches potentially relevant. We excluded 2023 of these after abstract review and another 223 after full-text review. The 236 articles selected for review encompassed 278 outcomes because some articles addressed several aspects of care and outcomes (Figure). The full list of the included studies can be found in the reference list in the Supplement. Quality outcomes (n = 170) were evaluated more than safety (n = 46) and efficiency (n = 62) outcomes combined, and care processes (n = 103) outcomes were more than twice as common as health outcomes (n = 47). Studies with a simple pre–post design were most common (31%), followed by RCTs (25%) and time-series studies (11%). More than one half (53%) evaluated commercial health IT products, although approximately one quarter did not report whether the products that were evaluated were commercial or “homegrown.”

Meaningful Use Functionalities

Most studies addressed clinical decision support (CDS)(n = 85 [36%]); computerized provider order entry (CPOE) (n = 49 [21%]); or multifunctional health IT interventions (n = 47 [20%]), which evaluated broad IT interventions, such as electronic health records (EHRs), that encompassed many of the functionalities required under meaningful use. Twelve of the 25 functionalities in the meaningful use regulations, such as “capacity to track vital signs” or “maintain medication allergy lists,” were not specifically evaluated in any studies. These features, however, were likely critical to the functionality of IT interventions (such as CDS) that were evaluated in many studies. Nevertheless, for some aspects of meaningful use, such as “implement systems to protect privacy and security,” no eligible studies were found.

Quality Outcomes

Overall, 147 articles assessed the effect of health IT on 170 quality-related outcomes (Tables 6 and 7 of the Supplement). More than one half of these studies assessed CDS alerts and reminders; the most commonly studied outcomes were medication management, screening and preventive care, and process quality for diabetes and venous thromboembolism. More than three quarters of studies of alerts and reminders, including large quasi-experimental studies and RCTs, reported positive effects. Notable among these included a 1-year prospective, cluster randomized trial in 12 primary care pediatric practices that reported that EHR-embedded CDS substantially improved the use of asthma control medications, spirometry, and the maintenance of care plans (10); a study of nearly 20 000 surgical patients that found that CDS was associated with a 30% increase in adherence to infection prevention guidelines (11); a controlled before-and-after study compared 360 primary care physicians in New York, New York, with a matched set of control physicians and reported that EHR-sensitive measures of process quality (such as eye examinations and urine testing for patients with diabetes) improved substantially only in practices that received “high levels of technical assistance” in implementing their EHRs (12); and a before-and-after study showing that a CDS intervention in a commercial health IT system was associated with a significant increase in venous thromboembolism prophylaxis and a substantial decline in the rate of venous thromboembolism among nearly 40 000 patients admitted to a single academic medical center (13). Eighteen percent of studies reported no statistically significant improvements in quality or even negative effects. An illustrative example of these studies reported that on the basis of a nationally representative sample of physician office visits, EHRs were associated with poor depression care among patients diagnosed with multiple chronic conditions. The authors of this study hypothesized that EHR workflows gave precedence to the treatment of physical conditions, and therefore psychosocial problems were left unaddressed (14).

Safety Outcomes

Forty-six studies investigated the effects of health IT functionalities on patient safety outcomes, focusing exclusively on medication safety (Tables 8 and 9 of the Supplement). Approximately 78% of these studies reported at least some positive effects. Of note, benefits were found for a wide range of medication safety outcomes in various care settings. Automated dose calculation features within CPOE systems were found to have significant relative reductions in errors in medication dosage ranging from 37% to 80% (1517), and authors of 1 study reported that innovative features, such as incorporation of an order verification screen with a patient picture, was associated with complete elimination of incorrect patient orders (18). However, a few studies reported that, in some cases, health IT did not have the desired effect on medication safety, and “alert fatigue” and incongruent workflows were described as barriers to successful use of these systems.

Efficiency Outcomes

We identified 58 articles that assessed the effect of health IT on 62 efficiency-related outcomes (Tables 10 and 11 of the Supplement). Cost effects ranged from a 75% decrease to a 69% increase in the targeted costs. A few of the studies clustered in the range of 6% to 12% increases in the targeted costs. Understanding the relationship between health IT and health care utilization is complicated by external factors, such as the payment environment in which the health care providers operate. We identified some large studies that reported that patient and provider access to health IT were associated with increased health care utilization. However, other studies found that health IT can have the opposite effect. Overall, 85% of the studies evaluating utilization concluded that the effects of health IT led to an appropriate increase or decrease in utilization. Finally, studies that assessed the time burden or timeliness of care reported mixed results. Positive findings included shorter emergency department length of stay, reduced diagnostic turnaround times, shorter time to the initiation of appropriate therapies, and more in-person time with patients (1927). Nevertheless, some studies found that health IT was associated with increased documentation and that in some instances, even when providers were able to spend more time with their patients, much of that time was spent interacting with the computer.

Effect of Context and Implementation

Most studies reported the size, location, and academic status of the facility where the health IT system was being implemented. They also reported that staff are trained on the new system. Only 9% of studies reported on 2 or more contextual domains in addition to size, location, and academic status, and only 8% of studies reported on 4 or more implementation components (Tables 12 to 15 of the Supplement). Only 5 studies reported adequately on both context and implementation. Sixty-one percent and 64% of studies reported nothing beyond the basics for context and implementation, respectively, and 42% reported nothing beyond the basics for both context and implementation. Among 64 studies that assessed the effectiveness of a specific health IT intervention in multiple sites, only 5 studies quantitatively assessed for a difference in health IT effectiveness across sites, and 4 additional studies qualitatively reported on context sensitivity of the intervention.

The most commonly reported context domains were financial status in 24% of studies (however, 70% of these received credit because they were done in countries with government-financed health care or the Veterans Affairs health systems or Kaiser Permanente, leaving only 12 U.S. studies reporting financial status) and existing infrastructure in 16%. The most commonly reported specific implementation components were timeline of implementation in 28% and description of education and training in 26%.

This review was done using methods similar to 3 previous broad-based reviews of health IT (46) between 1995 and 2013. We found that the number of published health IT evaluation studies is increasing rapidly: Such studies increased by approximately 13% per year before 2007, and roughly 25% per year from 2008 to 2012. Table 1 shows the total number of article outcomes arrayed by the type of health IT functionalities studied and health care outcomes evaluated for all health IT studies identified in the 4 literature reviews. The summary data show that CDS and CPOE have been studied extensively, and other functionalities, such as health information exchange and functionalities that allow patients to access their own electronic records, are not as well-studied. Evaluations of the effect of health IT on health care quality predominate and make up more than one half of all studies included in all 4 systematic reviews.

Table Jump PlaceholderTable 1. Health IT Evaluation Studies Between 1995 and 2013, by Study Outcome Type 

Table 2 presents data for the 2 most recent broad-based health IT systematic reviews (since July 2007) arrayed by functionality and a broad measure of their effect. Combined CDS and CPOE has been the subject of more than 230 evaluations, most of which have reported positive results. Multifunctional health IT interventions have also been studied extensively. Although most of these latter studies report positive results, they found mixed results more often than did studies of CPOE or CDS. In addition, studies of other specific meaningful use functionalities, such as health information exchange and e-prescribing, are much less common but also reported positive findings more often than not.

Table Jump PlaceholderTable 2. Health IT Evaluation Studies Between 2007 and 2013, by Study Outcome Result 

Our update shows that the published literature on health IT is expanding rapidly and that much of this expansion is attributable to commercial health IT systems. In the original review by Chaudhry and colleagues, studies of commercial systems constituted a negligible proportion of the literature (less than 5%). In the more recent review by Goldzweig and colleagues, this proportion had increased to only approximately 8%. In the current review, more than 50% of the eligible studies were explicitly about commercial health IT systems. This increase is a welcome change because Chaudhry and colleagues identified evaluation of commercial systems as the top research priority in health IT.

Although the health IT evaluation literature base is expanding rapidly, we are concerned that there has not been a commensurate increase in our understanding of the effect of health IT or how it can be used to improve health and health care. Study questions, research methods, and reporting of study details have not sufficiently adapted to meet the needs of clinicians, health care administrators, and health policymakers and are falling short of addressing the future needs of the health care system.

Nevertheless, some broad conclusions can be drawn. Most studies of CDS report positive or mixed-positive results, and existing systematic reviews of specific CDS systems are mostly positive with respect to changes in processes of care (2833). We conclude that CDS generally results in improvements in the processes targeted by the decision support. If neutral or negative results are reported in new studies of CDS, these results are more likely to be because of specifics of the particular intervention, context, or implementation than an indication that the general construct of computerized decision support is not a beneficial IT functionality for improving health care quality. The same is true for CPOE: Most evaluations have reported positive or mixed-positive effects, and most existing systematic reviews likewise conclude that CPOE reduces medication errors (3435). We can conclude that CPOE effectively decreases medication errors. Health care providers should be encouraged to adopt CDS and CPOE, and future studies of CDS and CPOE should concentrate on how to make them work better rather than testing the hypothesis of whether they work at all. In contrast to this, the evidence base on other functionalities, such as patient care reminders or patient specific education, have small numbers of studies, and any new studies add proportionately much more to our existing knowledge, both about the general construct of the functionality plus the potential for context and implementation sensitivity of the effects. The lack of reporting about key elements of context and implementation of health IT was noted in the review by Chaudhry and colleagues, and despite calls then and more recently for better reporting on context and implementation—and even suggestions for specific items to report on (3637)—we still find that crucial elements of context and implementation are missing from most published health IT studies.

This phenomenon of underreporting may be partially explained by the fact that early studies of health IT functionalities were trying to determine only whether a particular health IT functionality created value and to what extent. This was perhaps the most important research question when health IT was a novel phenomenon because it led to a demonstration of the potential of the new technology. With the increasing adoption of EHRs and other forms of health IT, it is no longer sufficient to ask whether health IT creates value; going forward, the most useful studies will help us understand how to realize value from health IT (38).

Stakeholders with interests in health IT, including policymakers, research funders, and journal editors, can help researchers shift their focus from if to how by promoting, soliciting, and publishing research that empirically studies the mediating effects of contextual and implementation factors on the relationship between health IT and key health care outcomes. Such a shift in focus will greatly advance the science of health IT evaluation and greatly increase our understanding of how the positive effects of health IT can be maximized and negative effects can be avoided or remediated. As a start, policymakers and other research funders could require that researchers funded by their organizations or reported in their journals explicitly analyze and report a basic set of contextual and implementation factors. Reporting frameworks for health IT intervention based on expert consensus are beginning to emerge (38), and other frameworks for similar health care interventions, such as patient safety interventions, could easily be adapted to health IT interventions (37).

The potential for publication bias is always a limitation of systematic reviews of this type. Researchers often do not evaluate the potential for negative effects, and even when identified, negative results are potentially less likely to be published. In addition, all of the included studies were weighted equally, regardless of study design or sample size. However, we considered factors that were related to the generalizability of the evidence, such as sample size, inclusion of several measures, and use of statistical methods, when drawing conclusions. Finally, health IT systems and the outcomes evaluated were heterogeneous and often incompletely described.

In sum, the health IT literature is expanding rapidly but failing to produce a commensurate amount of useful knowledge. Although most studies reported that health IT interventions had statistically and clinically significant benefits, sometimes these were not as large as the developers had expected, and there are also examples where benefits were not realized. Now that health IT is being widely adopted, researchers should refocus their efforts to show how health IT can be used to realize value. The most common characteristics of published studies are still pre–post studies of a CDS or CPOE at a single site that report nothing beyond the basics in terms of context and implementation. Such studies should be discouraged. The most important improvement that can be made in health IT evaluations is increased measurement of and reporting of context, implementation, and context-sensitivity of effectiveness.

The number of health information technology (IT) evaluation studies is rapidly increasing, driven primarily by increased evaluation of commercial health IT applications.

Most evaluations focus on clinical decision support and computerized provider order entry.

Most published health IT implementation studies report positive effects on quality, safety, and efficiency.

Insufficient reporting of contextual and implementation factors makes it impossible to determine why most health IT implementations are successful but some are not.

The most important improvement that can be made in health IT evaluations is increased measurement, analysis, and reporting of the effects of contextual and implementation factors.

Mathematica Policy Research, Harvard School of Public Health, Robert Wood Johnson Foundation.  Health Information Technology in the United States 2013: Better Information Systems for Better Care. 2013. Accessed at www.rwjf.org/en/research-publications/find-rwjf-research/2013/07/health-information-technology-in-the-united-states-2013.html on 21 August 2013.
 
Blumenthal D, Tavenner M. The “meaningful use” regulation for electronic health records. N Engl J Med. 2010; 363:501-4.
PubMed
CrossRef
 
Centers for Medicare & Medicaid Services (CMS), HHS. Medicare and Medicaid programs; electronic health record incentive program. Final rule. Fed Regist. 2010; 75:44313-588.
PubMed
 
Chaudhry B, Wang J, Wu S, Maglione M, Mojica W, Roth E, et al. Systematic review: impact of health information technology on quality, efficiency, and costs of medical care. Ann Intern Med. 2006; 144:742-52.
PubMed
CrossRef
 
Goldzweig CL, Towfigh A, Maglione M, Shekelle PG. Costs and benefits of health information technology: new trends from the literature. Health Aff (Millwood). 2009; 28:w282-93.
PubMed
CrossRef
 
Buntin MB, Burke MF, Hoaglin MC, Blumenthal D. The benefits of health information technology: a review of the recent literature shows predominantly positive results. Health Aff (Millwood). 2011; 30:464-71.
PubMed
CrossRef
 
Dalal SR, Shekelle PG, Hempel S, Newberry SJ, Motala A, Shetty KD. A pilot study using machine learning and domain knowledge to facilitate comparative effectiveness review updating. Med Decis Making. 2013; 33:343-55.
PubMed
CrossRef
 
Evidence Partners . DistillerSR. 2011. Accessed at http://systematic-review.net on 21 August 2013.
 
Shekelle PG, Pronovost PJ, Wachter RM, Taylor SL, Dy S, Foy R, et al; PSP Technical Expert Panel.  Assessing the Evidence for Context-Sensitive Effectiveness and Safety of Patient Safety Practices: Developing Criteria (Prepared under Contract no. HHSA-290-2009-10001C). AHRQ Publication No. 11-0006-EF. Rockville, MD: Agency for Healthcare Research and Quality; December 2010.
 
Bell LM, Grundmeier R, Localio R, Zorc J, Fiks AG, Zhang X, et al. Electronic health record-based decision support to improve asthma care: a cluster-randomized trial. Pediatrics. 2010; 125:770-7.
PubMed
CrossRef
 
Schwann NM, Bretz KA, Eid S, Burger T, Fry D, Ackler F, et al. Point-of-care electronic prompts: an effective means of increasing compliance, demonstrating quality, and improving outcome. Anesth Analg. 2011; 113:869-76.
PubMed
CrossRef
 
Ryan AM, Bishop TF, Shih S, Casalino LP. Small physician practices in New York needed sustained help to realize gains in quality from use of electronic health records. Health Aff (Millwood). 2013; 32:53-62.
PubMed
CrossRef
 
Galanter WL, Thambi M, Rosencranz H, Shah B, Falck S, Lin FJ, et al. Effects of clinical decision support on venous thromboembolism risk assessment, prophylaxis, and prevention at a university teaching hospital. Am J Health Syst Pharm. 2010; 67:1265-73.
PubMed
 
Harman JS, Rost KM, Harle CA, Cook RL. Electronic medical record availability and primary care depression treatment. J Gen Intern Med. 2012; 27:962-7.
PubMed
CrossRef
 
Terrell KM, Perkins AJ, Hui SL, Callahan CM, Dexter PR, Miller DK. Computerized decision support for medication dosing in renal insufficiency: a randomized, controlled trial. Ann Emerg Med. 2010; 56:623-9.
PubMed
CrossRef
 
Wang HY, Lu CL, Wu MP, Huang MH, Huang YB. Effectiveness of an integrated CPOE decision-supporting system with clinical pharmacist monitoring practice in preventing antibiotic dosing errors. Int J Clin Pharmacol Ther. 2012; 50:375-82.
PubMed
 
Chen AR, Lehmann CU. Computerized provider order entry in pediatric oncology: design, implementation, and outcomes. J Oncol Pract. 2011; 7:218-22.
PubMed
CrossRef
 
Hyman D, Laire M, Redmond D, Kaplan DW. The use of patient pictures and verification screens to reduce computerized provider order entry errors. Pediatrics. 2012; 130:211-9.
PubMed
 
Furukawa MF. Electronic medical records and the efficiency of hospital emergency departments. Med Care Res Rev. 2011; 68:75-95.
PubMed
 
Duffy WJ, Kharasch MS, Du H. Point of care documentation impact on the nurse-patient interaction. Nurs Adm Q. 2010; 34:1-E10.
PubMed
CrossRef
 
Bell DS, Cima L, Seiden DS, Nakazono TT, Alcouloumre MS, Cunningham WE. Effects of laboratory data exchange in the care of patients with HIV. Int J Med Inform. 2012; 81:74-82.
PubMed
CrossRef
 
Refuerzo JS, Straub H, Murphy R, Salter L, Ramin SM, Blackwell SC. Computerized physician order entry reduces medication turnaround time of labor induction agents. Am J Perinatol. 2011; 28:253-8.
PubMed
CrossRef
 
Georgiou A, Lang S, Rosenfeld D, Westbrook JI. The use of computerized provider order entry to improve the effectiveness and efficiency of coagulation testing. Arch Pathol Lab Med. 2011; 135:495-8.
PubMed
 
Spalding SC, Mayer PH, Ginde AA, Lowenstein SR, Yaron M. Impact of computerized physician order entry on ED patient length of stay. Am J Emerg Med. 2011; 29:207-11.
PubMed
CrossRef
 
Bennett KJ, Steen C. Electronic medical record customization and the impact upon chart completion rates. Fam Med. 2010; 42:338-42.
PubMed
 
Fernando TJ, Nguyen DD, Baraff LJ. Effect of electronically delivered prescriptions on compliance and pharmacy wait time among emergency department patients. Acad Emerg Med. 2012; 19:102-5.
PubMed
CrossRef
 
Bernstein JA, Imler DL, Sharek P, Longhurst CA. Improved physician work flow after integrating sign-out notes into the electronic medical record. Jt Comm J Qual Patient Saf. 2010; 36:72-8.
PubMed
 
Hemens BJ, Holbrook A, Tonkin M, Mackay JA, Weise-Kelly L, Navarro T, et al, CCDSS Systematic Review Team. Computerized clinical decision support systems for drug prescribing and management: a decision maker-researcher partnership systematic review. Implement Sci. 2011; 6:89.
PubMed
CrossRef
 
Nieuwlaat R, Connolly SJ, Mackay JA, Weise-Kelly L, Navarro T, Wilczynski NL, et al, CCDSS Systematic Review Team. Computerized clinical decision support systems for therapeutic drug monitoring and dosing: a decision maker–researcher partnership systematic review. Implement Sci. 2011; 6:90.
PubMed
CrossRef
 
Roshanov PS, Misra S, Gerstein HC, Garg AX, Sebaldt RJ, Mackay JA, et al, CCDSS Systematic Review Team. Computerized clinical decision support systems for chronic disease management: a decision maker–researcher partnership systematic review. Implement Sci. 2011; 6:92.
PubMed
CrossRef
 
Roshanov PS, You JJ, Dhaliwal J, Koff D, Mackay JA, Weise-Kelly L, et al, CCDSS Systematic Review Team. Can computerized clinical decision support systems improve practitioners' diagnostic test ordering behavior? A decision maker–researcher partnership systematic review. Implement Sci. 2011; 6:88.
PubMed
CrossRef
 
Sahota N, Lloyd R, Ramakrishna A, Mackay JA, Prorok JC, Weise-Kelly L, et al, CCDSS Systematic Review Team. Computerized clinical decision support systems for acute care management: a decision maker–researcher partnership systematic review of effects on process of care and patient outcomes. Implement Sci. 2011; 6:91.
PubMed
CrossRef
 
Souza NM, Sebaldt RJ, Mackay JA, Prorok JC, Weise-Kelly L, Navarro T, et al, CCDSS Systematic Review Team. Computerized clinical decision support systems for primary preventive care: a decision maker–researcher partnership systematic review of effects on process of care and patient outcomes. Implement Sci. 2011; 6:87.
PubMed
CrossRef
 
Radley DC, Wasserman MR, Olsho LE, Shoemaker SJ, Spranca MD, Bradshaw B. Reduction in medication errors in hospitals due to adoption of computerized provider order entry systems. J Am Med Inform Assoc. 2013; 20:470-6.
PubMed
CrossRef
 
Shamliyan TA, Duval S, Du J, Kane RL. Just what the doctor ordered. Review of the evidence of the impact of computerized physician order entry system on medication errors. Health Serv Res. 2008; 43:32-53.
PubMed
 
Shekelle PG, Pronovost PJ, Wachter RM, Taylor SL, Dy SM, Foy R, et al. Advancing the science of patient safety. Ann Intern Med. 2011; 154:693-6.
PubMed
CrossRef
 
Talmon J, Ammenwerth E, Brender J, de Keizer N, Nykänen P, Rigby M. STARE-HI—Statement on reporting of evaluation studies in Health Informatics. Int J Med Inform. 2009; 78:1-9.
PubMed
CrossRef
 
Jha A.  The Wrong Question on Electronic Health Records. 18 September 2012. Accessed at http://blogs.sph.harvard.edu/ashish-jha/the-wrong-question-on-electronic-health-records on 21 August 2013.
 

Tables

Table Jump PlaceholderTable 1. Health IT Evaluation Studies Between 1995 and 2013, by Study Outcome Type 
Table Jump PlaceholderTable 2. Health IT Evaluation Studies Between 2007 and 2013, by Study Outcome Result 

References

Mathematica Policy Research, Harvard School of Public Health, Robert Wood Johnson Foundation.  Health Information Technology in the United States 2013: Better Information Systems for Better Care. 2013. Accessed at www.rwjf.org/en/research-publications/find-rwjf-research/2013/07/health-information-technology-in-the-united-states-2013.html on 21 August 2013.
 
Blumenthal D, Tavenner M. The “meaningful use” regulation for electronic health records. N Engl J Med. 2010; 363:501-4.
PubMed
CrossRef
 
Centers for Medicare & Medicaid Services (CMS), HHS. Medicare and Medicaid programs; electronic health record incentive program. Final rule. Fed Regist. 2010; 75:44313-588.
PubMed
 
Chaudhry B, Wang J, Wu S, Maglione M, Mojica W, Roth E, et al. Systematic review: impact of health information technology on quality, efficiency, and costs of medical care. Ann Intern Med. 2006; 144:742-52.
PubMed
CrossRef
 
Goldzweig CL, Towfigh A, Maglione M, Shekelle PG. Costs and benefits of health information technology: new trends from the literature. Health Aff (Millwood). 2009; 28:w282-93.
PubMed
CrossRef
 
Buntin MB, Burke MF, Hoaglin MC, Blumenthal D. The benefits of health information technology: a review of the recent literature shows predominantly positive results. Health Aff (Millwood). 2011; 30:464-71.
PubMed
CrossRef
 
Dalal SR, Shekelle PG, Hempel S, Newberry SJ, Motala A, Shetty KD. A pilot study using machine learning and domain knowledge to facilitate comparative effectiveness review updating. Med Decis Making. 2013; 33:343-55.
PubMed
CrossRef
 
Evidence Partners . DistillerSR. 2011. Accessed at http://systematic-review.net on 21 August 2013.
 
Shekelle PG, Pronovost PJ, Wachter RM, Taylor SL, Dy S, Foy R, et al; PSP Technical Expert Panel.  Assessing the Evidence for Context-Sensitive Effectiveness and Safety of Patient Safety Practices: Developing Criteria (Prepared under Contract no. HHSA-290-2009-10001C). AHRQ Publication No. 11-0006-EF. Rockville, MD: Agency for Healthcare Research and Quality; December 2010.
 
Bell LM, Grundmeier R, Localio R, Zorc J, Fiks AG, Zhang X, et al. Electronic health record-based decision support to improve asthma care: a cluster-randomized trial. Pediatrics. 2010; 125:770-7.
PubMed
CrossRef
 
Schwann NM, Bretz KA, Eid S, Burger T, Fry D, Ackler F, et al. Point-of-care electronic prompts: an effective means of increasing compliance, demonstrating quality, and improving outcome. Anesth Analg. 2011; 113:869-76.
PubMed
CrossRef
 
Ryan AM, Bishop TF, Shih S, Casalino LP. Small physician practices in New York needed sustained help to realize gains in quality from use of electronic health records. Health Aff (Millwood). 2013; 32:53-62.
PubMed
CrossRef
 
Galanter WL, Thambi M, Rosencranz H, Shah B, Falck S, Lin FJ, et al. Effects of clinical decision support on venous thromboembolism risk assessment, prophylaxis, and prevention at a university teaching hospital. Am J Health Syst Pharm. 2010; 67:1265-73.
PubMed
 
Harman JS, Rost KM, Harle CA, Cook RL. Electronic medical record availability and primary care depression treatment. J Gen Intern Med. 2012; 27:962-7.
PubMed
CrossRef
 
Terrell KM, Perkins AJ, Hui SL, Callahan CM, Dexter PR, Miller DK. Computerized decision support for medication dosing in renal insufficiency: a randomized, controlled trial. Ann Emerg Med. 2010; 56:623-9.
PubMed
CrossRef
 
Wang HY, Lu CL, Wu MP, Huang MH, Huang YB. Effectiveness of an integrated CPOE decision-supporting system with clinical pharmacist monitoring practice in preventing antibiotic dosing errors. Int J Clin Pharmacol Ther. 2012; 50:375-82.
PubMed
 
Chen AR, Lehmann CU. Computerized provider order entry in pediatric oncology: design, implementation, and outcomes. J Oncol Pract. 2011; 7:218-22.
PubMed
CrossRef
 
Hyman D, Laire M, Redmond D, Kaplan DW. The use of patient pictures and verification screens to reduce computerized provider order entry errors. Pediatrics. 2012; 130:211-9.
PubMed
 
Furukawa MF. Electronic medical records and the efficiency of hospital emergency departments. Med Care Res Rev. 2011; 68:75-95.
PubMed
 
Duffy WJ, Kharasch MS, Du H. Point of care documentation impact on the nurse-patient interaction. Nurs Adm Q. 2010; 34:1-E10.
PubMed
CrossRef
 
Bell DS, Cima L, Seiden DS, Nakazono TT, Alcouloumre MS, Cunningham WE. Effects of laboratory data exchange in the care of patients with HIV. Int J Med Inform. 2012; 81:74-82.
PubMed
CrossRef
 
Refuerzo JS, Straub H, Murphy R, Salter L, Ramin SM, Blackwell SC. Computerized physician order entry reduces medication turnaround time of labor induction agents. Am J Perinatol. 2011; 28:253-8.
PubMed
CrossRef
 
Georgiou A, Lang S, Rosenfeld D, Westbrook JI. The use of computerized provider order entry to improve the effectiveness and efficiency of coagulation testing. Arch Pathol Lab Med. 2011; 135:495-8.
PubMed
 
Spalding SC, Mayer PH, Ginde AA, Lowenstein SR, Yaron M. Impact of computerized physician order entry on ED patient length of stay. Am J Emerg Med. 2011; 29:207-11.
PubMed
CrossRef
 
Bennett KJ, Steen C. Electronic medical record customization and the impact upon chart completion rates. Fam Med. 2010; 42:338-42.
PubMed
 
Fernando TJ, Nguyen DD, Baraff LJ. Effect of electronically delivered prescriptions on compliance and pharmacy wait time among emergency department patients. Acad Emerg Med. 2012; 19:102-5.
PubMed
CrossRef
 
Bernstein JA, Imler DL, Sharek P, Longhurst CA. Improved physician work flow after integrating sign-out notes into the electronic medical record. Jt Comm J Qual Patient Saf. 2010; 36:72-8.
PubMed
 
Hemens BJ, Holbrook A, Tonkin M, Mackay JA, Weise-Kelly L, Navarro T, et al, CCDSS Systematic Review Team. Computerized clinical decision support systems for drug prescribing and management: a decision maker-researcher partnership systematic review. Implement Sci. 2011; 6:89.
PubMed
CrossRef
 
Nieuwlaat R, Connolly SJ, Mackay JA, Weise-Kelly L, Navarro T, Wilczynski NL, et al, CCDSS Systematic Review Team. Computerized clinical decision support systems for therapeutic drug monitoring and dosing: a decision maker–researcher partnership systematic review. Implement Sci. 2011; 6:90.
PubMed
CrossRef
 
Roshanov PS, Misra S, Gerstein HC, Garg AX, Sebaldt RJ, Mackay JA, et al, CCDSS Systematic Review Team. Computerized clinical decision support systems for chronic disease management: a decision maker–researcher partnership systematic review. Implement Sci. 2011; 6:92.
PubMed
CrossRef
 
Roshanov PS, You JJ, Dhaliwal J, Koff D, Mackay JA, Weise-Kelly L, et al, CCDSS Systematic Review Team. Can computerized clinical decision support systems improve practitioners' diagnostic test ordering behavior? A decision maker–researcher partnership systematic review. Implement Sci. 2011; 6:88.
PubMed
CrossRef
 
Sahota N, Lloyd R, Ramakrishna A, Mackay JA, Prorok JC, Weise-Kelly L, et al, CCDSS Systematic Review Team. Computerized clinical decision support systems for acute care management: a decision maker–researcher partnership systematic review of effects on process of care and patient outcomes. Implement Sci. 2011; 6:91.
PubMed
CrossRef
 
Souza NM, Sebaldt RJ, Mackay JA, Prorok JC, Weise-Kelly L, Navarro T, et al, CCDSS Systematic Review Team. Computerized clinical decision support systems for primary preventive care: a decision maker–researcher partnership systematic review of effects on process of care and patient outcomes. Implement Sci. 2011; 6:87.
PubMed
CrossRef
 
Radley DC, Wasserman MR, Olsho LE, Shoemaker SJ, Spranca MD, Bradshaw B. Reduction in medication errors in hospitals due to adoption of computerized provider order entry systems. J Am Med Inform Assoc. 2013; 20:470-6.
PubMed
CrossRef
 
Shamliyan TA, Duval S, Du J, Kane RL. Just what the doctor ordered. Review of the evidence of the impact of computerized physician order entry system on medication errors. Health Serv Res. 2008; 43:32-53.
PubMed
 
Shekelle PG, Pronovost PJ, Wachter RM, Taylor SL, Dy SM, Foy R, et al. Advancing the science of patient safety. Ann Intern Med. 2011; 154:693-6.
PubMed
CrossRef
 
Talmon J, Ammenwerth E, Brender J, de Keizer N, Nykänen P, Rigby M. STARE-HI—Statement on reporting of evaluation studies in Health Informatics. Int J Med Inform. 2009; 78:1-9.
PubMed
CrossRef
 
Jha A.  The Wrong Question on Electronic Health Records. 18 September 2012. Accessed at http://blogs.sph.harvard.edu/ashish-jha/the-wrong-question-on-electronic-health-records on 21 August 2013.
 

Letters

NOTE:
Citing articles are presented as examples only. In non-demo SCM6 implementation, integration with CrossRef’s "Cited By" API will populate this tab (http://www.crossref.org/citedby.html).

Comments

Submit a Comment
Submit a Comment

Supplements

Supplemental Content
Supplement. Tables

Summary for Patients

Clinical Slide Sets

Terms of Use

The In the Clinic® slide sets are owned and copyrighted by the American College of Physicians (ACP). All text, graphics, trademarks, and other intellectual property incorporated into the slide sets remain the sole and exclusive property of the ACP. The slide sets may be used only by the person who downloads or purchases them and only for the purpose of presenting them during not-for-profit educational activities. Users may incorporate the entire slide set or selected individual slides into their own teaching presentations but may not alter the content of the slides in any way or remove the ACP copyright notice. Users may make print copies for use as hand-outs for the audience the user is personally addressing but may not otherwise reproduce or distribute the slides by any means or media, including but not limited to sending them as e-mail attachments, posting them on Internet or Intranet sites, publishing them in meeting proceedings, or making them available for sale or distribution in any unauthorized form, without the express written permission of the ACP. Unauthorized use of the In the Clinic slide sets will constitute copyright infringement.

Toolkit

Want to Subscribe?

Learn more about subscription options

Advertisement
Related Articles
Related Point of Care
Topic Collections
PubMed Articles
Forgot your password?
Enter your username and email address. We'll send you a reminder to the email address on record.
(Required)
(Required)