Simulation Exercises as a Patient Safety Strategy: A Systematic Review FREE

Simulation is a versatile technique that may be applied in patient safety strategies across a variety of factors that contribute to patient harm.

Heterogeneous evidence across multiple topic areas shows that training with simulation-based exercises increases technical and procedural performance.

Heterogeneous evidence shows that simulation-based exercises can improve team performance and interpersonal dynamics.

Limited evidence suggests that improvements in patient outcomes attributable to simulation exercises can occur at the health system level.

It is well-known, both in medical practice and in other professions, that error rates decrease with experience (1). Yet, an important challenge is how to train physicians and ensure that they maintain competency while minimizing the potential for patient harm. Many medical educators now regard the traditional medical training model “see one, do one, teach one” as unstructured and inadequate (2). In contrast, simulation exercises allow patient-safe clinician training. Although all physicians must eventually perform procedures on and manage critical events for an actual patient for the first time, simulation can make initial interactions with patients safer.

Clinical expertise and mastery within a specialty do not increase simply as a function of experience (3), and likewise, patient safety issues are not likely to decrease simply as a function of more practice hours. Deliberate practice, or practice that includes reflection on performance, increases mastery (4), and simulation exercises offer opportunities for deliberate practice with flexibility to adjust procedure complexity and provide regular practice for rare treatments. Experienced clinicians also must maintain proficiency in a wide array of skills, most of which are known to deteriorate over time without practice (5). Simulation can serve to maintain clinical skills and may be part of maintenance of board certification, as is the case for the American Board of Anesthesiology (6).

The versatility of simulation techniques affords many potential benefits to those working to improve patient safety. First, simulation is designed to match user needs and has been associated with increased technical performance (7) and knowledge acquisition and clinical reasoning (8). Second, simulation can replicate rare, complex, or high-stakes scenarios known to affect individual and team performance (9 - 10). Third, mistakes are not only allowed in simulation, they enhance learning through reflection and debriefing (11 - 12). Fourth, new technologies or procedures may be tested in simulation before implementation in real time with real patients (13). Finally, teams can simulate patient care flow in situ for critical events (14) or adequacy of new facilities and equipment (15).

However, studies evaluating the relationship between these benefits and patient safety outcomes, including potential harms, have not been thoroughly evaluated. The purpose of this systematic review is to examine evidence on the benefits and harms of using simulation to improve patient safety in medicine.

Research demonstrating the benefits of simulation comes from studies about simulation as well as studies using simulation (16). Research about simulation directly examines the effect of a simulation technique as an intervention on behaviors and actions at the health professional or team level that could directly improve patient safety if that training were widely implemented. In contrast, studies using simulation harness these techniques as a laboratory to investigate new technologies and human performance for insights into potential causal pathways to improve safety.

Simulation is used along the translational pathway from health care provider actions in simulated “laboratory” contexts to similar actions in clinical settings to patient outcomes (17). As such, simulation is considered a technique rather than any 1 specific technology (18).

Simulation to enhance patient safety has 4 general purposes: education (for example, in transitioning trainees from content knowledge to experiential practice, and in continuing education); assessment (for example, in quality control or quality improvement, or usability testing); research (for example, regarding clinician behaviors) and health system integration (for example, team processes) (19). These purposes are not mutually exclusive, and each may span a range of complexity. A classic low-fidelity example of partial task training is simulation of intramuscular medication administration by inserting a needle into an orange. Individual dynamic medical management exercises may include high-fidelity simulations that utilize anatomically accurate mannequins and vital sign monitors. Patient safety may also be enhanced through full scenario team management, in which a human patient simulator and a fully simulated care environment, such as entire operating rooms or emergency department bays, are utilized.

On a basic level, simulations improve patient safety by allowing physicians to become better trained without putting patients at risk and, importantly, by providing protected time for reflection and debriefing—where most of the learning takes place (11 - 12). The challenge is matching the best simulation method to the desired learning objectives while recognizing the costs of each method (18). Because simulation is a broad technique, faculty training and time are often a more important investment than are specific expensive simulation equipment. Practitioners must be appropriately trained to effectively use simulation techniques, as well as any specific technologies, to accomplish the relevant training, assessment, or systems probing goals.

The simulation needs to feel real enough for participants to be able to suspend disbelief, enabling them to feel, think, and act much as they would in a real scenario (12,19). If the learning objective is mainly to practice cognitive skills for diagnosis or treatment, a verbal simulation, such as, “What would you do if …,” may be sufficient. In contrast, if development of management skills, such as situational awareness or team communication, is the focus, a more accurate replication of the actions and team presence become important for the simulation experience.

Methodology to capture literature in this review involved 3 mechanisms. First, we used structured search strategies to search PubMed and the Cochrane Library from their beginning to 31 October 2012. These searches were limited to meta-analyses; systematic reviews; and randomized, controlled trials (RCTs) or observational studies published in English. Second, practitioners with expertise in simulation provided recommendations on key articles, including issues in implementation and empirical research on simulation and patient safety. Finally, the reference lists of articles captured by using the first 2 methods were scanned for relevant literature. Abstracts of references captured by these searches (n = 913) were screened by a single reviewer.

Studies, including systematic reviews and meta-analyses, were included in the review if they reported evaluative results of patient outcomes or changes in clinician actions in patient care. Studies that only provided laboratory-based results were excluded.

Data from the 38 studies that met the inclusion criteria were abstracted by a single investigator. Given the varied nature of the included studies and the broad area of simulation, quality assessment of individual studies was limited to reporting study design. Selected studies are described with narrative synthesis. The Supplement provides a complete description of the search strategies, article flow diagram, and evidence tables.

This review was supported by the Agency for Healthcare Research and Quality, which had no role in the selection or review of the evidence or the decision to submit the manuscript for publication.

Of the 38 included studies, 22 were RCTs, 11 were prospective observational studies, and 5 were retrospective analyses of previous simulation interventions. Table 1 shows the distribution of study methodology and aspects of simulation interventions by targeted areas for improvement in patient safety. Thirty-four studies reported patient outcomes from care provided by trainees at varied levels of education or specialties; postgraduate residents and fellows were highly represented. Of the 27 studies that specified a setting for the simulation, academic medical settings predominated (n = 23).

Five RCTs and 1 prospective before–after study on training for colonoscopy and upper gastrointestinal endoscopy found better initial performance in actual patients when physicians received simulation-based training (20 - 25). Studies generally reported a similar training period requirement to ultimately reach desirable levels of procedure mastery (20 - 25). Safety outcomes focused primarily on patient discomfort (for example, insufflation). Simulation training was associated with less discomfort in 1 study (20), no difference in another (21), and greater patient discomfort in a third study (24). No critical patient safety events or major complications were reported. A systematic review (26) that addressed virtual reality–based simulation for endoscopy also found no studies that reported major complications or critical patient safety events.

In a prospective randomized mixed-methods study on simulation training for bronchoscopy, there was no observed difference in procedure time between participants who did and did not have simulation training (27). Another study showed that training for thoracentesis was associated with fewer pneumothoraces and procedures advancing to thoracostomy when coupled with simulation (28). Finally, cordocentesis procedure time was shorter and success rate was higher with simulation training, although there were no statistically significant differences in procedure-related fetal loss or overall fetal loss (29).

A meta-analysis of laparoscopic training with virtual reality simulators reported that procedure time was no faster but was more accurate among simulation-trained clinicians than traditional video-trained clinicians (standardized mean difference, 0.68 [95% CI, 0.05 to 1.31]) (30). Simulation training for laparoscopic cholecystectomy was associated with improved performance, 3-fold fewer errors, an 8-fold decreased variation in error making (31), and increased “respect for tissue” during the procedure (32 - 34). Laparoscopic simulation practice improved global scores on the Objective Structured Assessment of Technical Skills (OSATS) during cholecystectomies (35). Simulation training for extraperitoneal hernia repair was associated with increased individual OSATS item scores for knowledge of procedure, knowledge of instruments, and use of assistants, but this association was not significant when these individual item scores were aggregated into a global OSATS score (36). Cataract surgeries performed by residents trained with simulation had a lower rate of sentinel complications than did surgeries performed by residents who were trained before simulation was implemented (37). Finally, faster procedures and improved performance during prostate resection was observed among physicians trained with simulation (38).

A recent meta-analysis of RCTs and observational studies (39) showed that simulation-based education in central venous catherization techniques improved learner outcomes and performance during actual procedures. For example, simulation-based education resulted in fewer needle passes (standardized mean difference, −0.58 [CI, −0.95 to −0.20]) and reduced pneumothoraces (relative risk, 0.62 [CI, 0.40 to 0.97]) (39).

Several RCTs and observational studies that we reviewed confirmed that simulation-based education improved performance (40 - 51). Two prospective studies and 1 RCT reported that simulation training decreased rates of catheter-related bloodstream infection (41,46,49), but 1 prospective controlled cohort study reported no difference in rates (48) attributable to simulation-based training. Studies showed mixed results for other major complications and critical patient safety events.

Three RCTs reported data on other procedures and processes. In 1 RCT, a simulation-based training curriculum for pediatric residents using high-fidelity models was associated with non–statistically significant increases in performance of basic clinical procedural skills, such as bag–mask ventilation, venipuncture, peripheral venous catheter placement, and lumbar puncture (52). Bachelor's-level nursing students made fewer medication administration errors in external training rotations when simulation training was added to coursework (53). Among paramedic students, simulation-based training did not lead to improved performance during their first 15 intubations in terms of overall success rate, success rate on first attempt, or complications (54).

Researchers retrospectively investigated the effect of an annual mandatory 1-day workshop and training program for all midwifery (including community-based practitioners) and obstetric emergency staff in a tertiary care center (55). The workshop used simulation exercises for 7 common obstetric emergencies: shoulder dystocia, postpartum hemorrhage, eclampsia, delivery of twins, breech presentation, adult resuscitation, and neonatal resuscitation. Compared with the 2-year period before the training program was implemented, there was a statistically significant decrease in the rate of births with 5-minute Apgar scores of 6 or less and hypoxic–ischemic encephalopathy in the 2-year period after implementation. The decrease in rate of moderate to severe hypoxic–ischemic encephalopathy only approached statistical significance.

In an RCT (56), primary care physicians in a large multidisciplinary medical group were randomly assigned to 1 of 3 groups: no simulation control, simulation alone, or simulation combined with a physician leader program. The simulation training provided a series of interactive virtual encounters with patients who had newly diagnosed diabetes or who had indicated or contraindicated adjustments to their insulin regimen. When combined for comparison with the control group, physicians in the simulation groups prescribed renal-contraindicated metformin significantly less often to patients with diabetes. In another RCT (57), residents who participated in full-scenario simulation training for elective coronary artery bypass graft surgery had increased Anesthesiologists' Nontechnical Skills Assessment scores, and this difference was observed at 5-week follow-up.

Three studies reported patient outcomes after simulation-based training for resuscitation teams (14,58 - 59). An RCT (58) reported no differences attributable to simulation training for actual team performance on rates of ventilation, return of spontaneous circulation, or survival to discharge. However, a prospective before–after study examined resuscitation outcomes after implementation of the TeamSTEPPS team-building program coupled with simulation (59). This study reported several improvements in communication, as well as reductions in time to computed tomography, intubation, and the operating room. Finally, in a retrospective case–control study (14), simulation training was associated with a higher correct response rate based on the American Heart Association standards for resuscitation.

Studies generally provided additive or supplemental interventions to training as usual, and no study reported data indicating increased potential for or actual harm to patients that resulted from implementing simulation techniques. However, it is conceivable that simulation exercises would place demand on valuable resources that could be applied elsewhere in patient safety efforts. We found no evaluations of such considerations.

A meta-analysis (8) of simulation in education programs for health professionals found that 564 of 609 studies (92.6%) examined techniques provided through dedicated simulation centers. Thirty-four studies (5.6%) examined simulation in situ, and 11 studies (1.8%) reported from both contexts. Among studies cited in our review, academic medical systems and academically affiliated hospitals predominated (21 - 23,27 - 29,35 - 36,38,41 - 46,49 - 54,58,60 - 61). However, studies also reported outcomes of use of simulation in tertiary care facilities (25,45,50,58), trauma centers (44,59), and multispecialty medical groups (55 - 56). We found no reported data on the effect of context on the effectiveness of simulation exercises for improving patient safety.

Gaba (18) conceptualized a framework for simulation techniques that may aid implementation and ultimately enhance patient safety (Table 2). The framework includes 11 dimensions that form a comprehensive set of considerations proposed to enhance the development and the effectiveness of simulation exercises. Application of each dimension guides specification and decision making on critical choices about the simulation exercise. In practice, objectives of implementing simulation are aligned with the needs of learners and the goals of trainers from level of participation to training in the particular simulation technique. In addition, sufficient time for creating meaningful exercises with debriefing, equipment matched to the simulation need that recreates sufficient realism, and adequate space or storage for in situ simulations will increase the likelihood of success (18). Resources used in simulations must be available when needed and kept safe from being used inappropriately for patient care (for example, expired medications). Technical support and maintenance may be required for complex or high-tech simulators.

Rosen and colleagues (19) highlighted the importance of cognitive fidelity (vs. physical fidelity) in a simulated exercise: Simulations that engage the participant in ways that cognitively best reflect the actual task are likely to be more effective. Debriefing is considered crucial when implementing simulation requires instructor training (11 - 12) and is considered a best practice in simulation-based medical education (62).

The cost of implementing simulation exercises ranges from low to high, depending on the type of exercise and personnel and equipment resources involved (18). Instructor and learner time are likely to be the most expensive and crucial aspects of simulation in the long run. Start-up costs for a comprehensive simulation center may be accounted for differently from ongoing costs for exercises, which complicates the ability to categorize the expected cost for simulation as a patient safety strategy. Unfortunately, research addressing cost savings attributable directly to simulation remains sparse, although some studies have reported up to a 7-to-1 return on simulation costs through reduction in hospital days for bloodstream infections (21,49).

Simulation has continued to gain momentum in patient safety efforts in the past decade because it allows for exercising and improving aspects of health care delivery without any known risks to patients. Simulation has been used in patient safety for the purposes of education, assessment, research, and integration of system-level strategies. These efforts have been reported in the literature as research about simulation: that is, research evaluating the translation of simulation-based education to enhanced patient safety. In contrast, other research has focused on using simulation as a laboratory to discover potential leverage points for patient safety (16).

Our review found that studies reporting patient outcomes or systems of care have been done primarily in academic settings, although researchers have used simulation in diverse clinical specialties, experience levels, and care settings. These studies varied in terms of individual quality, but the majority were randomized or had methodologically sound controlled prospective designs. Researchers have replicated standardized simulation training for central venous catherization, and although this approach is promising for patient safety in that area, we did not find other examples of replication studies in our review. We also did not find analysis of contextual effects on the validity of simulation to improve patient safety. The generalizability of any one technique is likely to vary according to many factors, such as those in Gaba's 11-dimensional framework (18), and the adequacy of resources dedicated to simulation (for example, debriefing).

At this juncture, simulation seems to have a favorable effect on quicker acquisition and improved performance of technical skills. Although not yet thoroughly studied, simulation of complex or high-stakes procedures seems to be a promising technique to increase patient-safe behavior at the clinician and team levels. Simulation has the potential to enhance patient safety through structured assessment and debriefing in quality improvement initiatives. It has been used to assess practices that would be difficult or unsafe to study empirically in real time with actual patients. Likewise, simulation has been endorsed for ongoing competency and continuing education, as well as advancement to mastery-level practices.

A previous systematic review (7) reported that simulation contributes to enhanced knowledge acquisition and improved clinical performance. Simulation techniques have been used in translating results from the within-simulation laboratory to patient- and health care system–level outcomes (17). Another systematic review (4) suggested that protected time for debriefing in a learning experience is a crucial component of simulation techniques. To our knowledge, our review is the first to examine the effects that simulation exercises have on patient safety outcomes, and in particular outcomes in patients outside of simulation laboratory settings (that is, during clinical care).

Our review has limitations. First, it is possible that the broad search strategies missed studies that may be captured with targeted and comprehensive strategies dedicated to each simulation technique, clinical specialty, or application. Second, given the relative infancy of the research on simulation exercises, the field may be prone to selective reporting of studies with positive findings, leading to potential publication bias. Finally, we limited our assessment of quality of evidence to study design and did not perform a structured assessment of the strength of evidence. Therefore, the overall strength of the evidence for simulation exercises to improve patient safety should be interpreted with caution.

In conclusion, simulation is a versatile technique that continues to gain momentum in a variety of clinical settings and applications, including patient safety strategies. Although evidence is largely heterogeneous at this time, our review suggests the potential for simulation exercises to contribute to patient safety through increased technical and procedural performance and improved team performance. Limited research using health system–level observations suggests that simulation may enhance patient safety, although more research is needed on the potential for simulation to contribute to system-level differences in patient safety outcomes. Systematic reviews of simulation for specific procedures have begun reporting patient safety outcomes (26,30); more reviews of this nature would enhance our understanding of the overall contribution of simulation techniques to patient safety. Future systematic reviews would benefit from investigators using a consistent framework, such as that developed by Gaba (18), to describe the intervention and its context and implementation.