1 Cook DA, Hatala R, Brydges R, et al. Technology-enhanced simulation for health professions education: a systematic review and meta-analysis. JAMA 2011;306:978–88.
2 Cheng A, Grant V, Auerbach M. Using simulation to improve patient safety: dawn of a new era. JAMA Pediatr 2015;169:419.
3 Slakey DP, Simms ER, Rennie KV, et al. Using simulation to improve root cause analysis of adverse surgical outcomes. Int J Qual Health Care 2014;26:144–50.
4 Dieckmann P, Phero JC, Issenberg SB, et al. The first research consensus summit of the society for simulation in healthcare: conduction and a synthesis of the results. Simul Healthc 2011;6:S1–9.
5 Cheng A, Auerbach M, Calhoun A, et al. Building a community of practice for researchers: the international network for simulation-based pediatric innovation, research and education. Simul Healthc 2018;13:S28–34.
6 Pucher PH, Aggarwal R, Qurashi M, et al. Randomized clinical trial of the impact of surgical ward-care checklists on postoperative care in a simulated environment. Br J Surg 2014;101:1666–73.
7 Cheng A, Auerbach M, Hunt EA, et al. Designing and conducting simulation-based research. Pediatrics 2014;133:1091–101.
8 Munroe B, Buckley T, Curtis K, et al. Designing and implementing full immersion simulation as a research tool. Australas Emerg Nurs J 2016;19:90–105.
9 LeBlanc VR, Manser T, Weinger MB, et al. The study of factors affecting human and systems performance in healthcare using simulation. Simul Healthc 2011;6:S24–9.
10 Marshall M, Pronovost P, Dixon-Woods M. Promotion of improvement as a science. Lancet 2013;381:419–21.
11 The Health Foundation. Improvement science. Evidence scan. 22. London: The Health Foundation, 2011.
12 Dixon-Woods M. Systematic reviews and qualitative methods. In: Silverman D, ed. Qualitative research: theory, method and practice. 3rd edn. London: Sage, 2010:331–46.
13 Portela MC, Pronovost PJ, Woodcock T, et al. How to study improvement interventions: a brief overview of possible study types. BMJ Qual Saf 2015;24:325–36.
14 Eccles M, Grimshaw J, Campbell M, et al. Research designs for studies evaluating the effectiveness of change and improvement strategies. Qual Saf Health Care 2003;12:47–52.
15 Gaba DM. The future vision of simulation in healthcare. Simul Healthc 2007;2:126–35.
16 Tun JK, Alinier G, Tang J, et al. Redefining simulation fidelity for healthcare education. Simul Gaming 2015;46:159–74.
17 Dieckmann P, Rall M. Simulators in anaesthetic training to enhance patient safety. In: Cashman J, Grounds M, eds. Recent Advances in Anaesthesia and Intensive Care.
Cambridge: Cambridge University Press, 2007:213–32.18 Chiniara G, Cole G, Brisbin K, et al. Simulation in healthcare: a taxonomy and a conceptual framework for instructional design and media selection. Med Teach 2013;35:e1380–95.
19 Hamstra SJ, Brydges R, Hatala R, et al. Reconsidering fidelity in simulation-based training. Acad Med 2014;89:387–92.
20 Kutzin JM. Simulation Design Considerations 2.0: optimizing space and operations. Clin Simul Nurs 2016;12:187–96.
21 Patterson MD, Geis GL, Falcone RA, et al. In situ simulation: detection of safety threats and teamwork training in a high risk emergency department. BMJ Qual Saf 2013;22:468–77.
22 Patterson MD, Blike GT, Nadkarni VM, et al. In Situ Simulation: Challenges and Results. In: Henriksen K, Battles JB, Keyes MA, eds. Advances in patient safety: new directions and alternative approaches. Vol 3: Performance and Tools. Rockville (MD): Agency for Healthcare Research and Quality (US), 2008.
23 Taffinder NJ, McManus IC, Gul Y, et al. Effect of sleep deprivation on surgeons’ dexterity on laparoscopy simulator. Lancet 1998;352:1191.
24 Schuetz M, Gockel I, Beardi J, et al. Three different types of surgeon-specific stress reactions identified by laparoscopic simulation in a virtual scenario. Surg Endosc 2008;22:1263–7.
25 Leblanc VR, Regehr C, Tavares W, et al. The impact of stress on paramedic performance during simulated critical events. Prehosp Disaster Med 2012;27:369–74.
26 Cook DA. One drop at a time: research to advance the science of simulation. Simul Healthc 2010;5:1–4.
27 Thackray D, Roberts L. Exploring the clinical decision-making used by experienced cardiorespiratory physiotherapists: A mixed method qualitative design of simulation, video recording and think aloud techniques. Nurse Educ Today 2017;49:96–105.
28 Fanning RM, Gaba DM. The role of debriefing in simulation-based learning. Simul Healthc 2007;2:115–25.
29 Rudolph JW, Simon R, Rivard P, et al. Debriefing with good judgment: combining rigorous feedback with genuine inquiry. Anesthesiol Clin 2007;25:361–76.
30 Fossum M, Alexander GL, Göransson KE, et al. Registered nurses’ thinking strategies on malnutrition and pressure ulcers in nursing homes: a scenario-based think-aloud study. J Clin Nurs 2011;20(17-18):2425–35.
31 Bucknall TK, Forbes H, Phillips NM, et al. An analysis of nursing students’ decisionmaking in teams during simulations of acute patient deterioration. J Adv Nurs 2016;72:2482–94.
32 Cain CL, Riess ML, Gettrust L, et al. Malignant hyperthermia crisis: optimizing patient outcomes through simulation and interdisciplinary collaboration. AORN J 2014;99:300–11.
33 Harrison Y, Horne JA. The impact of sleep deprivation on decision making: a review. J Exp Psychol Appl 2000;6:236–49.
34 McNeer RR, Bennett CL, Dudaryk R. Intraoperative noise increases perceived task load and fatigue in anesthesiology residents: a simulation-based study. Anesth Analg 2016;122:2068–81.
35 Kaba A, Beran TN. Impact of peer pressure on accuracy of reporting vital signs: An interprofessional comparison between nursing and medical students. J Interprof Care 2016;30:116–22.
36 Kaba A, Beran TN, White D. Accuracy of interpreting vital signs in simulation: An empirical study of conformity between medical and nursing students. J Interprof Educ Pract 2016;3:9–18.
37 Goodacre S. Uncontrolled before-after studies: discouraged by Cochrane and the EMJ. Emerg Med J 2015;32:507–8.
38 Kobayashi L, Dunbar-Viveiros JA, Sheahan BA, et al. In situ simulation
comparing in-hospital first responder sudden cardiac arrest resuscitation using semiautomated defibrillators and automated external defibrillators. Simul Healthc 2010;5:82–90.
39 Hulscher ME, Laurant MG, Grol RP. Process evaluation on quality improvement interventions. Qual Saf Health Care 2003;12:40–6.
40 Moore GF, Audrey S, Barker M, et al. Process evaluation of complex interventions: medical research council guidance. BMJ 2015;350:h1258.
41 Cheng A, Kessler D, Mackinnon R, et al. Conducting multicenter research in healthcare simulation: Lessons learned from the INSPIRE network. Adv Simul 2017;2.
42 Cheng A, Hunt EA, Grant D, et al. Variability in quality of chest compressions provided during simulated cardiac arrest across nine pediatric institutions. Resuscitation 2015;97:13–19.
43 Alexander JA, Hearld LR. The science of quality improvement implementation: developing capacity to make a difference. Med Care 2011;49:S6–20.
44 Kessler JB, Vesterlund L. The External Validity of Laboratory Experiments: The Misleading Emphasis on Quantitative Effects. In: Fréchette GR, Schotter A, eds.
Handbook of Experimental Economic Methodology. Oxford: Oxford University Press, 2015:391–406.
45 Falk A, Heckman JJ. Lab experiments are a major source of knowledge in the social sciences. Science 2009;326:535–8.
46 Arriaga AF, Bader AM, Wong JM, et al. Simulation-Based trial of surgical-crisis checklists. N Engl J Med 2013;368:246–53.
47 Reames BN, Krell RW, Campbell DA, et al. A checklist-based intervention to improve surgical outcomes in Michigan: evaluation of the keystone surgery program. JAMA Surg 2015;150:208.
48 Haynes AB, Weiser TG, Berry WR, et al. A surgical safety checklist to reduce morbidity and mortality in a global population. N Engl J Med 2009;360:491–9.
49 Bergs J, Lambrechts F, Simons P, et al. Barriers and facilitators related to the implementation of surgical safety checklists: a systematic review of the qualitative evidence. BMJ Qual Saf 2015;24:776–86.
50 Schmidt E, Goldhaber-Fiebert SN, Ho LA, et al. Simulation exercises as a patient safety strategy: a systematic review. Ann Intern Med 2013;158:426.
51 Korndorffer JR, Arora S, Sevdalis N, et al. The American College of Surgeons/Association of Program directors in surgery national skills curriculum: adoption rate, challenges and strategies for effective implementation into surgical residency programs. Surgery 2013;154:13–20.
52 Roe BE, Just DR. Internal and external validity in economics research: tradeoffs between experiments, field experiments, natural experiments, and field data. Am J Agric Econ 2009;91:1266–71.
53 Mitchell G. Revisiting truth or triviality: the external validity of research in the psychological laboratory. Perspect Psychol Sci 2012;7:109–17.
54 Schram A. Artificiality: The tension between internal and external validity in economic experiments. J Eco Method 2005;12:225–37.
55 Scandura TA, Williams EA. Research methodology in management: current practices, trends, and implications for future research. Acad Manage J 2000;43:1248–64.
56 Zendejas B, Brydges R, Wang AT, et al. Patient outcomes in simulation-based medical education: a systematic review. J Gen Intern Med 2013;28:1078–89.
57 Pucher PH, Tamblyn R, Boorman D, et al. Simulation research to enhance patient safety and outcomes: recommendations of the simnovate patient safety domain group. BMJ Simul Technol Enhanc Learn 2017;3(Suppl 1):S3–7.
58 Benishek LE, Lazzara EH, Gaught WL, et al. The Template of Events for Applied and Critical Healthcare Simulation (TEACH Sim): a tool for systematic simulation scenario design. Simul Healthc 2015;10:21–30.
59 Alinier G. Developing high-fidelity health care simulation scenarios: a guide for educators and professionals. Simul Gaming 2011;42:9–26.
60 Rothwell PM. External validity of randomised controlled trials: “to whom do the results of this trial apply?”. Lancet 2005;365:82–93.
61 Glasgow RE. What types of evidence are most needed to advance behavioral medicine? Ann Behav Med 2008;35:19–25.
62 Steckler A, McLeroy KR. The importance of external validity. Am J Public Health 2008;98:9–10.
63 Vissers G, Heyne G, Peters V, et al. The validity of laboratory research in social and behavioral science. Quality and Quantity 2001;35:129–45.
64 Levitt SD, List JA. Viewpoint: On the generalizability of lab behaviour to the
field: Generalizability of lab behaviour to the field. Can J Eco/Revue canadienne d’économique 2007;40:347–70.
65 Jones J, Wilkins M, Caird J, et al. An experimental study on the impact of clinical interruptions on simulated trainee performances of central venous catheterization. Adv Simul 2017;2:5.
66 Anderson CA, Lindsay JJ, Bushman BJ. Research in the psychological laboratory: truth or triviality? Current Dir Psyc Sci 1999;8:3–9.
67 Cook DA, Beckman TJ. Current concepts in validity and reliability for psychometric instruments: theory and application. Am J Med 2006;119:166.e7–16.
68 Downing SM. Validity: on meaningful interpretation of assessment data. Med Educ 2003;37:830–7.
69 Beaubien JM, Baker DP. The use of simulation for training teamwork skills in health care: how low can you go? Qual Saf Health Care 2004;13:i51–6.
70 Fürst RV, Polimanti AC, Galego SJ, et al. Ultrasound-guided vascular access simulator for medical training: proposal of a simple, economic and effective model. World J Surg 2017;41:681–6.
71 Dieckmann P, Gaba D, Rall M. Deepening the theoretical foundations of patient simulation as social practice. Simul Healthc 2007;2:183–93.
72 Norman G, Dore K, Grierson L. The minimal relationship between simulation fidelity and transfer of learning. Med Educ 2012;46:636–47.
73 Grantcharov TP, Bardram L, Funch-Jensen P, et al. Laparoscopic performance after one night on call in a surgical department: prospective study. BMJ 2001;323:1222–3.
74 Crofts JF, Bartlett C, Ellis D, et al. Training for shoulder dystocia: a trial of simulation using low-fidelity and high-fidelity mannequins. Obstet Gynecol 2006;108:1477–85.
75 Sheringham J, Sequeira R, Myles J, et al. Variations in GPs’ decisions to investigate suspected lung cancer: a factorial experiment using multimedia vignettes. BMJ Qual Saf 2017;26:449–59.
76 Roussin CJ, Larraz E, Jamieson K, et al. Psychological safety, self-efficacy, and speaking up in interprofessional health care simulation. Clin Simul Nurs 2018;17:38–46.
77 Rudolph JW, Raemer DB, Simon R. Establishing a safe container for learning in simulation: the role of the presimulation briefing. Simul Healthc 2014;9:339–49.
78 Cheng A, Kessler D, Mackinnon R, et al. Reporting guidelines for health care simulation research: extensions to the CONSORT and STROBE Statements. Simul Healthc 2016;11:238–48.
79 Neuschwander A, Job A, Younes A, et al. Impact of sleep deprivation on anaesthesia residents’ non-technical skills: a pilot simulation-based prospective randomized trial. Br J Anaesth 2017;119:125–31.
80 Truog RD, Meyer EC. Deception and death in medical simulation. Simul Healthc 2013;8:1–3.
81 Calhoun AW, Boone MC, Miller KH, et al. Case and commentary: using simulation to address hierarchy issues during medical crises. Simul Healthc 2013;8:13–19.
82 Gaba DM. Simulations that are challenging to the psyche of participants: how much should we worry and about what? Simul Healthc 2013;8:4–7.
83 Corvetto MA, Taekman JM. To die or not to die? A review of simulated death. Simul Healthc 2013;8:8–12.
84 Smith AB, Lammers SE. The Ethics of Simulation. In: Palaganas JC, Maxworthy JC, Epps CA, eds. Defining excellence in simulation programs. First edn. Philadelphia: Wolters Kluwer, 2015:592–6.
85 Scerbo MW, Calhoun AW, Paige JT, et al. The Second society for simulation in healthcare research summit: beyond our boundaries. Simul Healthc 2018;13:S1–6.
86 Paige JT, Terry Fairbanks RJ, Gaba DM. Priorities related to improving healthcare safety through simulation. Simul Healthc 2018;13:S41–50.
87 Greenhalgh T, Thorne S, Malterud K. Time to challenge the spurious hierarchy of systematic over narrative reviews? Eur J Clin Invest 2018;48:e12931.
88 Raemer DB, Kolbe M, Minehart RD, et al. Improving Anesthesiologists’ ability to speak up in the operating room: a randomized controlled experiment of a simulation-based intervention and a qualitative analysis of hurdles and enablers. Acad Med 2016;91:530–9.
89 Cumin D, Skilton C, Weller J. Information transfer in multidisciplinary operating room teams: a simulation-based observational study. BMJ Qual Saf 2017;26:209–16.
90 Nielsen DS, Dieckmann P, Mohr M, et al. Augmenting Health Care Failure Modes and Effects Analysis With Simulation. Simul Healthc 2014;9:48–55.
91 Medwid K, Smith S, Gang M. Use of in-situ simulation to investigate latent safety threats prior to opening a new emergency department. Saf Sci 2015;77:19–24.
92 Garcia BH, Elenjord R, Bjornstad C, et al. Safety and efficiency of a new generic package labelling: a before and after study in a simulated setting. BMJ Qual Saf 2017;26:817–23.
93 Ersdal HL, Vossius C, Bayo E, et al. A one-day helping babies breathe course improves simulated performance but not clinical management of neonates. Resuscitation 2013;84:1422–7.
94 Thomas I, Nicol L, Regan L, et al. Driven to distraction: a prospective controlled study of a simulated ward round experience to improve patient safety teaching for medical students. BMJ Qual Saf 2015;24:154–61.
95 Clarke S, Horeczko T, Carlisle M, et al. Emergency medicine resident crisis resource management ability: a simulation-based longitudinal study. Med Educ Online 2014;19:25771.
96 Renju J, Andrew B, Nyalali K, et al. A process evaluation of the scale up of a youthfriendly health services initiative in northern Tanzania. J Int AIDS Soc 2010;13:32.
97 Clay-Williams R, Baysari M, Taylor N, et al. Service provider perceptions of transitioning from audio to video capability in a telehealth system: a qualitative evaluation. BMC Health Serv Res 2017;17:558.
98 Ben-Assuli O, Ziv A, Sagi D, et al. Cost-Effectiveness Evaluation of EHR: simulation of an abdominal aortic aneurysm in the emergency department. J Med Syst 2016;40:141.
99 Endestad T, Wortinger LA, Madsen S, et al. Package design affects accuracy recognition for medications. Hum Factors 2016;58:1206–16.
100 Kessler DO, Walsh B, Whitfill T, et al. Disparities in adherence to pediatric sepsis guidelines across a spectrum of emergency departments: a multicenter, crosssectional observational in situ simulation study. J Emerg Med 2016;50:403–15.