Cognitive Aids in Resuscitation (EIT #629): Systematic Review

Conflict of Interest Declaration

The ILCOR Continuous Evidence Evaluation process is guided by a rigorous ILCOR Conflict of Interest policy. The following Task Force members and other authors were recused from the discussion as they declared a conflict of interest: (none applicable)

The following Task Force members and other authors declared an intellectual conflict of interest and this was acknowledged and managed by the Task Force Chairs and Conflict of Interest committees: (none applicable)

CoSTR Citation

Gilfoyle E, Duff J, Bhanji F, Scholefield B, Bray J, Bigham B, Breckwoldt J, Cheng A, , Glerup Lauridsen K, Hsieh M, Iwami T, Lockey A, Ma M, Monsieurs K, Okamoto D, Pellegrino J, Yeung J, Finn J, Greif R on behalf of the EIT Task Force.

Cognitive Aids in Resuscitation Training Consensus on Science with Treatment Recommendations [Internet] Brussels, Belgium: International Liaison Committee on Resuscitation (ILCOR) Education, Implementation and Teams Task Force, 2020 January 3. Available from: http://ilcor.org

Methodological Preamble

The continuous evidence evaluation process for the production of Consensus on Science with Treatment Recommendations (CoSTR) started with a systematic review of the use of cognitive aids during resuscitation conducted by Elaine Gilfoyle, Jonathan Duff, Robert Greif, Farhan Bhanji and Barney Scholefield with involvement of clinical content experts. These data were taken into account when formulating the Treatment Recommendations. A previous CoSTR published in 2010 {Bhanji 2010 S920}, did not recommend routine use of checklists during resuscitation or training, but there has been a large body of literature published since then.

We defined a cognitive aid as: “presentation of prompts aimed to encourage recall of information in order to increase the likelihood of desired behaviors, decisions, and outcomes” {Fletcher 2014 148}. Examples of cognitive aids include checklists, device apps, video clips, pictures.

Our goal was to describe the impact of the use of cognitive aids during cardiopulmonary resuscitation in the real-life environment, however we found that there were no studies. Therefore, the TF elected to describe the literature that addressed the topic in 2 indirect ways: 1) real-life trauma resuscitation, where the clinical environment may be similar enough to cardiac arrest and 2) simulated cardiac arrest environments. The outcomes listed below refer to these 2 types of studies.

There was high heterogeneity among studies (such as types, format of intervention, methods of outcome assessments, duration of follow-up, timing of assessment). We were unable to perform a meta-analysis and have conducted a narrative synthesis of the findings. This synthesis is structured around each predefined outcome (simulation and trauma are discussed separately, per outcome).

PICOST

PROSPERO Registration Submitted November 23, 2019

Consensus on Science

1. For the critical outcome of survival to hospital discharge we identified no studies in the setting of cardiac arrest but found very low quality evidence in the setting of trauma resuscitation in three studies (one randomized trial {Fitzgerald 2011 218} and two observational studies {Bernhard 2007 362; Lasosher 2017 954} downgraded for risk of bias, indirectness and imprecision). These studies enrolled 4659 patients, but not all studies reported numbers of patients who survived, so calculating overall odds ratio was not possible.

2. For the important outcome of quality of performance in actual resuscitations, we identified no studies in the setting of cardiac arrest but found very low quality evidence in the setting of trauma resuscitation (one randomized trial {Fitzgerald 2011 218} and three observational studies { Bernhard 2007 362; Kelleher 2014 1129; Lasosher 2017 954}, downgraded for risk of bias, inconsistency, indirectness and imprecision). These studies enrolled 5094 patients, but reported quality of performance using different metrics, so calculating overall odds ratio was not possible.

Fitzgerald et al {Fitzgerald 2011 218} reported fewer errors in teams who used a cognitive aid (incident rate ratio 0.889, (95% CI 0.793-0.996), p=0.04) but found that compliance to trauma algorithms was not significantly improved with the use of a cognitive aid (incident rate ratio 1.020 (95%CI 0.989-1.051), p=0.21).

Lashosher et al {Lasosher 2017 954} reported that almost all aspects of completing primary and secondary trauma surveys improved with using the cognitive aid, and ordering radiologic investigations improved post vs pre p<0.001 except ordering abdominal CT scan.

Bernhard et al {Bernhard 2007 362} reported that time to completion of required radiologic investigations in trauma patients improved using a cognitive aid. They also found that teams performed more life-saving interventions (laparotomy and decompressive craniectomy) when using a cognitive aid (19% pre-implementation of cognitive aid vs 29% of post, p<0.05).

Kelleher et al {Kelleher 2014 1129} reported that most primary and secondary survey tasks were completed more consistently when teams used a cognitive aid. Primary and secondary survey tasks overall were more likely to be completed (primary survey: adjusted odds ratio [aOR] = 2.66, 95% CI 2.07 to 3.42; secondary survey: aOR = 2.46 (95% CI 2.04 to 2.98).{Kelleher 2014 1129} The average adjusted time to task completion was 9 seconds (–0.15 minutes, 95% CI = –0.23 to –0.08 minutes) faster in the postchecklist implementation period. {Kelleher 2014 1129}

3. For the important outcome of skill performance 1 year from course conclusion in simulated resuscitations, we identified no studies.

4. For the important outcome of time to starting CPR between course conclusion and 1 year in simulated resuscitations, we identified very low quality evidence in one randomized trial {Renna 2016 582}, downgraded for indirectness and imprecision). They examined this outcome in only four resuscitation teams and found no difference (15 sec without vs 14 sec with cognitive aid).

5. For the important outcome of chest compression rate between course conclusion and 1 year in simulated resuscitations, we identified very low quality evidence in two randomized trials {Ward 1997 221; Williamson 2005 140}, downgraded for risk of bias, inconsistency, indirectness and imprecision). Ward et al {Ward 1997 221} found no significant differences in the percentages of lay provider participants who performed the correct compression rate with no cognitive aid using either a short or long version of a checklist-type of cognitive aid (43% control vs 34% short vs 54% long, NS). Williamson et al {Williamson 2005 140}, found a significantly higher chest compression rate in lay provider participants who used a cognitive aid (94.5/min control vs 99.0/min cognitive aid, p<0.05), but note that neither group achieved a mean rate within the recommended rates of 100 to 120/min.

6. For the important outcome of chest compression depth between course conclusion and 1 year in simulated resuscitations, we identified very low quality evidence in two randomized trials {Ward 1997 221; Williamson 2005 140} downgraded for risk of bias, indirectness and imprecision).

Ward et al {Ward 1997 221} found no significant differences in the percentage of compressions with proper depth performed by lay provider participants who had access to either a short or long version of a checklist-type of cognitive aid (34% control vs 34% short vs 43% long, NS).

Williamson et al {Williamson 2005 140} found no significant differences in the percentage of compressions with proper depth performed by lay provider participants who had access to a cognitive aid (36.6mm control vs 42.2mm cognitive aid, NS). Note that neither group achieved a mean depth in the recommended range of 50 to 60 mm.

7. For the important outcome of chest compression fraction/hands off time (HOT) between course conclusion and 1 year in simulated resuscitations, we identified very low quality evidence in one randomized trial {Renna 2016 582}, downgraded for risk of bias, indirectness and imprecision). No significant differences in percentage HOT were found when resuscitation teams used a cognitive aid (18.9% when four teams did not vs 15.8% when four teams did use a cognitive aid, NS).

8. For the important outcome of ventilations between course conclusion and 1 year in simulated resuscitations, we identified very low quality evidence in two randomized trials {Ward 1997 221; Williamson 2005 140}, downgraded for risk of bias, indirectness and imprecision).

Ward et al {Ward 1997 221} found no significant differences in the percentage of ventilations with proper technique performed by lay provider participants who had access to either a short or long version of a checklist-type of cognitive aid (50% control vs 47% short vs 56% long, NS).

Williamson et al {Williamson 2005 140} found significant differences in the percentage of ventilations with proper tidal volume performed by lay provider participants who had access to a cognitive aid (audio prompts) (55.5% control vs 84.8% cognitive aid, p<0.01).

9. For the important outcome of time to start CPR at course conclusion in simulated resuscitations, we identified low quality evidence in four randomized trials {Hunt 2015 189; Merchant 2010 538; Paal 2012 472; Rössler 2013 982} downgraded for risk of bias, indirectness and imprecision) and one observational study {Renna 2016 582} downgraded for risk of bias, indirectness and imprecision). All studies demonstrated statistically significant and likely clinically significant delays in starting CPR for lay provider participants who used a cognitive aid compared with those who did not. Hunt 78.2 sec control vs 159.5 sec cognitive aid, p<0.001 {Hunt 2015 189}; Merchant 18 sec [95% CI 15 to 21 sec] control vs 48 sec [95% CI 47 to 49 sec] cognitive aid{Merchant 2010 538}, Paal 93.3 sec control vs 165.3 sec cognitive aid, p<0.001 {Paal 2012 472}, Rössler 23 sec control vs 63 sec flowchart, p<0.0001 {Rössler 2013 982}).

10. For the important outcome of chest compression rate at course conclusion in simulated resuscitations, we identified very low quality evidence from six randomized trials {Hunt 2015 189; Merchant 2010 538; Paal 2012 472; Rössler 2013 982; Ward 1997 221; Williamson 2005 140} downgraded for risk of bias, inconsistency, indirectness and imprecision).

Hunt et al {Hunt 2015 189}; reported no significant differences in mean chest compression rate by lay provider participants who used a cognitive aid (117/sec control vs 127.9/sec cognitive aid, NS). Merchant et al {Merchant 2010 538} reported a higher mean chest compression rate by lay provider participants who used a cognitive aid (compression rate (100/minute [95% confidence interval (CI) 97

to 103/minute] versus 44/minute [95% CI 38 to 50/minute]). Paal et al {Paal 2012 472} reported that a higher percentage of lay provider participants who used the correct chest compression rate when using a cognitive aid (14% control vs 44% cognitive aid, p<0.001).

Rössler et al {Rössler 2013 982} reported no significant differences in mean chest compression rate delivered by lay provider participants who used a cognitive aid (76/min control vs 78/min flowchart, NS).

Ward {Ward 1997 221} reported no significant differences in percentage of lay provider participants who used a correct chest compression rate when using either a short or long version of a checklist-type of cognitive aid (45% control vs 50% short vs 51% long, NS).

Williamson et al {Williamson 2005 140} reported a higher mean chest compression rate delivered by lay provider participants who used a cognitive aid (52.3.min control vs 87.3/min cognitive aid, p<0.01).

11. For the important outcome of chest compression depth at course conclusion in simulated resuscitations, we found low quality evidence from five randomized trials {Merchant 2010 538; Paal 2012 472; Rössler 2013 982; Ward 1997 221; Williamson 2005 140} downgraded for risk of bias, indirectness and imprecision). Only one study found a difference in chest compression depth achieved by lay provider participants but not in the recommended range of depth. Merchant 31 mm [95% CI 38 to 44 mm] control vs 41 mm [95% CI 28 to 34 mm ] cognitive aid {Merchant 2010 538}. All other studies showed no statistically significant or percentage of compressions in target using cognitive aids vs. not: Paal 21 mm control vs 21 mm cognitive aid {Paal 2012 472}, Rössler 41 mm control vs 42 mm cognitive aid {Rössler 2013 982}, Ward 46% control vs 46% short checklist vs 45% long checklist {Ward 1997 221}, Williamson 36.0% control vs 34.4% audio prompt {Williamson 2005 140}).

12. For the important outcome of chest compression fraction (CCF)/hands off time (HOT) at course conclusion in simulated resuscitations, we found very low quality evidence from four randomized trials {Hawkes 2015 1084; Hunt 2015 189; Merchant 2010 538; Rössler 2013 982}, downgraded for risk of bias, inconsistency and indirectness).

Hawkes {Hawkes 2015 1084} reported similar hands-off time in lay providers with and without a cognitive aid.

Hunt et al {Hunt 2015 189} showed no difference in CCF if layprovider participants used cognitive aids, but they included time to starting CPR (75.4% control vs 72.2% cognitive aid, NS). However, it is important to note that time to starting CPR was significantly longer in the cognitive aid group, so it is possible that CCF was actually better in the cognitive aid group, if time to starting CPR was taken into consideration.

Merchant et al {Merchant 2010 538} showed a difference in CCF between lay provider participants who used cognitive aids (50.6% control vs 58.9% cognitive aid), also the use of the cognitive aid was also accompanied by a delay in time to starting CPR.

Rössler et al {Rössler 2013 982} showed that, if accounting for delays in starting CPR, lay provider participants had lower HOT if used a cognitive aid (146 sec control vs 87 sec cognitive aid, p<0.0001).

13. For the important outcome of ventilations at course conclusion in simulated resuscitations, we found low quality evidence from three randomized trials {Paal 2012 472; Ward 1997 221; Williamson 2005 140}.

Paal {Paal 2012 472} reported that there was no difference in the percentage of participants who performed the correct ventilation rate if using cognitive aids or not (15% control vs 20% cognitive aid, NS).

Ward et al {Ward 1997 221} reported no differences in correct ventilations performed by lay provider participants using a checklist-type of cognitive aid (44% control vs 44% short vs 51% long, NS).

Williamson et al Williamson 2005 140} reported more ventilations performed with the correct technique by layprovider participants who used cognitive aids (control 15% vs 51% cognitive aids, p<0.01).

14. For the important outcome of knowledge at course conclusion in simulated resuscitations, we found no studies.

Treatment Recommendations

We recommend against the use of cognitive aids for the purposes of lay providers initiating CPR (weak recommendation, low certainty of evidence).

We suggest the use of cognitive aids for health care providers during trauma resuscitation (weak recommendation, very low certainty of evidence). In the absence of studies on cardiopulmonary resuscitation no evidence based recommendation can be made.

There is insufficient data to suggest for or against the use of cognitive aids in lay provider training.

We suggest the use of cognitive aids for training of health care providers in resuscitation (weak recommendation, very low certainty of evidence).

Justification and Evidence to Decision Framework Highlights

The EIT Task Force prioritized this topic because international resuscitation councils commonly provide cognitive aids to resuscitation course participants and health care organizations (algorithms, pocket cards, flowcharts, infographics, etc.). However, it has not been determined if they are effective in improving patient outcomes or provider performance during actual resuscitation.

Cognitive aids may improve performance and patient outcome by

• decreasing cognitive load of individuals or team collectively {Harrison 2006 554}.
• Limitations to working memory, systems 1 (i.e. automatic, fast, often unconscious) decision-making or cognitive processes and the impact of stress and distraction in resuscitation may impair rapid, accurate decision-making {Leblanc 2009 S25}, which can be improved by cognitive aids.
• standardizing communication among resuscitation team members {Leonard 2004 i90}
• allow for better situation awareness/shared mental model among team members {Stanton 2017 7}

However cognitive aids may

• promote fixation errors and groupthink {Kaba 2016 404}
• impair communication among team members {Marshall 2013 1169}
• be distracting especially when not developed well (flow, colour, how easy to read, confusing to follow etc) so may worsen performance/patient outcome

Our recommendation has been divided into different contexts, as we believe that the evidence for routine implantation of cognitive aids during resuscitation and training is conflicting. For lay providers, there is consistent evidence that there are potentially clinically important delays in initiating CPR; however, the evidence for impact on other CPR quality metrics (eg rate, depth, CCF) is less consistent.

There is almost no evidence for the use of cognitive aids by trained health care providers during cardiopulmonary resuscitation. However, there is substantial evidence, albeit not entirely consistent, available to show that trauma resuscitation teams generally adhere to resuscitation guidelines better, make fewer errors and perform key clinical tasks more frequently if they use cognitive aids. We believe that the trauma resuscitation environment is sufficiently similar to the cardiopulmonary resuscitation environment that our recommendation can be extrapolated. We acknowledge that our assumption may be incorrect, and that there may be important differences between the cardiac arrest and trauma resuscitation clinical environments,

When selecting our performance outcomes, we elected to include studies that measured data related to discrete tasks. There were a large number of studies that used composite scores as their primary outcome (e.g. score calculated based on completion of several clinical tasks). We excluded these studies for this systematic review, as it was very difficult to compare and consolidate the results.

None of the studies examined provided evidence to describe implementation concerns, e.g. training or resource implications. However, it appears feasible to provide cognitive aids for resuscitation providers to use during training and actual resuscitation.

Knowledge Gaps

Real-Life Cardiac Arrest Studies

As mentioned above, given that resuscitation councils are de facto endorsing the use of cognitive aids by providing pocket cards and algorithm posters, we feel that there is urgent need to adequately study the impact of cognitive aids in the real-world cardiac arrest environment.

Simulated Cardiac Arrest Studies with Healthcare Providers using cognitive aids

The one study that examines healthcare provider performance {Renna 2016 5822} is a very small proof of concept pilot study and was not sufficiently powered to be able to demonstrate any effects of cognitive aids on performance in this population. Future, larger studies in this area will allow us to strengthen our recommendation for this provider group.

Human Factors

There is no standard format to the types of cognitive aids developed and examined in the studies included in this systematic review. It is likely that providers respond differently to different kinds of cognitive aids, so it is very difficult to consolidate findings from different studies to form a unified conclusion.

There is much known about how human beings interact with cognitive aids in other clinical (e.g. WHO Safe Surgery Checklist) and non-clinical environments (e.g. aviation, power plants and large-scale industry). However, in order for the scientific community to develop the most effective, targeted cognitive aid for resuscitation, it should be a focus of research moving forward to examine human factors, specifically impacts on situational awareness (e.g. attention/distraction), cognitive load, and communication. This may help us better understand why cognitive aids seem to help providers perform some clinical tasks more completely and efficiently (e.g. trauma primary and secondary survey tasks) but seem to impair providers perform some other clinical tasks (e.g. initiating CPR).

Attachments

Evidence-to-Decision Table: EIT-629-Cognitive-Aids-TFSR

References

Bernhard M, Becker TK, Nowe T, Mohorovicic M, Sikinger M, Brenner T, Richter GM, Radeleff B, Meeder PJ, Büchler MW, Böttiger BW, Martin E, Gries A. Introduction of a treatment algorithm can improve the early management of emergency patients in the resuscitation room. Resuscitation. 2007

Jun;73(3):362-73. Epub 2007 Feb 6.

Bhanji F, Mancini ME, Sinz E, Rodgers DL, McNeil MA, Hoadley TA, Meeks RA, Hamilton MF, Meaney PA, Hunt EA, Nadkarni VM, Hazinski MF. Part 16: education, implementation, and teams: 2010 American Heart Association Guidelines for Cardiopulmonary Resuscitation and Emergency Cardiovascular Care. Circulation. 2010 Nov 2;122(18 Suppl 3):S920-33. doi: 10.1161/CIRCULATIONAHA.110.971135. Review.

Fitzgerald M, Cameron P, Mackenzie C, Farrow N, Scicluna P, Gocentas R, Bystrzycki A, Lee G, O'Reilly G, Andrianopoulos N, Dziukas L, Cooper DJ, Silvers A, Mori A, Murray A, Smith S, Xiao Y, Stub D, McDermott FT, Rosenfeld JV. Trauma resuscitation errors and computer-assisted decision support. Arch Surg. 2011 Feb;146(2):218-25. doi: 10.1001/archsurg.2010.333.

Fletcher KA, Bedwell WL. Cognitive aids: design suggestions for the medical field. InProceedings of the International Symposium on Human Factors and Ergonomics in Health Care 2014 Jun (Vol. 3, No. 1, pp. 148-152). Sage India: New Delhi, India: SAGE Publications

Harrison TK, Manser T, Howard SK, Gaba DM. Use of cognitive aids in a simulated anesthetic crisis. Anesth Analg. 2006 Sep;103(3):551-6.

Hawkes GA, Murphy G, Dempsey EM, Ryan AC. Randomised controlled trial of a mobile phone infant resuscitation guide. J Paediatr Child Health. 2015 Nov;51(11):1084-8. doi: 10.1111/jpc.12968. Epub 2015 Aug 19.

Hunt EA, Heine M, Shilkofski NS, Bradshaw JH, Nelson-McMillan K, Duval-Arnould J, Elfenbein R. Exploration of the impact of a voice activated decision support system (VADSS) with video on resuscitation performance by lay rescuers during simulated cardiopulmonary arrest. Emerg Med J. 2015 Mar;32(3):189-94. doi: 10.1136/emermed-2013-202867. Epub 2013 Nov 15.

Kaba A, Wishart I, Fraser K, Coderre S, McLaughlin K. Are we at risk of groupthink in our approach to teamwork interventions in health care? Med Educ. 2016 Apr;50(4):400-8. doi: 10.1111/medu.12943.

Kelleher DC, Carter EA, Waterhouse LJ, Parsons SE, Fritzeen JL, Burd RS. Effect of a checklist on advanced trauma life support task performance during pediatric trauma resuscitation. Acad Emerg Med. 2014 Oct;21(10):1129-34. doi: 10.1111/acem.12487.

Lashoher A, Schneider EB, Juillard C, Stevens K, Colantuoni E, Berry WR, Bloem C, Chadbunchachai W, Dharap S, Dy SM, Dziekan G, Gruen RL, Henry JA, Huwer C, Joshipura M, Kelley E, Krug E, Kumar V, Kyamanywa P, Mefire AC, Musafir M, Nathens AB, Ngendahayo E, Nguyen TS, Roy N, Pronovost PJ, Khan IQ, Razzak JA, Rubiano AM, Turner JA, Varghese M, Zakirova R, Mock C. Implementation of the World Health Organization Trauma Care Checklist Program in 11 Centers Across Multiple Economic Strata: Effect on Care Process Measures. World J Surg. 2017 Apr;41(4):954-962. doi: 10.1007/s00268-016-3759-8.

LeBlanc VR. The effects of acute stress on performance: implications for health professions education. Acad Med. 2009 Oct;84(10 Suppl):S25-33. doi: 10.1097/ACM.0b013e3181b37b8f. Review.

Leonard M, Graham S, Bonacum D. The human factor: the critical importance of effective teamwork and communication in providing safe care. Qual Saf Health Care. 2004 Oct;13 Suppl 1:i85-90.

Marshall S. The use of cognitive aids during emergencies in anesthesia: a review of the literature. Anesth Analg. 2013 Nov;117(5):1162-71. doi: 10.1213/ANE.0b013e31829c397b. Review.

Merchant RM, Abella BS, Abotsi EJ, Smith TM, Long JA, Trudeau ME, Leary M, Groeneveld PW, Becker LB, Asch DA. Cell phone cardiopulmonary resuscitation: audio instructions when needed by lay rescuers: a randomized, controlled trial. Ann Emerg Med. 2010 Jun;55(6):538-543.e1. doi: 10.1016/j.annemergmed.2010.01.020. Epub 2010 Mar 4.

Paal P, Pircher I, Baur T, Gruber E, Strasak AM, Herff H, Brugger H, Wenzel V, Mitterlechner T. Mobile phone-assisted basic life support augmented with a metronome. J Emerg Med. 2012 Sep;43(3):472-7. doi: 10.1016/j.jemermed.2011.09.011. Epub 2012 Jan 17.

Renna TD, Crooks S, Pigford AA, Clarkin C, Fraser AB, Bunting AC, Bould MD, Boet S. Cognitive Aids for Role Definition (CARD) to improve interprofessional team crisis resource management: An exploratory study. J Interprof Care. 2016 Sep;30(5):582-90. doi: 10.1080/13561820.2016.1179271. Epub 2016 Jun 13.

Rössler B, Ziegler M, Hüpfl M, Fleischhackl R, Krychtiuk KA, Schebesta K. Can a flowchart improve the quality of bystander cardiopulmonary resuscitation? Resuscitation. 2013 Jul;84(7):982-6. doi: 10.1016/j.resuscitation.2013.01.001. Epub 2013 Jan 7.

Stanton NA, Salmon PM, Walker GH, Salas E, Hancock PA. State-of-science: situation awareness in individuals, teams and systems. Ergonomics. 2017 Apr;60(4):449-466. doi: 10.1080/00140139.2017.1278796. Epub 2017 Feb 6. Review.

Ward P, Johnson LA, Mulligan NW, Ward MC, Jones DL. Improving cardiopulmonary resuscitation skills retention: effect of two checklists designed to prompt correct performance. Resuscitation. 1997 Jun;34(3):221-5.

Williamson LJ, Larsen PD, Tzeng YC, Galletly DC. Effect of automatic external defibrillator audio prompts on cardiopulmonary resuscitation performance. Emerg Med J. 2005 Feb;22(2):140-3.