System Performance Improvement (EIT #640): Systematic Review

Conflict of Interest Declaration

The ILCOR Continuous Evidence Evaluation process is guided by a rigorous ILCOR Conflict of Interest policy. The following Task Force members and other authors declared an intellectual conflict of interest and this was acknowledged and managed by the Task Force Chairs and Conflict of Interest committees: no conflict of interest.

CoSTR Citation

Hsieh M, Ma M, Ko Y, Bigham B, Bhanji F, Bray J, Breckwoldt J, Cheng A, Duff J, Glerup Lauridsen K, Gilfoyle E, Iwami T, Lockey A, Monsieurs K, Okamoto D, Pellegrino J, Yeung J, Finn J, Greif R on behalf of the EIT Task Force.

System Performance Improvement Consensus on Science with Treatment Recommendations [Internet] Brussels, Belgium: International Liaison Committee on Resuscitation (ILCOR) Education, Implementation and Teams Task Force, 2020 January 3. Available from: http://ilcor.org

Methodological Preamble (and Link to Published Systematic Review if applicable)

The continuous evidence evaluation process for the production of Consensus on Science with Treatment Recommendations (CoSTR) started with a systematic review of system performance improvement conducted by the librarian at the library of National Taiwan University Hospital, Taipei, Taiwan with involvement of clinical content experts. Evidence for literature associated system performance improvement was sought and considered by the Education, Implementation and Teams Task Force.

PICOST

The PICOST (Population, Intervention, Comparator, Outcome, Study Designs and Timeframe)

Population: Among resuscitation systems who are caring for patients in cardiac arrest in any setting.

Intervention: System performance improvement.

Comparators: No system performance improvement.

Outcomes: Survival with favorable neurologic outcome at discharge (critical); Survival to hospital discharge (critical); Skill performance in actual resuscitations (important); Survival to admission (important); System level improvement (important)

Study Designs: Inclusion: (1) Studies associated with system performance improvement for personnel in organizations or systems caring for patients with cardiac arrest. System performance improvement is defined as hospital-level, community-level or country-level improvement related to structure, care pathways, process and quality of care. (2) All years and all languages are included as long as there is an English abstract. (3) Randomized controlled trials (RCTs) and non-randomized studies (non-randomized controlled trials, interrupted time series, controlled before-and-after studies, cohort studies, case-control studies).

Exclusion: Unpublished studies (e.g., conference abstracts, trial protocols), letters, editorials, comments and case reports.

Timeframe: (1) The results of the same review (EIT 640) was published in 2015 ILCOR guidelines (Resuscitation 2015;95:e203–e224). The search date was until July 1, 2014. (2) Re-running existing search strategy: Since Nov 1, 2013 to Nov 14, 2019.

PROSPERO submission [161882]: Dec 11, 2019

Risk of bias assessment for randomized controlled trials.

^a CPR process information was available in 74% of the subjects.

Risk of bias assessment for non-randomized controlled trials.

Interventions among included studies

Consensus on Science

For the critical outcome of “survival with favorable neurologic outcome at discharge”, we identified moderate certainty of evidence from 1 cluster-randomized trial {Hostler 2011 d512} (downgraded for imprecision) and very low certainty of evidence from 18 non-randomized controlled trails {Davis 2015 63; Hwang 2017 87; Pearson 2016 165; Sporer 2017 1; Kim 2017 e016925; Park 2018 124; Hopkins 2016 e002892; Hubner 2017 38; Wolfe 2014 1688; Couper 2015 2321; Stub 2015 45; Anderson 2016 37; Knight 2014 243; Rios 2019 234; Grunau 2018 118; Diepen 2017 e005716; Ewy 2013 113; Bradley 2012 1349} (downgraded for risk of bias). Among these studies, different interventions for system performance improvement were implemented, in different contexts (IHCA vs OHCA) and the heterogeneity of the studies precludes any meta-analysis. Thirteen of these studies {Davis 2015 63; Hwang 2017 87; Pearson 2016 165; Sporer 2017 1; Kim 2017 e016925; Park 2018 124; Hubner 2017 38; Wolfe 2014 1688; Stub 2015 45; Anderson 2016 37; Rios 2019 234; Grunau 2018 118; Ewy 2013 113} showed that patients had significantly higher chance of survival with favorable neurologic outcome at discharge after interventions for system performance improvement were implemented. The other six studies {Hopkins 2016 e002892; Couper 2015 2321; Knight 2014 243; Diepen 2017 e005716; Hostler 2011 d512; Bradley 2012 1349}, including one cluster-randomized trial {Hostler 2011 d512}, showed no significant improvement after interventions were implemented.

For the critical outcome of “survival to hospital discharge”, we identified moderate certainty of evidence from 1 cluster-randomized trial {Hostler 2011 d512} (downgraded for imprecision) and very low certainty of evidence from 21 non-randomized controlled trails {Davis 2015 63; Hwang 2017 87; Spitzer 2019 158; Pearson 2016 165; Sporer 2017 1; Kim 2017 e016925; Park 2018 124; Hopkins 2016 e002892; Hubner 2017 38; Wolfe 2014 1688; Couper 2015 2321; Stub 2015 45; Anderson 2016 37; Knight 2014 243; Rios 2019 234; Grunau 2018 118; Diepen 2017 e005716; Nehme 2015 56; Ewy 2013 113; Edelson 2008 1063; Bradley 2012 1349} (downgraded for risk of bias). The heterogeneity of the studies precludes any meta-analysis. Fourteen of these studies {Davis 2015 63; Hwang 2017 87; Pearson 2016 165; Kim 2017 e016925; Park 2018 124; Hubner 2017 38; Wolfe 2014 1688; Stub 2015 45; Anderson 2016 37; Knight 2014 243; Rios 2019 234; Grunau 2018 118; Nehme 2015 56; Ewy 2013 113} showed that patients had significantly higher chance of survival to hospital discharge after interventions for system performance improvement were implemented. The other eight studies {Spitzer 2019 158; Sporer 2017 1; Hopkins 2016 e002892; Couper 2015 2321; Diepen 2017 e005716; Hostler 2011 d512; Edelson 2008 1063; Bradley 2012 1349}, including one cluster-randomized trial {Hostler 2011 d512}, showed no significant improvement after interventions were implemented.

For the important outcome of “skill performance in actual resuscitations”, we identified moderate certainty of evidence from 1 cluster-randomized trial {Hostler 2011 d512} (downgraded for risk of bias) and very low certainty of evidence from 13 non-randomized controlled trails {Hwang 2017 87; Spitzer 2019 158; Weston 2017 96; Hunt 2018 e009860; Hubner 2017 38; Wolfe 2014 1688; Couper 2015 2321; Knight 2014 243; Grunau 2018 118; Olasveengen 2007 427; Lyon 2012 70; Edelson 2008 1063; Bradley 2012 1349} (downgraded for risk of bias). The heterogeneity of the studies precludes any meta-analysis. The interventions of these studies all consisted of strategies to improve the quality of resuscitation, including skills of basic life support and advanced life support. Twelve of these studies {Hwang 2017 87; Spitzer 2019 158; Weston 2017 96; Hunt 2018 e009860; Hubner 2017 38; Wolfe 2014 1688; Knight 2014 243; Grunau 2018 118; Lyon 2012 70; Hostler 2011 d512; Edelson 2008 1063; Bradley 2012 1349}, including one cluster-randomized trial {Hostler 2011 d512}, reported that rescuers had significant improved skill performance in actual resuscitations after interventions were implemented. The other two studies {Couper 2015 2321; Olasveengen 2007 427} showed no significant improvement after interventions were implemented.

For the important outcome of “survival to admission”, we have identified moderate certainty of evidence, from 1 cluster-randomized trial {Hostler 2011 d512} (downgraded for imprecision) and very low certainty of evidence from 5 non-randomized controlled trails {Pearson 2016 165; Sporer 2017 1; Hopkins 2016 e002892; Rios 2019 234; Nehme 2015 56}(downgraded for risk of bias). The heterogeneity of the studies precludes any meta-analysis. Three of these studies {Pearson 2016 165; Rios 2019 234; Nehme 2015 56} showed that patients had significantly higher chance of survival to admission after interventions for system performance improvement were implemented. The other three studies {Sporer 2017 1; Hopkins 2016 e002892; Hostler 2011 d512}, including one cluster-randomized trial {Hostler 2011 d512}, showed no significant improvement after interventions were implemented.

For the important outcome of “system level improvement”, we have identified very low certainty of evidence (downgraded for risk of bias), from 11 non-randomized controlled trials {Davis 2015 63; Hwang 2017 87; Sporer 2017 1; Kim 2017 e016925; Park 2018 124; Hopkins 2016 e002892; Adabag 2017 95; Rios 2019 234; Grunau 2018 118; Diepen 2017 e005716; Nehme 2015 56}. The heterogeneity of the studies precludes any meta-analysis. All studies included individual interventions to improve specific system level variables, and all studies achieved all or partial goals. These system level variables included rate of bystander cardiopulmonary resuscitation (CPR) or automated external defibrillators, rate of prehospital or in-hospital therapeutic hypothermia, and the use of automatic CPR device and CPR feedback device.

Treatment Recommendations

We recommend that organisations or communities that treat cardiac arrest evaluate their performance and target key areas with the goal to improve performance. (Strong recommendation, Very low certainty of evidence)

Justification and Evidence to Decision Framework Highlights

We recognize that the evidence in support of this recommendation comes from studies, most of which are of moderate to very low certainty of evidence. However, the majority of studies found that interventions to improve system performance not only improved system level variables and skill performance in actual resuscitations among rescuers, but also clinical outcomes of patients with out-of-hospital or in-hospital cardiac arrest, such as survival to hospital discharge and survival with favorable neurologic outcome at discharge. We recognize that such interventions need money, personnel, and stakeholder buy-in to improve system performance. Some systems may not have adequate resources to implement system performance improvement.

Values and preferences statement: In making this recommendation we place increased value on the benefits of system performance improvement, which have no known risks, given our knowledge that system performance improvement could show a large effect size in a beneficial direction.

Knowledge Gaps

(1) Identify the most appropriate strategy to improve system performance.

(2) Better understand the influence of local community and organizational characteristics.

(3) To evaluate the cost-effectiveness of the individual interventions for improving system performance.

Attachments

Evidence-to-Decision Table: EIT 640 System Performance Improvement

References

Adabag S, Hodgson L, Garcia S, Anand V, Frascone R, Conterato M, et al. Outcomes of sudden cardiac arrest in a state-wide integrated resuscitation program: Results from the Minnesota Resuscitation Consortium. Resuscitation. 2017;110:95-100.

Anderson ML, Nichol G, Dai D, Chan PS, Thomas L, Al-Khatib SM, et al. Association between hospital process composite performance and patient outcomes after in-hospital cardiac arrest care. JAMA Cardiology. 2016;1(1):37-45.

Bradley SM, Huszti E, Warren SA, Merchant RM, Sayre MR, Nichol G. Duration of hospital participation in Get With the Guidelines-Resuscitation and survival of in-hospital cardiac arrest. Resuscitation. 2012 Nov;83(11):1349-57.

Couper K, Kimani PK, Abella BS, Chilwan M, Cooke MW, Davies RP, et al. The system-wide effect of real-time audiovisual feedback and postevent debriefing for in-hospital cardiac arrest: The cardiopulmonary resuscitation quality improvement initiative. Critical Care Medicine. 2015;43(11):2321-31.

Davis DP, Graham PG, Husa RD, Lawrence B, Minokadeh A, Altieri K, et al. A performance improvement-based resuscitation programme reduces arrest incidence and increases survival from in-hospital cardiac arrest. Resuscitation. 2015;92:63-9.

Del Rios M, Weber J, Pugach O, Nguyen H, Campbell T, Islam S, et al. Large urban center improves out-of-hospital cardiac arrest survival. Resuscitation. 2019;139:234-40.

Edelson DP, Litzinger B, Arora V, Walsh D, Kim S, Lauderdale DS, et al. Improving in-hospital cardiac arrest process and outcomes with performance debriefing. Arch Intern Med. 2008 May 26;168(10):1063-9.

Ewy GA, Sanders AB. Alternative approach to improving survival of patients with out-of-hospital primary cardiac arrest. J Am Coll Cardiol. 2013 Jan 15;61(2):113-8.

Grunau B, Kawano T, Dick W, Straight R, Connolly H, Schlamp R, et al. Trends in care processes and survival following prehospital resuscitation improvement initiatives for out-of-hospital cardiac arrest in British Columbia, 2006-2016. Resuscitation. 2018;125:118-25.

Hopkins CL, Burk C, Moser S, Meersman J, Baldwin C, Youngquist ST. Implementation of pit crew approach and cardiopulmonary resuscitation metrics for out-of-hospital cardiac arrest improves patient survival and neurological outcome. Journal of the American Heart Association. 2016;5(1):e002892.

Hostler D, Everson-Stewart S, Rea TD, Stiell IG, Callaway CW, Kudenchuk PJ, et al. Effect of real-time feedback during cardiopulmonary resuscitation outside hospital: prospective, cluster-randomised trial. BMJ. 2011 Feb 4;342:d512.

Hubner P, Lobmeyr E, Wallmüller C, Poppe M, Datler P, Keferböck M, et al. Improvements in the quality of advanced life support and patient outcome after implementation of a standardized real-life post-resuscitation feedback system. Resuscitation. 2017;120:38-44.

Hunt EA, Jeffers J, McNamara L, Newton H, Ford K, Bernier M, et al. Improved cardiopulmonary resuscitation performance with CODE ACES2: A resuscitation quality bundle. Journal of the American Heart Association. 2018;7(24):e009860.

Hwang WS, Park JS, Kim SJ, Hong YS, Moon SW, Lee SW. A system-wide approach from the community to the hospital for improving neurologic outcomes in out-of-hospital cardiac arrest patients. European journal of emergency medicine : official journal of the European Society for Emergency Medicine. 2017;24(2):87-95.

Kim YT, Shin SD, Hong SO, Ahn KO, Ro YS, Song KJ, et al. Effect of national implementation of utstein recommendation from the global resuscitation alliance on ten steps to improve outcomes from Out-of-Hospital cardiac arrest: A ten-year observational study in Korea. BMJ Open. 2017;7(8):e016925.

Knight LJ, Gabhart JM, Earnest KS, Leong KM, Anglemyer A, Franzon D. Improving code team performance and survival outcomes: implementation of pediatric resuscitation team training. Critical care medicine. 2014;42(2):243-51.

Lyon RM, Clarke S, Milligan D, Clegg GR. Resuscitation feedback and targeted education improves quality of pre-hospital resuscitation in Scotland. Resuscitation. 2012 Jan;83(1):70-5.

Nehme Z, Bernard S, Cameron P, Bray JE, Meredith IT, Lijovic M, et al. Using a cardiac arrest registry to measure the quality of emergency medical service care: decade of findings from the Victorian Ambulance Cardiac Arrest Registry. Circ Cardiovasc Qual Outcomes. 2015 Jan;8(1):56-66.

Olasveengen TM, Tomlinson AE, Wik L, Sunde K, Steen PA, Myklebust H, et al. A failed attempt to improve quality of out-of-hospital CPR through performance evaluation. Prehosp Emerg Care. 2007 Oct-Dec;11(4):427-33.

Park JH, Shin SD, Ro YS, Song KJ, Hong KJ, Kim TH, et al. Implementation of a bundle of Utstein cardiopulmonary resuscitation programs to improve survival outcomes after out-of-hospital cardiac arrest in a metropolis: A before and after study. Resuscitation. 2018;130:124-32.

Pearson DA, Darrell Nelson R, Monk L, Tyson C, Jollis JG, Granger CB, et al. Comparison of team-focused CPR vs standard CPR in resuscitation from out-of-hospital cardiac arrest: Results from a statewide quality improvement initiative. Resuscitation. 2016;105:165-72.

Spitzer CR, Evans K, Buehler J, Ali NA, Besecker BY. Code blue pit crew model: A novel approach to in-hospital cardiac arrest resuscitation. Resuscitation. 2019;143:158-64.

Sporer K, Jacobs M, Derevin L, Duval S, Pointer J. Continuous Quality Improvement Efforts Increase Survival with Favorable Neurologic Outcome after Out-of-hospital Cardiac Arrest. Prehospital emergency care : official journal of the National Association of EMS Physicians and the National Association of State EMS Directors. 2017;21(1):1-6.

Stub D, Schmicker RH, Anderson ML, Callaway CW, Daya MR, Sayre MR, et al. Association between hospital post-resuscitative performance and clinical outcomes after out-of-hospital cardiac arrest. Resuscitation. 2015;92:45-52.

van Diepen S, Girotra S, Abella BS, Becker LB, Bobrow BJ, Chan PS, et al. Multistate 5-Year Initiative to Improve Care for Out-of-Hospital Cardiac Arrest: Primary Results From the HeartRescue Project. Journal of the American Heart Association. 2017;6(9):e005716.

Weston BW, Jasti J, Lerner EB, Szabo A, Aufderheide TP, Colella MR. Does an individualized feedback mechanism improve quality of out-of-hospital CPR? Resuscitation. 2017;113:96-100.

Wolfe H, Zebuhr C, Topjian AA, Nishisaki A, Niles DE, Meaney PA, et al. Interdisciplinary ICU cardiac arrest debriefing improves survival outcomes*. Critical care medicine. 2014;42(7):1688-95.