**2. ECMO outcomes prediction scores to date**

The need for predictive scoring algorithms is particularly vital with deployment of a high resource therapy like extracorporeal life support. Over the last several decades various groups have published predictive scores typically focusing on mortality, with the hope to help guide clinical decision making for optimal patient selection prior to cannulation, and often to help stratify patient risk. The overall goal being to help identify patients most likely to benefit, and these scores have ranged in accuracy (measured by the Area Under the Receiver Operative Curve) from 0.65 to 0.89 [7].

To date, these scores have been focused on the broad pathophysiology of the disease necessitating ECMO support; respiratory failure requiring veno-venous (VV) ECMO and cardiac failure necessitating support by veno-arterial (VA) ECMO.

#### **2.1 Respiratory failure**

#### *2.1.1 Neonatal - congenital diaphragmatic hernia*

Congenital diaphragmatic hernia (CDH) continues to be one of the most common reasons for ECMO use in the neonatal age group [8]. This unique pathology is characterized by a failure of diaphragmatic development, associated lung hypoplasia and subsequent persistent pulmonary hypertension of the newborn (PPHN) and as such typically presents with isolated refractory respiratory or combined respiratory and cardiac failure shortly after birth [9].

As worsening hypoxia and hypercapnia after birth often exacerbate pulmonary hypertension in turn leading to worsening gas exchange and hemodynamics, the potential need exists for ECMO as a rescue therapy for those with severe CDH [9]. Prediction of CDH severity is often identified prenatally and is based off genetic testing, the sidedness of the CDH, liver position (proportion in the thorax), and observed to expected fetal lung volume by magnetic resonance imaging (MRI) [10]. Despite decades-long use of ECMO for CDH, there continue to ongoing controversies surrounding ECMO use in this unique population. Outcomes remain variable, controversy exists regarding if the optimal support modality is VV or VA ECMO and the timing of ECMO initiation, as well as if surgical repair is best performed early or late and finally if surgical repair should be performed while on ECMO support due to the increased possibility of associated bleeding complications [11–14]. The CDH working groups for both the Extracorporeal Life Support Organization (ELSO) and its European chapter (EuroELSO) have published entry criteria for utilizing severity of hypoxemia, impaired ventilation and impaired tissue perfusion prior to ECMO as indications and prematurity, weight, comorbidities and duration of mechanical ventilation as relative contraindications [15–17].

Currently two CDH ECMO specific mortality risk prediction models exist utilizing data from the ELSO registry data from 2000 to 2015, including over 4000 ECMO supported neonates [18]. Using multivariable logistic regression analyses using both complete data sets and 10 imputed data sets, the authors developed two predictive models, a pre-ECMO mortality risk score and another on-ECMO mortality risk score.

#### *ECMO Predictive Scores, Past, Present, and Future DOI: http://dx.doi.org/10.5772/intechopen.106191*

By dividing the studied cohort into a two thirds development cohort and a one third validation cohort, the authors included pre-ECMO ventilatory settings and adequacy of gas exchange as measured by blood pH in addition to CDH specific risk factors including CDH sidedness and repair on ECMO. The developed pre-ECMO model performed modestly with C statistics of 0.65. With the addition of on-ECMO complications; neurologic, renal and infectious, the on-ECMO Prediction model performance improved to C = 0.73 (**Table 1**). The authors recognized the value of these models specifically in research and quality improvement projects with cautious application in patient management.

#### *2.1.2 Neonatal*

Two commonly utilized scores exist to predict ECMO mortality risk in the general neonatal population beyond CDH.

The Pittsburgh Index for Pre-ECMO Risk (PIPER) was developed utilizing 5455 neonatal respiratory VA-ECMO patients from the ELSO registry from 2000 to 2010 to predict survival to hospital discharge [19]. PIPER was developed on seven pre-ECMO variables including patient characteristics; age, weight and the diagnosis of CDH in addition to markers of hemodynamic compromise and severity of respiratory failure prior to ECMO initiation; mean arterial blood pressure (MAP), pH, partial pressure of oxygen (pO2) and use of inhaled nitric oxide (iNO). The authors found that each increasing quartile had a 15% increased risk of mortality. Despite the focus on pre-ECMO risk, the authors also conducted further modeling to include on-ECMO variables such as ECMO duration and complications (hemorrhagic, mechanical, neurologic, pulmonary and renal), which increased the predictive power of the PIPER model from an area under the receiver operator curve (AUROC) of 0.74 to 0.79. While not developed to focus only on CDH patients, CDH was much more common in the highest PIPER quartiles. Older age at ECMO initiation was also found to be associated with decreased survival [19].

The Neonatal Risk Estimate Score for Children Using Extracorporeal Respiratory Support (Neo-RESCUERs) was similarly designed to predict mortality for neonates receiving ECMO respiratory support [20]. It was developed and validated on 4592 neonates in the ELSO registry between 2008 and 2013, with January 1st 2012 as the cutoff date between the initial derivation cohort and the subsequent internal validation dataset. Validation was performed on patients with complete data in addition to those with imputed data. The investigators included patient demographics (age, gestational age, birth weight, sex, diagnosis), markers of respiratory and cardiac failure (pH, partial pressure of carbon dioxide (PaCO2), ratio of arterial partial pressure of oxygen to fraction of inspired oxygen (PF ratio), oxygenation index (OI), and history of cardiac arrest) in addition to renal failure and comorbid conditions prior to ECMO cannulation (**Table 1**). They demonstrated their lowest decile having a predicted mortality of 4.4% compared to an observed mortality of 7%, and their highest decile having a predicted mortality of 67.5% and observed mortality of 65.6%. CDH in this group had 11-fold higher adjusted odds for mortality compared to meconium aspiration, pre-ECMO renal failure also had a much higher odds of mortality. As is the case with similar registry-based scores, the authors acknowledge the significant limitations with reliance on only the available variables in addition to the retrospective nature of the study and the inclusion of only ECMO supported patients. As such, they recommend the use of Neo-RESCUERS as a benchmarking and risk stratification tool rather than an ECMO decision support tool.



*ECMO Predictive Scores, Past, Present, and Future DOI: http://dx.doi.org/10.5772/intechopen.106191*

*ECMO = extracorporeal membrane oxygenation, CDH = congenital diaphragmatic hernia, MAP = mean arterial pressure, pO2 = partial pressure of oxygen, iNO = inhaled nitric oxide, VA = veno-arterial, ELSO = Extracorporeal Life Support Organization, CPR = cardiopulmonary resuscitation, PHIS = Pediatric Health Information System, MV = mechanical ventilation, VV = veno-venous, HFOV = high frequency oscillatory ventilation, CNS = central nervous system, iNO = inhaled nitric oxide, BMI = body mass index, bicarb = bicarbonate, SOFA = Sequential Organ Failure Assessment, ICU = intensive care unit, ARDS = acute respiratory distress syndrome, PTSD = post-traumatic stress disorder, HRQL = Health Related Quality of Life.*

#### **Table 1.**

*ECMO mortality prediction scores for respiratory failure.*

#### *2.1.3 Pediatric*

Two common scores exist to predict mortality in pediatric respiratory failure. The Pediatric Risk Estimation Score for Children Using Extracorporeal Respiratory Support (Ped-RESCUERS) and the newer Pediatric Pulmonary Rescue with Extracorporeal Membrane Oxygenation Prediction (P-PREP) score.

Ped-RESCUERS was developed on patients aged 29 days to 18 years, utilizing the ELSO registry from 2009 to 2014, with the 2013 to 2014 cohort as the validation data set to predict survival to hospital discharge [21]. The model was developed and validated on 2458 pediatric patients undergoing ECMO for respiratory support, with an overall observed mortality rate was 39.8%. In addition to variables of severity of respiratory failure up to 6 hours prior to ECMO initiation (pH and PaCO2), the authors also included the duration of mechanical ventilation and type of mechanical ventilatory support prior to cannulation and specific diagnoses such as pertussis, bronchiolitis and malignancy in the model development (**Table 1**). The model had modest performance with an AUROC of 0.69 in the development cohort and 0.63 in the validation data set. Similar to clinical experience, they found those with bronchiolitis and asthma had relatively better outcomes than those with cancer or pertussis, and those requiring ECMO later in their course had increased mortality. Interestingly higher pre-ECMO PaCO2 was associated with less mortality, the authors speculate this may be due to the association with asthma and bronchiolitis and their subsequent survival benefit [21]. Similar to Neo-RESCUERS, the authors acknowledge the inability to use Ped-RESCUERS as a decision support tool on an individual patient level as the model was only developed on patients already supported on ECMO, but rather as a risk adjustment tool to help facilitate interinstitutional comparisons.

The P-PREP Score was developed and internally validated on 4352 children more than 7 days old to less than 18 years old requiring ECMO for respiratory failure in the ELSO registry between 2008 and 2013 and then externally validated in 2007 patients from the Pediatric Health Information System (PHIS) dataset [22]. Predictive variables included mode of ECMO support, and pre-ECMO variables such as length of mechanical ventilation, severity of hypoxia and diagnosis categories and comorbidities (**Table 1**). As the relative timing of comorbidities to ECMO initiation is not recorded in the ELSO registry, all comorbidities were assumed to be present at the time of ECMO cannulation. Of interest, the year of ECMO support was included in the final P-PREP model given the lower mortality rates in the 2009–2013 era compared to 2001–2009, although not assigned a score in the model calculation. Mode of ECMO support and severity of hypoxia were excluded from the external validation model as these variables are not collected in the PHIS database. The model had modest performance in the development, internal and external validation data sets (AUROC of 0.69, 0.66 and 0.69 respectively). Mortality was significantly higher in patients with two or more extrapulmonary organ system failures. Similar to previous models, P-PREP was limited to patients already supported on ECMO and as such could not be used as a ECMO decision making risk tool but the authors suggest it may aid in prognostication and family counseling for patients supported on ECMO for respiratory failure [22].

### *2.1.4 Adult*

Since the 2009 H1N1 influenza A pandemic, there has been an exponential increase in ECMO use for adult respiratory failure [23] and as such several risk prediction scores have been developed for this population.

Perhaps the earliest widely used score was the ECMOnet score – developed on 60 patients from a multicenter 2009 H1N1 influenza A pandemic cohort [24]. The authors' goal was to add consideration to extrapulmonary organ function, and not just the severity of respiratory failure in risk stratification for VV ECMO use in

respiratory failure, aiming to aid in resource allocation and timing of ECMO initiation. Pre-ECMO predictors associated with mortality included hospital length of stay, bilirubin, creatinine, hematocrit and mean arterial pressure with a good model performance with an AUROC of 0.85 (**Table 1**). The results were then validated on an external dataset of 74 patients with acute respiratory distress syndrome (ARDS) supported on ECMO with an AUROC of 0.69. This tool provided early evidence of the importance of the consideration of extrapulmonary organ function at time of ECMO initiation.

The Predicting dEath for Severe ARDS on VV-ECMO (PRESERVE) score was developed on data from 140 patients with refractory ARDS in three French intensive care units from 2008 to 2012 to identify factors associated with death by 6 months post-ICU discharge [25]. A large portion of patients (26%) in this group also had H1N1 influenza. Eight pre-ECMO predictors were identified including age, body mass index, immunocompromised status, prone positioning, sepsis-related organ failure, days of pre-ECMO mechanical ventilation, plateau and positive end expiratory pressures (PEEP) (**Table 1**). Interestingly PF ratio was not found to be associated with mortality, but during enrolment of the development cohort increasing evidence had been mounting on the benefit of prone positioning. When the authors forced it into the model, PF ratio was found to be associated with lower mortality. The PRESERVE model performed well with an AUROC of 0.89 at predicting all cause 6-months post-ICU discharge mortality. The goal of the PRESERVE score was to help clinicians select appropriate candidates, and uniquely they provided details on health-related quality of life (HRQL) evaluation at 6 months, finding high levels of persistent physical and emotional difficulties, including anxiety, depression and post-traumatic stress disorder (PTSD) [24].

The Respiratory Extracorporeal Membrane Oxygenation Survival Prediction (RESP) score is one of the most commonly clinically used and cited scores, developed from the ELSO registry data from 2000 to 2012 to predict survival to hospital discharge [26]. The authors also externally validated the score on the 140 patients used for the development of the PRESERVE score, using commonly available features in both datasets. Diagnostic groups, non-pulmonary infections and central nervous system dysfunction were significant predictors, again highlighting the importance of extrapulmonary organ function prior to ECMO initiation. The RESP score demonstrated the importance of ECMO specific mortality prediction scores as it performed better than simple severity of illness scores such as the simplified acute physiology score (SAPS II) and sequential organ failure assessment (SOFA). The RESP score includes five risk classes, with the lowest and highest classes having 8% and 82% mortality respectively. An important limitation of the score development was inability to incorporate prone positioning, as it was not reported in the ELSO registry data. As with the PRESERVE score, duration of mechanical ventilation above 7 days prior to ECMO initiation was found to be significantly associated with worse outcomes. The authors acknowledge the score limitation in only being developed on patients already supported on ECMO in addition to the lack of detailed biologic data in the ELSO registry beyond the immediate pre-ECMO blood gas values.

Since its development, the RESP score has been further externally validated in several independent studies [27, 28] and studies utilizing external databases comparing both the RESP and PRESERVE score have demonstrated similar accuracy [29, 30]. Though several of their variables overlap, which may explain their similar accuracy (**Table 1**). Interestingly, more recent data has shown that the RESP Score performed poorly at predicting mortality during the Coronavirus disease 2019 (COVID-19)

pandemic – with patients in lower risk classes paradoxically having worse survival than those in higher risk classes [31].

The Roch Score was development to identify factors associated with in-hospital mortality in 85 ARDS patients treated with ECMO following referral to an ECMO center between 2009 and 2013 [32]. All patients were cannulated by a mobile ECMO unit prior to transfer to the referral center. ECMO decision making followed criteria with exclusion of patients with prolonged mechanical ventilation or ARDS over 7 days, age above 70 years and those with SOFA scores above 18. The uniqueness of this score's development focusing on referral to an ECMO center, may support its utility in decision to transfer to ECMO centers. It included only limited variables (**Table 1**) including age, SOFA score and influenza diagnosis, making it suitable for timely decisions regarding transfer. The authors identified patients that were under 45 years of age and had a diagnosis of influenza had markedly better prognosis, independent of other organ dysfunctions [32].

Another score is the VV-ECMO mortality score [33], which was developed on a 116 adult patients single center cohort between 2007 and 2015. The authors included only patients with a PF ratio < 70, and independent predictors of inhospital mortality included pre-ECMO variables such as SOFA score, length of mechanical ventilation and immunocompromised status, which were translated into a simple 3 binary variable predictor. The lowest score had a mortality of 18%, while the highest had a mortality of 88%, and again emphasized the importance of extra-pulmonary organ dysfunction and severity of illness. The authors' goal was to develop a simple risk stratification tool to identify patients with the highest of a poor outcome with VV ECMO, for prognostication or consideration of alternative support modalities if possible.

The PREdiction of Survival on ECMO Therapy Score (PRESET) was derived on 108 ARDS patients between 2010 and 2015 [34]. In this work the authors analyzed the performance of 4 previous mortality risk scores; ECMOnet, RESP, PRESERVE and Roch with only the RESP and ECMOnet score demonstrating good accuracy. There was some evidence that the ECMOnet score was most appropriate only in cohorts of H1N1 patients. The authors then developed a new score, PRESET. The importance of extrapulmonary predictors such as mean arterial blood pressure, lactate, pH, platelets and pre-ECMO hospital length of stay were found. The lowest risk class had a 26% mortality rate, while the highest had a 93% mortality rate. Platelet count prior to ECMO initiation was found to be an important predictor, with a decrease of 100,000/ ul associated with 30% increased mortality. These findings were similar to those reported in other critically ill patients [35]. A decrease in pH by 0.1 was also found to increase mortality by 40%. Recent studies have also demonstrated the accuracy of the PRESET score in COVID-19 patients [36].

Some studies have aimed at externally validating several of these scores with varying results, with slightly lower accuracy than initially published in general and with some studies showing superior performance of general severity of illness scores at predicting in-hospital mortality [37–39].

#### **2.2 Cardiac failure**

#### *2.2.1 Pediatric*

The Pediatric Extracorporeal Membrane Oxygenation Prediction (PEP) model was developed for risk stratifying mortality of all pediatric and neonatal patients

requiring ECMO regardless of indication between 2012 and 2014 [40]. The authors included on data from 514 patients from the bleeding and thrombosis on extracorporeal membrane oxygenation (BATE) dataset to predict in-hospital mortality. Variables include indication for ECMO, age, CDH diagnosis, and laboratory markers in the 12 hours prior to ECMO initiation, utilizing the data points most proximal and prior to ECMO for analysis (**Table 2**). As with prior scores developed only on ECMO



*ECMO = extracorporeal membrane oxygenation, ELSO = Extracorporeal Life Support Organization, VA = venoarterial, DBP = diastolic blood pressure, PP = pulse pressure, bicarb = bicarbonate, AMI = acute myocardial infarction, CDH = congenital diaphragmatic hernia, MAS = meconium aspiration syndrome, PTT = partial thromboplastin time, INR = international normalization ratio, STAT = The Society of Thoracic Surgeons-European Association for Cardio-Thoracic Surgery, CPB = cardiopulmonary bypass, PP = pulse pressure, Bicarb = bicarbonate, BMI = body mass index, GCS = Glasgow coma scale.*

#### **Table 2.**

*ECMO mortality prediction scores for cardiac failure.*

patients' data, the authors acknowledged the inability to generalize the score for individual patient prediction or selection for ECMO support. The same group then performed an external validation of the PEP score on 4342 patients from the ELSO registry between 2012 and 2014 [41]. They found that the score AUROC decreased from 0.75 on the BATE data to 0.64 on the ELSO data with the most significant decrease in the highest risk deciles.

The Pediatric Survival after Veno-arterial ECMO (Pedi-SAVE) score was recently developed utilizing pediatric cardiac VA-ECMO patients from 2001 to 2015 from the ELSO registry from birth to 18 years of age [42]. The study included 10,091 patients, and both pre- and post-cannulation models were developed. Lowest risk patients in the pre- and post-cannulation groups had a 65% and 74% chance of survival respectively, compared to the highest risk groups having 33% and 22% comparatively. Pre-cannulation factors included type of congenital heart disease with better outcomes in patients with non-single ventricle physiology, age, pH, requirement for acid-buffer administration, number of cardiac procedures, the Society of Thoracic Surgeons-European Association for Cardio-Thoracic Surgery score (STAT), combined cardiopulmonary failure and failure to separate from cardiopulmonary bypass prior to ECMO initiation (**Table 2**). The post-cannulation model included pump flow rates in the first 24 hours on ECMO and on-ECMO complications [42]. The authors concluded that the developed scores could serve in risk adjustment, comparing outcomes between centers and across eras. They also acknowledge the limitations of a registry data-based predictive score with lack of granular patient physiological data, inability to control for data completeness and most importantly the inability to generalize the score to direct individual patient care or guide whether to provide or withdraw ECMO support on the individual patient level.

#### *2.2.2 Adult*

Several adult prognostic scores exist for ECMO use in the setting of cardiac failure exist. The survival after veno-arterial ECMO (SAVE) score sought to identify pre-ECMO factors which predict survival to hospital discharge, it utilized 3846 patients from the extracorporeal life support organization registry between 2003 and 2013 [43]. The authors performed further validation on an external dataset of 161 patients

#### *ECMO Predictive Scores, Past, Present, and Future DOI: http://dx.doi.org/10.5772/intechopen.106191*

who underwent VA ECMO support at a single institution. The developed score included 12 variables including underlying diagnosis leading to cardiogenic shock, age, weight, pre-ECMO organ failure, duration of mechanical ventilation prior to ECMO, pre-ECMO cardiac arrest and hemodynamic profile and degree of metabolic acidosis (**Table 2**). The score gave 5 risk classes with survival percentage ranging from 18 to 75%. On internal validation, the score had an AUROC of 0.68 that increased to 0.9 on external validation. The high accuracy may have been due to the validation set being a participant in the original registry, and it being a high volume highly regimented center. Their findings highlighted the importance of timing for ECMO initiation, with too early use exposing patients to unnecessary complications and too late use proving futile. The SAVE score also outperformed the SOFA score at both cannulation and ICU admission and other ICU severity of illness scores at discriminating patients who did not survive to hospital discharge, once again highlighting the importance of ECMO specific scoring methods [43].

Two other common adult cardiac prognostic scores were developed with more limited populations in mind. The prEdictioN of Cardiogenic shock OUtcome foR AMI (Acute Myocardial Infarction) patients salvaGed by VA-ECMO (ENCOURAGE) score was developed to predict in-ICU mortality from data on 138 patients from 2 ICUs between 2008 and 2013 [44]. Similarly, to the PRESERVE score the authors conducted quality of life assessments on ICU survivors, and found high rates of anxiety, depression and PTSD. Seven pre-ECMO predictive variables were identified (**Table 2**) including age, Glasgow coma score, creatinine, lactate, prothrombin activity, body mass index and sex. The lowest risk group had an 80% survival compared to the highest score group having only a 7% survival. In this group of patients with acute myocardial infraction, the ENCOURAGE score had an AUROC of 0.84 outperforming the SAVE score, and myocardial infraction scores such as the GRACE model [45], Dutch University Hospital Model [46] or the SHOCK Trial and Registry Scoring system [47]. Limitations included that development heavily relied on data from 2 highly specialized and experienced centers, limiting its generalizability, in addition some of the VA ECMO patients having low markers of impaired end organ perfusion (lactate and pH), suggesting either early ECMO initiation or less severe disease at initiation of ECMO.

The PREDICT VA-ECMO score was developed on a single center derivation cohort of 205 all-comers who received VA ECMO from 2010 to 2015, and validated on a cohort from an independent center from 2010 to 2017 [48]. In this work the authors set out to develop a dynamic model to predict hospital survival using on-ECMO variables at multiple time points. Two models were thus developed; the 6-hour PREDICT-VA-ECMO score utilizing variables at 6-hours post ECMO cannulation and the 12-hour PREDICT-VA-ECMO incorporating variables at 1, 6 and 12 hours post cannulation. The PREDICT VA-ECMO score out performed common severity of illness scores, as well as the SAVE score, but with higher accuracy with the 12-hour model compared to the 6-hour model (AUROC of 0.823 and 0.839 respectively), with comparable performance in the external validation cohort. Variables included lactate, pH and bicarbonate (**Table 2**). It demonstrated good prediction accuracy utilizing only a few variables which are easily obtained, and importantly was able to include evaluation of extracorporeal cardiopulmonary resuscitation (eCPR) patients.

Other published scores include a simple scoring system developed on a retrospective cohort of 100 patients between 2010 and 2017 at three institutions, to predict survival to discharge. Five variables were ultimately included, lactate >10 mmol/L, albumin <3 gm/dL and platelets <180,000/uL as well as age and duration of

pre-ECMO mechanical ventilation. The lowest score predicted a mortality of 10%, while the highest saw 100% expected mortality [49].

Some studies have also indicated that lactate and urine output are independent predictors of mortality in extracorporeal life support patients, and that the cardiacsurgery score (CASUS) has moderate accuracy compared to simpler severity of illness scores such as SOFA [50].

#### **2.3 Limitations of current approaches**

Several limitations exist when evaluating the current landscape of ECMO outcome prediction tools. Although there is good evidence that these scores perform better than standard severity of illness scores, external validation has often shown decreased accuracy. Concerningly many of these scores performed poorly during the novel COVID-19 pandemic, at a time when patient selection and resource allocation was integral [31, 36, 51].

The reasons for poor external validation are likely multifold, while scores developed on large international multicenter registry data should have excellent external validity, this ignores that these databases often have very poor granularity. They fail to account for large center variations which may account for mortality differences [52, 53], including variations in anticoagulation strategies [54], ventilator strategies, timing of ECMO initiation and expertise that may exist in high volume centers [52, 53, 55]. This is especially true when the underlying population receiving ECMO is evolving, as during the COVID-19 pandemic. Static scores lend themselves poorly to changing dynamics in population, as ECMO use has continued to expand since many of these scores have been developed. Furthermore, scores developed utilizing large international registry data and at high volume ECMO centers may translate poorly to centers outside the registry or lower volume or lower resource centers. Particularly concerning is the fact that many of the pediatric and neonatal scores lack any significant external validation, limiting confidence in their deployment.

Another explanation for many of the current generation of ECMO mortality scores' modest performance may be the tendency to rely on accuracy as a measure of performance. Accuracy while made up of sensitivity and specificity, may be less reliable if an outcome of interest is less common. Furthermore, clinicians at the bedside frequently care about positive and negative predictive values, but these are based off prevalence. If centers have large variations in the prevalence of mortality, scores may experience wide variance in their ability to predict the outcome of interest.

Using data from both an institutional database of 15 hospitals in a quaternary referral center and a multinational dataset spanning 42 countries from the COVID-19 pandemic, we evaluated the performance of several ECMO mortality prediction scores; RESP, PRESERVE, Roch and PRESET [56]. In order to more comprehensively evaluate the scores' performance, we reported both accuracy with AUROC and precision with area under the precision recall curve (AUPRC) (**Table 3**). Our results were consistent with previous reports during the pandemic with modest performance of all the studied scores, further emphasizing their limited clinical applicability, especially in the setting of global healthcare system resource limitation.

The current generation of ECMO mortality scores also fails to provide guidance to clinicians in what may be the most vital decision – timing of ECMO initiation. The lack of data regarding the patients' pre-ECMO evolving trajectory and reliance on static variables often immediately prior to ECMO cannulation, in addition to the lack


#### **Table 3.**

*Performance of commonly available ECMO mortality scores in the COVID-19 pandemic.*

of a true matched non-ECMO control cohort, significantly limit the utility of all the current scores as clinically applicable decision support tools.

Finally, only two of the scores (ENCOURAGE and PRESERVE) included any data on outcomes aside from mortality, providing information related to quality of life. As mortality improves it likely will become increasingly relevant to begin predicting on meaningful clinical outcomes like neurological function and quality of life.

### **3. Future directions**

The continuously expanding use of ECMO internationally has been coupled by an evolution in biomedical informatics, artificial intelligence and machine learning methodology. This evolution has opened the possibility of exploring more complex machine learning methodologies to develop predictive models to better mirror the clinical decision-making dilemma.

The COVID-19 pandemic has the unmasked the collaborative potential across institutions and even nations to share healthcare data in real time. This opens the possibility to develop more granular multi-institutional databases that expand the currently available registry data. Such databases would open the potential to not only incorporate variables well in advance of ECMO cannulation to better correlate the outcomes with patient trajectories, but also allow the comparison with a propensity matched cohort of non-ECMO critically ill patients to mirror the influence of decision and timing of provision of ECMO support.

The advances in ECMO technology, expanding candidacy and improved outcomes, demonstrate that the use of mortality as an objective outcome has become insufficient. The current landscape of ECMO use mandates a transition from survival to functional neurological outcomes as the goal of predictive modeling. Such an approach requires agreement in the ECMO community on clearly defined definitions for goal neurological outcomes, a task well overdue. Such objective outcomes could then be used as goals in predictive model development to better understand the influence of timing and provision of such high-risk resource-intensive therapy on both individual and system levels.
