EXTERNAL VALIDATION OF A NOVEL DIGITAL SIGNATURE IN CONTINUOUS CARDIORESPIRATORY MONITORING TO DETECT EARLY RESPIRATORY DETERIORATION OF ICU PATIENTS ===================================================================================================================================================== * Rachael A. Callcut * Yuan Xu * Christina Tsai * Andrea Villaroman * Anamaria J. Robles * Douglas E Lake * J Randall Moorman * Xiao Hu * Matthew T Clark ## ABSTRACT The goal of predictive analytics monitoring is the early detection of patients at high risk of subacute potentially catastrophic illnesses. A good example of a target illness is respiratory failure leading to urgent unplanned intubation, where early detection might lead to interventions that improve patient outcome. Previously, we identified signatures of this illness in the continuous cardiorespiratory monitoring data of Intensive Care Unit patients and devised algorithms to identify patients at rising risk. Here, we externally validated 3 logistic regression models to estimate risk of emergency intubation that were developed in Medical and Surgical ICUs at the University of Virginia. We calculated the model outputs for more than 8000 patients in University of California – San Francisco ICUs, 240 of whom underwent emergency intubation as determined by individual chart review. We found that the AUC of the models exceeded 0.75 in this external population, and that the risk rose appreciably over the 12 hours prior to the event. We conclude that abnormal signatures of respiratory failure in the continuous cardiorespiratory monitoring are a generalizable phenomenon. ## INTRODUCTION Patients in intensive care unit (ICU) that experience respiratory failure leading to emergent intubation have significantly longer hospital length-of-stay and higher in-hospital mortality1-3. Earlier identification of patients that will require intubation may allow earlier intervention to reduce morbidity and mortality. Patients in early stages of respiratory decompensation may be administered corticosteroids or bronchodilators, given supplemental oxygen, or placed on non-invasive positive-pressure ventilation4. Patients for whom non-invasive positive-pressure ventilation is insufficient to prevent respiratory failure, or recently extubated patients that require re-intubation, may be intubated electively rather than emergently5, 6. Timely intervention aimed at better preparation for intubation, team coordination, proper intubation medication selection, and avoidance of peri-intubation hypotension have been shown to improve outcome of patients requiring emergent intubation. Predictive analytics monitoring strives to identify high risk patients earlier by providing continuous risk estimates to clinical personnel in real-time. These early warning signals have potential to allow intervention that may alter the patient trajectory in a more favorable direction. In a prospective study using analogous predictive analytics for sepsis that were available at the bedside of preterm infants, mortality was reduced by more than 20%. Similarly, in adults, predictive analytics for sepsis may reduce rates of septic shock and associated mortality by up to 50%7. There are candidate risk marker models for emergent intubation developed at single centers and based on static demographics, comorbidities, injury type, and intermittent vital signs3, 8. Models based on continuous cardiorespiratory dynamics from bedside physiological monitors also exist but are not yet validated in an external cohort of all comer ICU patients1,2,9. Before using a predictive analytical model for prospective clinical practice, it is important to validate that model across differing patient populations and institutions. It is necessary to know if a model trained at one institution generalizes to provide early detection of patient deterioration at another institution. Different care units and institutions may have substantially different distributions of demographics, socio-economic groups, admitting practices, and care patterns, all of which may degrade the calibration and discriminatory performance of a model. This study tests the hypothesis that predictive dynamic analytical models for emergent intubation using continuously available data from ubiquitous physiologic monitors are well-suited for application at an external center. ## METHODS ### Study Design This is a retrospective cohort study of patients admitted to an ICU at the University of California San Francisco Medical Center (UCSF). We studied admissions to the two mixed medical/surgical ICUs, two neurological ICUs (NICUs), and coronary care (CCU) ICUs. Each ICU has continuous physiological monitoring archived by BedMaster (Excel Medical, Jupiter FL). The primary outcome was respiratory decompensation resulting in emergent intubation. ### Study population and primary outcome We studied consecutive ICU admissions from May 1, 2013 through April 30, 2015 and selected patients in which intubation occurred in the ICU. Intubation events prior to ICU admission (*e*.*g*., in the operating room or emergency department) were excluded. Patients with “Do Not Resuscitate” or “Do Not Intubate” orders (DNR/DNI) were excluded. Each intubation event was classified as planned (elective) or unplanned (emergent). Planned intubations included those done for procedures (such as endoscopy, interventional radiology procedures, or preceding elective operations) and those that were documented as elective in the medical record. All other intubations occurring in the ICU were considered unplanned. The procedure note and physician notes were examined to determine the reason for intubation. Timing of intubation was extracted from the medical record nursing/respiratory therapist documentation. The times were checked for accuracy by two independent practitioners reviewing each potential case of emergent intubation ### Identification of mechanically ventilated patients When evaluating the performance of a model for emergent intubation it is important to exclude patients at times when they are not at risk. Patients are not at risk for emergent intubation at times when they are already mechanically ventilated. In order to exclude data during these times, it is necessary to know the time of all intubations and extubations (not only the emergent intubations). We used the flowsheet entries from respiratory therapists to identify periods of mechanical ventilation and merged the results with known times of emergent intubations. We extracted the total ventilator respiratory rate flowsheet entry documented by RTs. Figure shows the probability density of time between RT documentation: most RT documentation of total ventilator respiratory rate occurs more frequently than q6-hours. Mechanical ventilation was defined as starting at the time of the first ventilator respiratory rate and ending at the time of the last ventilator respiratory rate. We split the period of mechanical ventilation when the time between consecutive measurements from a patient was larger than 16 hours, and identified the patient as not ventilated in the interim. Isolated measurements (*i*.*e*., those without another measurement within 16 hours) were used to define the start of a ventilation epoch with duration of 1 hour. For patients that were emergently intubated (*i*.*e*., for whom we know time of intubation based on individual chart review) we ensured that the epochs of mechanical ventilation started at the appropriate time. We split the ventilation epochs when an emergent intubation occurred in the middle, defining the time of extubation as the time of the preceding ventilator respiratory rate and the time of intubation as the time of emergent intubation. ### Predictors of emergent intubation We calculated 3 risk estimates for emergent intubation1,2. The models were developed on data from SICU and MICU patients at University of Virginia (UVa) health system, and use only continuous cardiorespiratory dynamics calculated from the bedside physiological monitors. From Politano *et al*.*2* we used the vital signs only model. Note that this model was developed on a subset of surgical ICU patients, excluding those on ancillary services. We also used the SICU and MICU models of Moss *et al*.*1*. The model of Politano was previously validated in the UVa for predicting upgrade from surgical intermediate care unit (IMU) to ICU, with and without intubation10. The SICU model of Moss was also validated at the same institution as part of a model for identifying low-risk patients at the time of surgical IMU and ICU discharge11. Model inputs are were calculated in 30 minute windows with 50% overlap: means and standard deviations of, and cross-correlations between, vital signs (heart rate, respiratory rate, peripheral oxygen saturation, and blood pressure), as well as statistical measures of cardiac dynamics (slope of log RR interval variance versus log scale for detrended fluctuation analysis, coefficient of sample entropy, and the standard deviation of RR intervals)12-15. Missing data were imputed with the median values from the UVa development cohort. Features and models were calculated using CoMET® (AMP3D, Charlottesville, VA). No features or models were available to the care team and all patients received standard of care. This study was approved by the UCSF institutional review board under a waiver of consent. ### Statistical analysis We evaluated the performance of the UVa urgent intubation risk models for identifying UCSF patients prior to emergent intubation. We used the continuous risk estimate and a binary response variable. The response was “1” for patients at times prior to emergent intubation (the event population), and “0” for those patients far from intubation and all other patients at all other times. We evaluated the performance of risk estimates for 4 to 24-hour windows prior to emergent intubation. Confidence intervals were determined by 200 bootstrap runs, resampled by hospital admission. ## RESULTS We studied 9,828 admissions for 8,434 patients to UCSF. There were 240 episodes of emergent intubation in 238 hospital admissions. There was continuous monitoring data before 225 (93%) of events. Table 1 shows characteristics of the study population. Patients with emergent intubation were less likely to be white, more likely to be Asian, had 3-fold higher mortality and stayed 19 days longer in hospital. We calculated 105.5 patient-years of risk estimates: the 3 risk estimates were calculated for 3.7 million 15-minute epochs. We censored 32.3% of measurements that were for patient that were already intubated or for patients following DNI orders. The incidence of emergent intubation was 0.9 per 100 ICU days, significantly lower than in the Moss1 and Politano2 development cohorts (2.1 and 2.8 per 100 ICU days respectively). View this table: [Table 1:](http://medrxiv.org/content/early/2021/05/26/2021.05.24.21257711/T1) Table 1: Characteristics of the study population Figure 2 (left) shows the number of events with continuous monitoring data leading up to the time of emergent intubation. Figure 2 (right) shows the time course of risk estimates for UCSF patients using the three UVa models for urgent unplanned intubation during the 48 hours preceding emergent intubation. Risk estimates doubled over the 12 to 24 hours prior to emergent intubation, from about 1.5 to more than 3. At each time we performed a signed rank test with the null hypothesis that risk estimates are equal to risk estimates from the same patient 12 hours prior, and a two-sided alternative hypothesis. White points identify times when we rejected the null hypothesis. Risk estimates were significantly higher beginning about 5 hours prior to emergent intubation. ![Figure 1:](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2021/05/26/2021.05.24.21257711/F1.medium.gif) [Figure 1:](http://medrxiv.org/content/early/2021/05/26/2021.05.24.21257711/F1) Figure 1: Probability density of the time between consecutive ventilator respiratory rate entries for all patients. ![Figure 2:](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2021/05/26/2021.05.24.21257711/F2.medium.gif) [Figure 2:](http://medrxiv.org/content/early/2021/05/26/2021.05.24.21257711/F2) Figure 2: (left) Number of events with continuous monitoring data as a function of time leading up to emergent intubation. (right) Average time course of risk estimates over the 48 hours leading up to the time of emergent intubation. White points indicate that the risk estimates at that time are significantly higher (p < 0.05) than risk estimates 12 hours prior. Figure 3 shows the performance of the 3 models for emergent intubation as a function of the window before intubation that is considered the event. For a 24-hour detection window, for example, all data within 24 hours prior to emergent intubation were identified as ‘event’ and all other data (from event patients far from the event, and from all non-event patients) was identified as control. The AUC rose from about 0.75 to about 0.78 for event windows 24 to 4 hours, respectively. The MICU model from Moss1 had slightly better performance than the SICU models from Moss2 and Politano2. ![Figure 3:](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2021/05/26/2021.05.24.21257711/F3.medium.gif) [Figure 3:](http://medrxiv.org/content/early/2021/05/26/2021.05.24.21257711/F3) Figure 3: Area under the receiver operating characteristic as a function of the window size before emergent intubation defined as the event, from 4 to 24 hours. The 95% confidence interval is indicated by errorbars and was determined by 200 bootstrap runs resampled by admission. Figure 4 shows the calibration of the 3 models using a 12-hour detection window. Well calibrated models have predicted risk equal to observed risk (dashed line). The SICU models from Moss1 and Politano2 exhibited excellent calibration in all but the highest risk patients, where they overestimated the risk. The MICU intubation model from Moss1 also overestimated risk for high risk patients and also deviated from the line of identity in the 2 lowest deciles of predicted risk. ![Figure 4:](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2021/05/26/2021.05.24.21257711/F4.medium.gif) [Figure 4:](http://medrxiv.org/content/early/2021/05/26/2021.05.24.21257711/F4) Figure 4: Calibration curves for the 3 models for emergent intubation. The observed relative risk is shown as a function of the predicted risk. Each point represents 10% of the data, and the line of identity (perfect calibration) is shown as a dashed line. ## DISCUSSION This study evaluated the performance of 3 predictive analytical models for early detection of respiratory failure leading to emergent intubation at an external center. These models are based upon converting continuous physiologic monitoring data into a novel digital signature that can be easily displayed to the bedside practitioner. The existing models were originally developed to predict emergent intubation in the UVa SICU (2 models) and MICU (1 model), and this study attempted to externally validated the performance using an independent cohort of patients from all ICUs at the University of California San Francisco. The major findings were that the models accurately identified patients at risk for emergent intubation (AUC > 0.75 starting at 12 hours pre-intubation), risk estimates continued to rise up to the time of emergent intubation, and these predictive models were well-calibrated. Whereas elective intubation in the operating room is an extremely safe procedure, emergent intubation occurring out of the operating room has been shown to be a significantly vulnerable time for patients with a high association of complications16-18. These complications include hemodynamic compromise or severe hypoxemia leading to cardiac arrest, esophageal intubation, aspiration, and pneumothorax17, 20-21. Cardiac arrest attributable to intubation complications occur between 2-4% of all emergent intubations with few of these patients surviving to hospital discharge even when initially successfully resuscitated18-20. Identifying patients earlier who are at risk of needing emergent intubation may be key in avoiding these poor outcomes18,21. Recent studies have linked poor outcome to a number of potentially mitigatable factors that are time-dependent. These include a worse outcome for patients intubated around change of nursing shift17. Wardi et al. hypothesized this finding to be a result of potential staff fatigue, hand-off errors, and lack of familiarity with a patient which combined lead to a tendency to overlook subtle but important changes in patient condition17. In each of these potential explanations, predictive algorithms such as those validated in this current study has theoretic potential to overcome these challenges at vulnerable moments for our critically ill patients. Essentially these algorithms convert complex data that is difficult for a clinician to assimilate in real-time into a patient specific digital signature. This signature serving as an early warning signal displayed at the bedside could aid in drawing attention to key changes in patient condition. Additionally, our data support that for those patients who ultimately require an emergent intubation, the decline in their physiologic state is often a slowly progressive process over many hours extending out beyond the preceding 12-24 hours. This time frame creates an opportunity to perform interventions that would make emergent intubation safer, and even possibly avoided. Prior research has shown a pre-intubation checklist integrating interventions to treat hemodynamic instability led to fewer complications17,18,22. This is consistent with investigations that have correlated the worse outcomes in emergent intubation in the Emergency Department (ED) occurring in those that develop hemodynamic compromise following intubation20,23. Commonly patients requiring emergent intubation are in shock with an increased shock index highly correlated with intubation associated cardiac arrest17,23. Volume depletion, vasodilation, acidosis, and reduced venous return resulting from increased positive pressure ventilation are all factors that contribute. Each of these could be addressed, albeit to differing degrees of success, in the lead time provided by an early warning signal. Trivedi et al. argue that the shock index which is calculating from a ratio of maximum heart rate to lowest systolic blood pressure is a helpful adjunct to make intervention in the immediate 60 minutes preceding intubation23. However, they acknowledge that addressing the ‘dynamic changes in patient status in the ICU…require continuous monitoring and interpretation of data before the development of overt hypotension and cardiorespiratory collapse23.’ Our study indicates that implementation and integration of digital signatures into clinical practice may provide even greater opportunity to induce timely clinical action as prediction of need for emergent intubation can be extended out to more than 12 hours pre-intubation. We can not definitively determine from this study if interventions during this window would avoid intubation, but it would likely make them safer. Both of these warrant further investigation in a prospective study. This study was limited in several other ways. We relied on surrogate data to initially identify patients who had mechanical ventilation initiated during their ICU stay. It is possible that due to documentation errors, the location of intubation initiation could have misclassified. However, to avoid this, each potential case of ICU intubation was reviewed by two clinicians to verify location of intubation, reason for intubation, and timing of intubation. Similarly, we attempted to identify tracheostomy patients that may go on and off mechanical ventilation and excluded those with mechanical ventilation initiation following tracheostomy. Determining the exact timing of emergent intubation was also challenging in this study. Physician documentation through intubation procedure notes were not a reliable source of the timing as it was apparent in chart review that this documentation is often done in a delayed manner. This is understandable as the physician attention around the event is focused on providing bedside care. To address this, the current study utilized the first nursing documentation of intubation medications being administered cross referenced with the respiratory therapist documentation of initial ventilator settings to confirm intubation timing. In all cases, these times were within 5 minutes of each other. In addition, we did not quantify the performance of predictive models in the context of known risk factors: diagnoses, demographics, or severity of illness 3, 8. We note, however, that these known risk factors are static indicators; though they may identify high risk patients, they do not rise leading up to the time of lung failure. Finally, we did not evaluate other predictors of emergent intubation such as shock index from vital signs and laboratory measurements. Adding independent streams of information improves performance24, though we found that models that are plug-and-play with bedside physiological monitors have excellent performance. ## CONCLUSION By leveraging the growth in digital technology paired with graphical visualization, early warning signals displayed at the bedside could aid in provider ability to identify subtle, but key changes in patient condition. Earlier identification of subtle changes in physiologic deterioration may provide a useful adjunct to clinicians to mitigate need for emergent intubation. ## Data Availability N/A ## AUTHOR CONTRIBUTIONS Study Design: Callcut, Clark, Moorman, Hu Data Analysis: Callcut, Xu, Clark, Lake, Moorman, Hu Data Interpretation: Callcut, Xu, Tsai, Villaroman, Robles, Lake, Clark, Moorman, Hu Writing: Callcut, Clark Critical Review: Callcut, Xu, Tsai, Villaroman, Robles, Lake, Clark, Moorman, Hu ## Footnotes * **EMAIL ADDRESSES Rachael A Callcut:** Rachael.Callcut{at}ucsf.edu, **Yuan Xu:** Yuan.Xu{at}ucsf.edu, **Christina Tsai:** Rachael.Callcut{at}ucsf.edu, **Andrea Villaroman:** Andrea.Villaroman{at}ucsf.edu, **Anamaria Robles:** Anamaria.Robles{at}ucsf.edu, **Doug Lake:** DEL2K{at}hscmail.mcc.virginia.edu, **Matthew Clark:** mclark{at}amp3d.biz, **J Randall Moorman:** rm3h{at}virginia.edu, **Xiao Hu:** Xiao.Hu{at}ucsf.edu * **CONFLICTS OF INTEREST:** Moorman and Clark are officers and own stock in Advanced Medical Predictive Devices, Diagnostics, and Displays * **FINANCIAL DISCLOSURES:** Dr. Callcut is supported by a Career Development Award from the NIH Big Data to Knowledge Initiative (NIH K01ES026834). * Received May 24, 2021. * Revision received May 24, 2021. * Accepted May 26, 2021. * © 2021, Posted by Cold Spring Harbor Laboratory This pre-print is available under a Creative Commons License (Attribution-NonCommercial-NoDerivs 4.0 International), CC BY-NC-ND 4.0, as described at [http://creativecommons.org/licenses/by-nc-nd/4.0/](http://creativecommons.org/licenses/by-nc-nd/4.0/) ## REFERENCES 1. 1.Moss TJ, Lake DE, Calland JF, Enfield KB, Delos JB, Fairchild KD, et al. Signatures of Subacute Potentially Catastrophic Illness in the ICU: Model Development and Validation. Crit Care Med. 2016;44(9):1639–48. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1097/CCM.0000000000001738&link_type=DOI) 2. 2.Politano AD, Riccio LM, Lake DE, Rusin CG, Guin LE, Josef CS, et al. Predicting the need for urgent intubation in a surgical/trauma intensive care unit. Surgery. 2013;154(5):1110–6. 3. 3.Kim D, Kobayashi L, Chang D, Fortlage D, Coimbra R. Development of a preliminary risk index to identify trauma patients at risk for an unplanned intubation. J Trauma Acute Care Surg. 2014;76(1):167–71. 4. 4.Nagata K, Morimoto T, Fujimoto D, Otoshi T, Nakagawa A, Otsuka K, et al. Efficacy of High-Flow Nasal Cannula Therapy in Acute Hypoxemic Respiratory Failure: Decreased Use of Mechanical Ventilation. Respir Care. 2015;60(10):1390–6. [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6ODoicmVzcGNhcmUiO3M6NToicmVzaWQiO3M6MTA6IjYwLzEwLzEzOTAiO3M6NDoiYXRvbSI7czo1MDoiL21lZHJ4aXYvZWFybHkvMjAyMS8wNS8yNi8yMDIxLjA1LjI0LjIxMjU3NzExLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 5. 5.Liu Y, An Z, Chen J, Tang Y, Han Q, Lu F, et al. Risk factors for noninvasive ventilation failure in patients with post-extubation acute respiratory failure after cardiac surgery. J Thorac Dis. 2018;10(6):3319–28. 6. 6.Keim-Malpass J, Enfield KB, Calland JF, Lake DE, Clark MT. Dynamic data monitoring improves predictive analytics for failed extubation in the ICU. Physiol Meas. 2018;39(7):075005. 7. 7.Shimabukuro DW, Barton CW, Feldman MD, Mataraso SJ, Das R. Effect of a machine learning-based severe sepsis prediction algorithm on patient survival and hospital length of stay: a randomised clinical trial. BMJ Open Respir Res. 2017;4(1):e000234. [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NzoiYm1qcmVzcCI7czo1OiJyZXNpZCI7czoxMToiNC8xL2UwMDAyMzQiO3M6NDoiYXRvbSI7czo1MDoiL21lZHJ4aXYvZWFybHkvMjAyMS8wNS8yNi8yMDIxLjA1LjI0LjIxMjU3NzExLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 8. 8.Ramachandran SK, Nafiu OO, Ghaferi A, Tremper KK, Shanks A, Kheterpal S. Independent predictors and outcomes of unanticipated early postoperative tracheal intubation after nonemergent, noncardiac surgery. Anesthesiology. 2011;115(1):44–53. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1097/ALN.0b013e31821cf6de&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=21552116&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F05%2F26%2F2021.05.24.21257711.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000291925400010&link_type=ISI) 9. 9.Clark MT, Vergales BD, Paget-Brown AO, Smoot TJ, Lake DE, Hudson JL, et al. Predictive monitoring for respiratory decompensation leading to urgent unplanned intubation in the neonatal intensive care unit. Pediatr Res. 2013;73(1):104–10. 10. 10.Blackburn HN, Clark MT, Moss TJ, Young JS, Moorman JR, Lake DE, et al. External validation in an intermediate unit of a respiratory decompensation model trained in an intensive care unit. Surgery. 2017;161(3):760–70. 11. 11.Blackburn HN, Clark MT, Moorman JR, Lake DE, Calland JF. Identifying the low risk patient in surgical intensive and intermediate care units using continuous monitoring. Surgery. 2018;163(4):811–8. 12. 12.Lake DE, Moorman JR. Accurate estimation of entropy in very short physiological time series: the problem of atrial fibrillation detection in implanted ventricular devices. Am J Physiol Heart Circ Physiol. 2011;300(1):H319–25. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1152/ajpheart.00561.2010&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=21037227&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F05%2F26%2F2021.05.24.21257711.atom) 13. 13.Goldberger AL, Amaral LA, Glass L, Hausdorff JM, Ivanov PC, Mark RG, et al. PhysioBank, PhysioToolkit, and PhysioNet: components of a new research resource for complex physiologic signals. Circulation. 2000;101(23):E215–20. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1161/01.CIR.101.23.e215&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=10851218&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F05%2F26%2F2021.05.24.21257711.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000087571900001&link_type=ISI) 14. 14.Peng CK, Buldyrev SV, Havlin S, Simons M, Stanley HE, Goldberger AL. Mosaic organization of DNA nucleotides. Phys Rev E Stat Phys Plasmas Fluids Relat Interdiscip Topics. 1994;49(2):1685–9. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1103/PhysRevE.49.1685&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=9961383&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F05%2F26%2F2021.05.24.21257711.atom) 15. 15.Peng CK, Havlin S, Stanley HE, Goldberger AL. Quantification of scaling exponents and crossover phenomena in nonstationary heartbeat time series. Chaos. 1995;5(1):82–7. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1063/1.166141&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=11538314&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F05%2F26%2F2021.05.24.21257711.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=A1995RA28200014&link_type=ISI) 16. 16.Arya R, Schrift D, Choe C, Al-Jahbeer M. Real-time Tracheal Ultrasound for the Confirmation of Endotracheal Intubations in the Intensive Care Unit: An Observational Study. J Ultrasound Med. 2018. 2018 Jul 29. doi: 10.1002/jum.14723. [Epub ahead of print] [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1002/jum.14723&link_type=DOI) 17. 17.Wardi G, Villar J, Nguyen T, Vyas A, Minokadeh A, Lasoff D, Tainter C, Beitler JR, Sell RE. Factors and outcomes associated with inpatient cardiac arrest following emergent endotracheal intubation. Resuscitation. 2017;121:76–80. 18. 18.Rochlen LR, Housey M, Gannon I, Mitchell S, Rooney DM, Tait AR, Engoren M. Assessing anesthesiology residents’ out-of-the-operating-room (OOOR) emergent airway management. BMC Anesthesiology. 2017;17:96. 19. 19.Mort TC. The incidence and risk factors for cardiac arrest during emergency tracheal intubation: a justification for incorporating the ASA Guidelines in the remote location. J Clinical Anesthesia. 2004;16(7):508–516. 20. 20.Heffner AC, Swords DS, Neale MN, Jones AE. Incidence and factors associated with cardiac arrest complication emergency airway management. Resuscitation. 2013;84(11):1500–4. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.resuscitation.2013.07.022&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=23911630&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F05%2F26%2F2021.05.24.21257711.atom) 21. 21.Keim-Malpass J, Kitzmiller RR, Skeeles-Worley A, Lindberg C, Clark MT, Tai R, et al. Advancing Continuous Predictive Analytics Monitoring: Moving from Implementation to Clinical Action in a Learning Health System. Crit Care Nurs Clin North Am. 2018;30(2):273–87. 22. 22.Jaber S, Jung B, Corne P, Sebbane M, Muller L, Chanques G, Verzilli D, Jonquet O, Eledjam JJ, Lefrant JY. An intervention to decrease complications related to endotrachael intubation in the intensive care unit: a prospective, multiple-center study. Intensive Care Med. 2010;36(2):248–55. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1007/s00134-009-1717-8&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=19921148&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F05%2F26%2F2021.05.24.21257711.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000273810500010&link_type=ISI) 23. 23.Trivedi S, Demirci O, Arteaga G, Kashyap R, Smischney NJ. Evaluation of preintubation shock index and modified shock index as predictors of postintubation hypotension and other short-term outcomes. J Crit Care. 2015;861.e1–e7. 24. 24.Moss TJ, Clark MT, Calland JF, Enfield KB, Voss JD, Lake DE, et al. Cardiorespiratory dynamics measured from continuous ECG monitoring improves detection of deterioration in acute care patients: A retrospective cohort study. PLoS One. 2017;12(8):e0181448. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1371/journal.pone.0181448&link_type=DOI)