A deep learning model for clinical outcome prediction using longitudinal inpatient electronic health records

Objective Recent advances in deep learning show significant potential in analyzing continuous monitoring electronic health records (EHR) data for clinical outcome prediction. We aim to develop a Transformer-based, Encounter-level Clinical Outcome (TECO) model to predict mortality in the intensive care unit (ICU) using inpatient EHR data.

Materials and Methods TECO was developed using multiple baseline and time-dependent clinical variables from 2579 hospitalized COVID-19 patients to predict ICU mortality, and was validated externally in an ARDS cohort (n=2799) and a sepsis cohort (n=6622) from the Medical Information Mart for Intensive Care (MIMIC)-IV. Model performance was evaluated based on area under the receiver operating characteristic (AUC) and compared with Epic Deterioration Index (EDI), random forest (RF), and extreme gradient boosting (XGBoost).

Results In the COVID-19 development dataset, TECO achieved higher AUC (0.89–0.97) across various time intervals compared to EDI (0.86–0.95), RF (0.87–0.96), and XGBoost (0.88–0.96). In the two MIMIC testing datasets (EDI not available), TECO yielded higher AUC (0.65–0.76) than RF (0.57–0.73) and XGBoost (0.57–0.73). In addition, TECO was able to identify clinically interpretable features that were correlated with the outcome.

Discussion TECO outperformed proprietary metrics and conventional machine learning models in predicting ICU mortality among COVID-19 and non-COVID-19 patients.

Conclusions TECO demonstrates a strong capability for predicting ICU mortality using continuous monitoring data. While further validation is needed, TECO has the potential to serve as a powerful early warning tool across various diseases in inpatient settings.

LAY SUMMARY In intensive care units (ICUs), accurately estimating the risk of death is crucial for timely and effective medical intervention. This study developed a new AI algorithm, TECO (Transformer-based, Encounter-level Clinical Outcome model), which uses electronic health records to continuously predict ICU mortality after admission, with the capability to update predictions on an hourly basis. TECO was trained on data from over 2,500 COVID-19 patients and was designed to analyze multiple types of continuous monitoring data collected during a patient’s ICU stay. We tested TECO’s performance against a widely used proprietary tool, the Epic Deterioration Index (EDI), and other machine learning methods, such as random forest and XGBoost, across three patient groups: COVID-19, ARDS (acute respiratory distress syndrome), and sepsis. TECO consistently showed better performance and was able to predict death risk earlier than other methods. Additionally, TECO identified key health indicators associated with ICU mortality, making its predictions more interpretable for clinicians. These findings suggest that TECO could become a valuable early warning tool, helping doctors monitor patients’ health and take timely action in a range of critical care situations.