Development of an electronic frailty index for predicting mortality in patients undergoing transcatheter aortic valve replacement using machine learning

Background: Electronic frailty indices can be useful surrogate measures of frailty. We assessed the role of machine learning to develop an electronic frailty index, incorporating demographics, baseline comorbidities, healthcare utilization characteristics, electrocardiographic measurements, and laboratory examinations, and used this to predict all-cause mortality in patients undergoing transaortic valvular replacement (TAVR). Methods: This was a territory-wide observational study of patients admitted to public hospitals from Hong Kong between 1 January 2000 and 31 December 2019 for TAVR. Significant univariate and multivariate predictors of all-cause mortality were identified using Cox regression. Importance ranking of variables was obtained with a gradient boosting survival tree (GBST) model, a supervised sequential ensemble learning algorithm, and used to build the frailty models. Comparisons were made between multivariate Cox, GBST and random survival forest models. Results: A total of 450 patients (49% females; median age at procedure 82.3 (interquartile range, IQR 79.0-86.0)) were included, of which 22 died during follow-up. A machine learning survival analysis model found that the most important predictors of mortality were APTT, followed by INR, severity of tricuspid regurgitation, cumulative hospital stays, cumulative number of readmissions, creatinine, urate, ALP, and QTc/QT intervals. GBST significantly outperformed random survival forests and multivariate Cox regression (precision: 0.91, recall: 0.89, AUC: 0.93, C-index: 0.96, and KS-index: 0.50) for mortality prediction. Conclusions: An electronic frailty index incorporating multi-domain data can efficiently predict all-cause mortality in patients undergoing TAVR. A machine learning survival learning model significantly improves the risk prediction performance of the frailty models.


Conclusions: An electronic frailty index incorporating multi-domain data can efficiently
predict all-cause mortality in patients undergoing TAVR. A machine learning survival learning model significantly improves the risk prediction performance of the frailty models.

Introduction
Aortic stenosis (AS), reduction in the effective orifice area of the semilunar cardiac valve at the interface of the left ventricle and the systemic arterial circulation, is a significant medical problem globally 1 . AS has a prevalence of 2-9% of the population over 65 years and 4% in those who are over 85 2 . It is associated with aging and confers a poor prognosis 3 .
There are different types of operations for the treatment of AS. Of these, transcatheter aortic valve replacement (TAVR) is a less invasive alternative to surgical aortic valve replacement for patients with severe, symptomatic AS, especially for those who are at high risk or intermediate risk of adverse events, e.g. those with a high number of comorbidities 4 .
Moreover, recent work has found that TAVR in low-risk patients is noninferior to surgical management 5 .
TAVR has been shown to have a satisfactory efficacy and safety and recommendations by guidelines from various international societies 6,7 . However, recent studies have found that frailty is a common finding in AS patients and is associated with increased mortality. Approximately half of patients who undergo TAVR die within four years of the procedure 8 . It is therefore crucial to weigh up risks and benefits before when offering TAVR to patients, in particular, trying to select those in whom the procedure is likely to confer greater gains in symptoms and prognosis, and identifying those who may not benefit, or indeed suffer harm, following TAVR.
Machine learning techniques have been widely applied in medical research.
Specifically, a gradient boosting survival tree model has recently been explored as an . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint The copyright holder for this this version posted December 26, 2020. ; efficient method for diagnosing coronary artery disease 9 . In this territory-wide study, we tested the hypothesis that an electronic frailty index incorporating demographics, baseline comorbidities, healthcare utilization characteristics, electrocardiographic measurements, and laboratory examinations using a gradient boosting approach can improve risk prediction for all-cause mortality.

Study design
This study was approved by The Joint Chinese University of Hong Kong -New

Data extraction and variables
The following data were extracted: 1) Baseline characteristics of gender, age at TAVR, age at first presentation with AS, TR severity, AR severity, MR severity, PR severity, complete recovery status, INR on the day of TAVR procedure; 2) baseline comorbidities including bradyarrhythmia, atrial fibrillation/flutter, tachyarrhythmia, diabetes mellitus, . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint The copyright holder for this this version posted December 26, 2020. ; hypertension, hyperlipidaemia, respiratory diseases, kidney diseases, endocrine disorders (other than diabetes mellitus) and gastrointestinal diseases using corresponding ICD-9 codes; 3) ECG measurements of ventricular rate, P-wave duration (PWD), PR interval, QRS duration, QT interval, corrected QT interval (QTc), P-wave axis, QRS axis, T-wave axis, Rwave amplitude in V5, and S-wave amplitude in V1; 4) healthcare utilization characteristics before TAVR presentation: cumulative hospital stay, cumulative number of hospital readmissions, and number of emergency readmissions (within 28 days of discharge); 5) laboratory tests: complete blood count, liver function tests, renal function tests. Details of ICD codes used for comorbidity identification are provided in the Supplementary Appendix.

Primary outcome and statistical analysis
The primary outcome was all-cause mortality. Descriptive statistics were presented for the overall cohort and categorized based on mortality status. Continuous variables were presented as median (95% confidence interval [CI]  is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint The copyright holder for this this version posted December 26, 2020. ; were performed were using a 15-inch MacBook Pro with 2.2 GHz Intel Core i7 Processor and 16 GB RAM (Hong Kong, China).

Development of a gradient boosting survival tree model
Survival analysis is a statistical method to deal with lifetime data, where the outcome is the time to occurrence of an event of interest, such as mortality. The most widely applied survival analysis model in biostatistics is the Cox proportional hazards model 10 . Gradient boosting, a class of machine learning methods, was developed based on the concept that a tree-based model after being sequentially combined with previous weak models (e.g. decision trees) in a stage-wise way can generate superior predictions for survival and other outcomes. Tree-structured survival models such as survival trees 11 and random survival forests 12 have been used to determine survival probabilities in medical studies. This inspired us to use a nonparametric ensemble tree model called a gradient boosting survival tree (GBST) that extends the survival tree models with the concept of gradient boosting 13 . GBST optimizes the survival probability of each time period simultaneously and therefore is able to significantly reduce the overall prediction error of a survival tree. In this study, GBST was used for mortality risk prediction of patients undergoing TAVR. A tree-structure based approach for ranking the importance value of different variables was used, to construct a machine learning based, electronic frailty index for predicting mortality outcome. To examine the GBST performance of survival risk discrimination and compare it with baseline models of random survival forests and a multivariate Cox regression model, we adopted a five-fold cross validation approach. The concordance index (C Index) proposed by Harrell et al. (1982) was used to measure the goodness of fit for the survival model, as the statistic provides a global assessment of the . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint The copyright holder for this this version posted December 26, 2020. ; model for the continuous event. The territory-wide cohort dataset also guarantees the c-Index not to be limited by censoring. We applied the c-index introduced above as well as a precision, recall, Kolmogorov-Smirnov (KS) index and the area under the receiver operating characteristics curve (AUC) to measure the goodness of model fit. The R packages, gbm (Version 2.1.5), randomForestSRC (Version 2.9.3), survival (Version 2.42-3) and ggplot2 (Version 3.3.2), were used to generate the mortality prediction results.

Baseline cohort characteristics
The baseline characteristics of this cohort of patients undergoing TAVR are shown in Table 1   whereas QRS duration, QT interval, P-wave axis, T-wave axis, R-wave amplitude in V5 and S-wave amplitude in V1 were lower in those who died compared to those who were alive ( Table 1).

Predictors of mortality and frailty model
. CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint The copyright holder for this this version posted December 26, 2020. ; Univariate Cox regression analysis was performed to identify significant predictors of all-cause mortality (  Table 3). Severe tricuspid regurgitation, INR, haematocrit and potassium were significant predictors after adjustment (P<0.001).

Results of machine learning survival analysis and frailty score construction
The tree number in the GBST model was set to 320 according to the sensitivity analysis results (Supplementary Figure 1) Table 3). GBST showed the best survival prediction performance over RSF and multivariate Cox model according to evaluation metrics of precision, recall, AUC, C-Index, and KS-Index. The advantage of the GBST model stems from its ability to sequentially add weak decision tree learning models to the ensemble model, which can correct the prediction errors of prior models to minimize the overall prediction error. The predicted out-of-bag (OOB) survivals and cumulative hazards with the introduced GBST model are shown in Supplementary Figure 3.

Discussion
The main findings of this territory-wide study of patients undergoing TAVR are twofold. Firstly, patient demographics, comorbidities, healthcare utilization statistics prior to the procedure, laboratory examinations and ECG measurements were significant predictors of mortality. Secondly, a nonparametric gradient boosting survival tree model outperformed random survival forest model and multivariate Cox regression model for all-cause mortality prediction.
Frailty has been shown to be a strong predictor of adverse outcomes in patients with heart failure, and in those undergoing cardiac interventional procedures [14][15][16][17][18][19][20] . Specifically related to TAVR, previous studies have examined the value of determining frailty for risk stratification. For example, a study in 2018 showed that 11% patients of average age of 83 died 2 years after TAVR, and a geriatric assessment frailty score cut-off at ≥ 4 predicted 2- year mortality with a specificity of 80% 21 . Another study showed that 242 out of 544 TAVR patients were frail 1 year after the procedure based on frailty definition 22 .
. CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

(which was not certified by peer review) preprint
The copyright holder for this this version posted December 26, 2020. ; However, in clinical situations, it may be impractical to fully assess frailty status of patients and surrogates that can accurately model or reflect frailty would save time for patient assessment. Therefore, clinician-researchers have designed the electronic frailty index based on the concept that frailty is caused by the accumulation of health deficits 23 . Segal et al.
designed the electronic frailty index by selecting candidate variables based on their potential correlations with frailty state rather than mortality directly 24 . An electronic frailty index has been used as an efficient variable to predict mortality in TAVR 25 . In our territory-wide cohort, we developed an electronic frailty index based on predictors that were identified by Cox regression analysis and showed that this can predict all-cause mortality in patients undergoing TAVR.
Survival analysis has been widely used in clinical and epidemiological studies based on electronic health records (EHRs) that provide rich and diverse information for modelling and prediction. Survival analysis models in the literature may be parametric or semiparametric, including survival tree analysis in the context of conditional inference trees 26 , survival forest analysis considering inverse probability of censoring weighting to compensate censoring 27 , random survival forest model with log-rank test 12 , censoring unbiased regression trees and forests with censoring unbiased loss functions 28 , ensemble tree method for right-censored survival data 29 . Traditional Cox proportional hazard models are used to identify the linear combinations. Tree structure-based survival analysis models have been widely applied in medical studies such as mortality prediction in systolic heart failure 30 . In our study, we demonstrated that a nonparametric gradient boosting survival tree model significantly improved mortality prediction in patients undergoing TAVR.

Conclusions
. CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

(which was not certified by peer review) preprint
The copyright holder for this this version posted December 26, 2020. ; An electronic frailty index incorporating multi-domain data can efficiently predict allcause mortality in patients undergoing TAVR. A machine learning-drive survival model significantly improves the risk prediction performance of the frailty models.

Funding
None.

Data availability
The data underlying this article will be shared on reasonable request to the corresponding author.

Conflicts of Interest
All authors declare no conflict of interest.     is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint The copyright holder for this this version posted December 26, 2020. ; . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint The copyright holder for this this version posted December 26, 2020. ;