Abstract
Background This study aimed to develop and evaluate a machine learning model and a novel clinical score for predicting outcomes in stroke patients undergoing endovascular thrombectomy.
Methods This retrospective study included all patients aged over 18 years with an anterior circulation stroke treated at a thrombectomy centre from 2010 to 2020. External validation data was obtained. The primary outcome variable was day 90 mRS ≥3. Existing clinical scores (SPAN and PRE) and Machine Learning (ML) models were compared. A novel clinical score (iSPAN) was derived by adding an optimised weighting of the most important ML features to the SPAN and compared results.
Results 812 patients were initially included (397 female, average age 73), 62 for external validation. The best performing clinical score and ML model were SPAN and XGBoost (sensitivity specificity and accuracy 0.967, 0.290, 0.628 and 0.783, 0.693, 0.738 respectively). A significant difference was found overall and XGBoost was more accurate than SPAN (p< 0.0018). The most important features were Age, mTICI and total number of passes. The addition of 11 points for mTICI of ≤ 2B and ≥ 3 points for 3 passes to the SPAN achieved the best accuracy and was used to create the iSPAN. iSPAN was not significantly less accurate than XGBoost (p>0.5). In the external validation set, iSPAN and SPAN achieved sensitivity, specificity, and accuracy of (0.735, 0.862, 0.79) and (0.471, 0.897, 0.67), respectively.
Conclusions iSPAN incorporates machine-derived features to achieve better predictions compared to existing scores. It is not inferior to the XGB model and is externally generalisable.
Key Points
An XGB model performed better than existing scores and other tested models for prognostication post EVT.
It identified mTICI and number of passes as important and modifiable factors.
Integrating these into the SPAN score (iSPAN) was not inferior to the XGB model and is generalisable and easier to use and interpret.
Competing Interest Statement
The authors have declared no competing interest.
Funding Statement
This work was performed within the Irish Clinical Academic Training (ICAT) Programme, supported by the Wellcome Trust and the Health Research Board (Grant No. 203930/B/16/Z), the Health Service Executive National Doctors Training and Planning and the Health and Social Care, Research and Development Division, Northern Ireland and the Faculty of Radiologists, Royal College of Surgeons in Ireland. This research was supported by Science Foundation Ireland (SFI) under Grant Number SFI/12/RC/2289_P2.
Author Declarations
I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.
Yes
The details of the IRB/oversight body that provided approval or exemption for the research described are given below:
Ethics committee/IRB of Beaumont Hospital waived ethical approval for this work
I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.
Yes
I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).
Yes
I have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable.
Yes
Footnotes
Funding This work was performed within the Irish Clinical Academic Training (ICAT) Programme, supported by the Wellcome Trust and the Health Research Board (Grant No. 203930/B/16/Z), the Health Service Executive National Doctors Training and Planning and the Health and Social Care, Research and Development Division, Northern Ireland and the Faculty of Radiologists, Royal College of Surgeons in Ireland. This research was supported by Science Foundation Ireland (SFI) under Grant Number SFI/12/RC/2289_P2.
Data Availability
Data available on reasonable request
Non standard abbreviations
- (AI)
- Artificial Intelligence
- (EVT)
- Endovascular thrombectomy
- (XGB)
- eXtreme Gradient Boosting
- (KNN)
- K-Nearest Neighbour
- (SVM)
- Support Vector Machine
- (ML)
- Machine Learning
- (MLP)
- Multilayer Perceptron
- (SPAN)
- Stroke Prediction using Age and NIHSS
- (our funders)
- The added “I” in iSPAN refers to Improved, artificial Intelligence, Insight (our research centre), and the ICAT programme
- (PRE)
- Pittsburgh Response to Endovascular therapy
