Abstract
Machine learning models have increasingly been used to identify predictors of treatment response in depression, and it is hoped that they may eventually help with clinical decision making. However, the performance of these models has generally been poor. One possible reason is that they are typically trained to predict aggregate scores of several depression symptoms; by contrast, individual symptoms may behave differently, be more predictable and/or more responsive to treatment. We tested this possibility by comparing the performance of machine learning models for predicting early response to psychotherapy based on 21 different outcome measures: (i) 16 individual depression symptoms, (ii) 4 latent symptom factors for sleep, appetite, motivation, and negative affect related symptoms, and (iii) total scores based on the widely used Quick Inventory of Depressive Symptomatology (QIDS). We used a large real-world dataset of 85 baseline features spanning sociodemographic, cognitive, clinical, lifestyle and physical health assessments in patients (N=776) initiating internet-delivered cognitive behavioural therapy (iCBT). For all 21 outcome measures, we developed elastic net models (N=543) and validated their performance in an unseen hold-out sample (N=233). In the hold-out dataset the model predicting total depression scores achieved an R2 of 40% variance explained, while there was substantial variability in model performance for individual symptoms (R2:2.1%-44%) and latent symptom factors (R2:26%-44%). Model comparisons revealed that most individual symptom and latent factor models with all 85 predictors were not superior to simpler benchmark models comprising only age, sex and baseline levels of the respective depression outcome measure. The benchmark was outperformed by models predicting total scores (ΔR2=0.054, p=0.034), sad mood (ΔR2=0.106, p=0.001), loss of interest (ΔR2=0.079, p=0.021) and a latent factor representing negative affect and thought (ΔR2=0.054, p=0.038). Specifically, these models benefitted from additional predictors, such as treatment expectation, suicidal ideation, social support, or functional impairment. Our predictive modelling approach suggests new avenues towards a more patient-centred precision psychiatry, by providing clinicians with individual-level prognoses and predictors for interventions at the symptom level.
Competing Interest Statement
DR and SH are current employees of and hold shares in SilverCloud Health, Amwell. CTL became an employee of SilverCloud Health, Amwellpost-completion of this researchas part of her PhD studentship, funded by SilverCloud Health, Amwell and the Irish Research Council via an industry-academia partnership; she holds no shares in the company.CMG reports no financial relationships with commercial interests but was the primary supervisor of CTL. KES acknowledges support by the Rene and Susanne Braginsky Foundation and the ETH Foundation.
Funding Statement
This work was funded by a fellowship awarded to CMG from MQ: transforming mental health (MQ16IP13). CMG holds additional funding from Science Foundation Irelands Frontiers for the Future Scheme (19/FFP/6418), and a European Research Council (ERC) Starting Grant (ERC-H2020-HABIT). SKB was supported by a Biotechnology and Biological Sciences Research Council [grant number: BB/T008709/1] Ph.D. Studentship.
Author Declarations
I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.
Yes
The details of the IRB/oversight body that provided approval or exemption for the research described are given below:
The study obtained ethical approval from the Research Ethics Committee of School of Psychology, Trinity College Dublin and the Northwest-Greater Manchester West Research Ethics Committee of the National Health Service, Health Research Authority and Health and Care Research Wales. All methods were performed in accordance with the relevant guidelines and regulations. All participants provided informed consent to participate in the study online before they proceeded to the screening stage of the study.
I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.
Yes
I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).
Yes
I have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable.
Yes
Footnotes
some typos in the abstract
Data Availability
The data used in this study can be made available upon reasonable request, at the discretion of the corresponding authors.
