RT Journal Article
SR Electronic
T1 Limitations of machine learning in psychiatry: Participation in the PAC 2018 depression challenge
JF medRxiv
FD Cold Spring Harbor Laboratory Press
SP 19000562
DO 10.1101/19000562
A1 Eitel, Fabian
A1 Stober, Sebastian
A1 Waller, Lea
A1 Dorfschmidt, Lena
A1 Walter, Henrik
A1 Ritter, Kerstin
YR 2019
UL http://medrxiv.org/content/early/2019/07/08/19000562.abstract
AB Classifying subjects with psychiatric diseases based on structural magnetic resonance imaging (MRI) data is a highly challenging task. Nevertheless, a number of studies report decoding accuracies of up to 90% using relatively small sample sizes. Here, we will present contradicting results on a large scale (N = 2240) dataset of subjects with and without depression as part of our participation in the Predictive Analytics Competition (PAC) 2018, in which we achieved the 3rd place. Contrary to our expectations, classification accuracies varied only little between a variety of simple and highly-complex classifiers and did not exceed 67% based on an internal validation set. We discuss our results in light of these opposing results and conclude that well designed challenges and large sample sizes provide a good way to get unbiased estimates of prediction performance.Competing Interest StatementThe authors have declared no competing interest.Funding StatementWe acknowledge support from the German Research Foundation (KR: DFG, 389563835), the Manfred and Ursula-M?ller Stiftung (KR), Charité - Universitätsmedizin Berlin (KR: Rahel-Hirsch scholarship) and Studienstiftung des Deutschen Volkes (LD).Author DeclarationsAll relevant ethical guidelines have been followed and any necessary IRB and/or ethics committee approvals have been obtained.YesAll necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived.YesAny clinical trials involved have been registered with an ICMJE-approved registry such as ClinicalTrials.gov and the trial ID is included in the manuscript.NAI have followed all appropriate research reporting guidelines and uploaded the relevant Equator, ICMJE or other checklist(s) as supplementary files, if applicable.NAData used in this study were collected from different sites and were only released for the purpose of the PAC challenge.https://www.photon-ai.com/pac