RT Journal Article
SR Electronic
T1 Effectiveness, Explainability and Reliability of Machine Meta-Learning Methods for Predicting Mortality in Patients with COVID-19: Results of the Brazilian COVID-19 Registry
JF medRxiv
FD Cold Spring Harbor Laboratory Press
SP 2021.11.01.21265527
DO 10.1101/2021.11.01.21265527
A1 Miranda de Paiva, Bruno Barbosa
A1 Delfino-Pereira, Polianna
A1 de Andrade, Claudio Moisés Valiense
A1 Gomes, Virginia Mara Reis
A1 Lima, Maria Clara Pontello Barbosa
A1 Souza-Silva, Maira Viana Rego
A1 Carneiro, Marcelo
A1 Martins, Karina Paula Medeiros Prado
A1 Sales, Thaís Lorenna Souza
A1 de Carvalho, Rafael Lima Rodrigues
A1 Pires, Magda C.
A1 F. Ramos, Lucas Emanuel
A1 Silva, Rafael T.
A1 Bezerra, Adriana Falangola Benjamin
A1 Schwarzbold, Alexandre Vargas
A1 Nunes, Aline Gabrielle Sousa
A1 Maurílio, Amanda de Oliveira
A1 Scotton, Ana Luiza Bahia Alves
A1 Costa, André Soares de Moura
A1 Castro, Andriele Abreu
A1 Farace, Bárbara Lopes
A1 Cimini, Christiane Corrêa Rodrigues
A1 De Carvalho, Cíntia Alcantara
A1 Silveira, Daniel Vitório
A1 Ponce, Daniela
A1 Pereira, Elayne Crestani
A1 Manenti, Euler Roberto Fernandes
A1 Cenci, Evelin Paola de Almeida
A1 Lucas, Fernanda Barbosa
A1 Rodrigues, Fernanda D’Athayde
A1 Anschau, Fernando
A1 Botoni, Fernando Antonio
A1 Graça Aranha, Fernando
A1 Bartolazzi, Frederico
A1 Bastos, Gisele Alsina Nader
A1 Vietta, Giovanna Grunewald
A1 Nascimento, Guilherme Fagundes
A1 Noal, Helena Carolina
A1 Duani, Helena
A1 Vianna, Heloisa Reniers
A1 Guimarães, Henrique Cerqueira
A1 Gomes, Isabela Moraes
A1 Salles Martins Costa, Jamille Hemétrio
A1 da Fonseca, Jéssica Rayane Corrêa Silva
A1 Guimarães, Júlia Di Sabatino Santos
A1 de Morais, Júlia Drumond Parreiras
A1 Rugolo, Juliana Machado
A1 Batista, Joanna D’arc Lyra
A1 de Alvarenga, Joice Coutinho
A1 Chatkin, José Miguel
A1 Ruschel, Karen Brasil
A1 Moreira, Leila Beltrami
A1 de Oliveira, Leonardo Seixas
A1 Zandoná, Liege Barella
A1 Pinheiro, Lílian Santos
A1 Monteiro, Luanna da Silva
A1 Sousa, Lucas de Deus
A1 Kopittke, Luciane
A1 Viana, Luciano de Souza
A1 de Castro, Luis César
A1 Assis, Luisa Argolo
A1 Santos, Luisa Elem Almeid
A1 Cabral, Máderson Alvares de Souza
A1 Raposo, Magda Cesar
A1 Floriani, Maiara Anschau
A1 Ferreira, Maria Angélica Pires
A1 Bicalho, Maria Aparecida Camargos
A1 de Godoy, Mariana Frizzo
A1 Nogueira, Matheus Carvalho Alves
A1 de Figueiredo, Meire Pereira
A1 Guimarães-Júnior, Milton Henriques
A1 De Sordi, Mônica Aparecida de Paula
A1 Sampaio, Natália da Cunha Severino
A1 de Oliveira, Neimy Ramos
A1 Assaf, Pedro Ledic
A1 Lutkmeier, Raquel
A1 Valacio, Reginaldo Aparecido
A1 Finger, Renan Goulart
A1 Senger, Roberta
A1 Menezes, Rochele Mosmann
A1 Silva, Rufino de Freitas
A1 Francisco, Saionara Cristina
A1 Guimarães, Silvana Mangeon Mereilles
A1 Araújo, Silvia Ferreira
A1 Oliveira, Talita Fischer
A1 Kurtz, Tatiana
A1 Fereguetti, Tatiani Oliveira
A1 de Oliveira, Thainara Conceição
A1 Diniz, Thulio Henrique Oliveira
A1 Ribeiro, Yara Cristina Neves Marques Barbosa
A1 Ramires, Yuri Carlotto
A1 Gonçalves, Marcos André
A1 Marcolino, Milena Soriano
YR 2021
UL http://medrxiv.org/content/early/2021/11/02/2021.11.01.21265527.abstract
AB Objective To provide a thorough comparative study among state-of-the-art machine learning methods and statistical methods for determining in-hospital mortality in COVID-19 patients using data upon hospital admission; to study the reliability of the predictions of the most effective methods by correlating the probability of the outcome and the accuracy of the methods; to investigate how explainable are the predictions produced by the most effective methods.Materials and Methods De-identified data were obtained from COVID-19 positive patients in 36 participating hospitals, from March 1 to September 30, 2020. Demographic, comorbidity, clinical presentation and laboratory data were used as training data to develop COVID-19 mortality prediction models. Multiple machine learning and traditional statistics models were trained on this prediction task using a folded cross-validation procedure, from which we assessed performance and interpretability metrics.Results The Stacking of machine learning models improved over the previous state-of-the-art results by more than 26% in predicting the class of interest (death), achieving 87.1% of AUROC and macro F1 of 73.9%. We also show that some machine learning models can be very interpretable and reliable, yielding more accurate predictions while providing a good explanation for the ‘why’.Conclusion The best results were obtained using the meta-learning ensemble model – Stacking. State-of the art explainability techniques such as SHAP-values can be used to draw useful insights into the patterns learned by machine-learning algorithms. Machine-learning models can be more explainable than traditional statistics models while also yielding highly reliable predictions.Competing Interest StatementThe authors have declared no competing interest.Funding StatementThis study was funded by CAPES and FAPEMIG Author DeclarationsI confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.YesThe details of the IRB/oversight body that provided approval or exemption for the research described are given below:The study protocol was approved by the Brazilian National Commission for Research Ethics (CAAE 30350820.5.1001.0008). Individual informed consent was waived due to the severity of the situation and the use of deidentified data, based on medical chart review only.I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.YesI understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).YesI have followed all appropriate research reporting guidelines and uploaded the relevant EQUATOR Network research reporting checklist(s) and other pertinent material as supplementary files, if applicable.YesAll data produced in the present study are available upon reasonable request to the authors.