Skip to main content
medRxiv
  • Home
  • About
  • Submit
  • ALERTS / RSS
Advanced Search

Convolutional neural networks on eye tracking trajectories in patients with spatial neglect

View ORCID ProfileBenedetta Franceschiello, View ORCID ProfileAlexia Bourgeois, Astrid Minier, View ORCID ProfileMicah M. Murray, Pierre Pouget, View ORCID ProfilePaolo Bartolomeo, View ORCID ProfileFabio Anselmi
doi: https://doi.org/10.1101/2020.07.02.20143941
Benedetta Franceschiello
1The LINE (Laboratory for Investigative Neurophysiology), Radiology Department, University Hospital Center and University of Lausanne, Lausanne, Switzerland
2Ophthalmology Department, Fondation Asile des aveugles and University of Lausanne, Lausanne, Switzerland
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Benedetta Franceschiello
  • For correspondence: benedetta.franceschiello@gmail.com
Alexia Bourgeois
3Laboratory of Cognitive Neurorehabilitation, Faculty of Medicine, University of Geneva, 1206 Geneva, Switzerland
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Alexia Bourgeois
Astrid Minier
1The LINE (Laboratory for Investigative Neurophysiology), Radiology Department, University Hospital Center and University of Lausanne, Lausanne, Switzerland
2Ophthalmology Department, Fondation Asile des aveugles and University of Lausanne, Lausanne, Switzerland
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Micah M. Murray
1The LINE (Laboratory for Investigative Neurophysiology), Radiology Department, University Hospital Center and University of Lausanne, Lausanne, Switzerland
2Ophthalmology Department, Fondation Asile des aveugles and University of Lausanne, Lausanne, Switzerland
4Sensory, Perceptual, and Cognitive Neuroscience Section, Center for Biomedical Imaging (CIBM), Lausanne, Switzerland
5Department of Hearing and Speech Sciences, Vanderbilt University, Nashville, TN, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Micah M. Murray
Pierre Pouget
6Sorbonne Université, Inserm U 1127, CNRS UMR 7225, Paris Brain Institute, ICM, Hôpital de la Pitié-Salpêtrière, 75013 Paris, France
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Paolo Bartolomeo
6Sorbonne Université, Inserm U 1127, CNRS UMR 7225, Paris Brain Institute, ICM, Hôpital de la Pitié-Salpêtrière, 75013 Paris, France
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Paolo Bartolomeo
Fabio Anselmi
7Center for Neuroscience and Artificial Intelligence, Department of Neuroscience, Baylor College of Medicine, Baylor Plaza, 77030, Houston, USA
8Center for Brains, Minds, and Machines | McGovern Institute for Brain Research at MIT,Cambridge, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Fabio Anselmi
  • Abstract
  • Full Text
  • Info/History
  • Metrics
  • Data/Code
  • Preview PDF
Loading

Abstract

Eye-movement trajectories are rich behavioral data, providing a window onto how the brain processes information. Analyses of these trajectories can be automated and benefit from machine learning algorithms. Among those, deep learning has recently proven very successful, setting new state-of-art results in many computer vision applications, including medical diagnosis systems. In this paper, we address the challenge of diagnosing and quantifying signs of visuo-spatial neglect from saccadic eye trajectories recorded in healthy controls and in brain-damaged patients with spatial neglect. We show how machine learning techniques, such as deep networks, can predict the patient’s status with unprecedented accuracy, benchmarking the algorithm prediction with structural Magnetic Resonance Images (MRI) of the patients’ brain lesions and their Diffusion Tensor Imaging (DTI) tracts. Preliminary evidence of correlation between MRI data and the algorithm scores suggest that a quantitative prediction of the patients’ impairment based only onto the behavioral data of eye trajectories seem possible, therefore opening to new horizons in the field of non-invasive diagnostics.

Competing Interest Statement

The authors have declared no competing interest.

Funding Statement

B.F., A.M., received financial support for this work from the Fondation Asile des Aveugles (grant #232933 to M.M.M.), a grantor advised by Carigest SA (#232920 to M.M.M.), as well as the Swiss National Science Foundation (grants #169206 to M.M.M.). A.B. acknowledges the support of Swiss National Science Foundation to Armin Schnider under grant 32003B-175472 and to Radek Ptak under grant 32003B-184702. The work of P.B. and P.P. is supported by Agence Nationale de la Recherche through ANR-16-CE37-0005 and ANR-10-IAIHU-06. F.A. is supported by the Intelligence Advanced Research Projects Activity (IARPA) via Department of Interior/Interior Business Center (DoI/IBC) contract number D16PC00003. The U.S. Government is authorized to reproduce and distribute reprints for Governmental purposes notwithstanding any copyright annotation thereon. Disclaimer: The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies or endorsements, either expressed or implied, of IARPA, DoI/IBC, or the U.S. Government.

Author Declarations

I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.

Yes

The details of the IRB/oversight body that provided approval or exemption for the research described are given below:

The dataset used in this study was reviewed by the INSERM ethical committee and received the approval of an Institutional Review Board (CPP Ile de France 1) https://www.sciencedirect.com/science/article/abs/pii/S0028393215001591

All necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived.

Yes

I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).

Yes

I have followed all appropriate research reporting guidelines and uploaded the relevant EQUATOR Network research reporting checklist(s) and other pertinent material as supplementary files, if applicable.

Yes

Data Availability

The dataset is detailed, analysed and discussed in https://www.sciencedirect.com/science/article/abs/pii/S0028393215001591 In the current work we only used the dateset for supplementary automatized analyses.

https://www.sciencedirect.com/science/article/abs/pii/S0028393215001591

Copyright 
The copyright holder for this preprint is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. All rights reserved. No reuse allowed without permission.
Back to top
PreviousNext
Posted July 05, 2020.
Download PDF
Data/Code
Email

Thank you for your interest in spreading the word about medRxiv.

NOTE: Your email address is requested solely to identify you as the sender of this article.

Enter multiple addresses on separate lines or separate them with commas.
Convolutional neural networks on eye tracking trajectories in patients with spatial neglect
(Your Name) has forwarded a page to you from medRxiv
(Your Name) thought you would like to see this page from the medRxiv website.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
Share
Convolutional neural networks on eye tracking trajectories in patients with spatial neglect
Benedetta Franceschiello, Alexia Bourgeois, Astrid Minier, Micah M. Murray, Pierre Pouget, Paolo Bartolomeo, Fabio Anselmi
medRxiv 2020.07.02.20143941; doi: https://doi.org/10.1101/2020.07.02.20143941
Digg logo Reddit logo Twitter logo CiteULike logo Facebook logo Google logo Mendeley logo
Citation Tools
Convolutional neural networks on eye tracking trajectories in patients with spatial neglect
Benedetta Franceschiello, Alexia Bourgeois, Astrid Minier, Micah M. Murray, Pierre Pouget, Paolo Bartolomeo, Fabio Anselmi
medRxiv 2020.07.02.20143941; doi: https://doi.org/10.1101/2020.07.02.20143941

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero
  • Tweet Widget
  • Facebook Like
  • Google Plus One

Subject Area

  • Neurology
Subject Areas
All Articles
  • Addiction Medicine (70)
  • Allergy and Immunology (168)
  • Anesthesia (50)
  • Cardiovascular Medicine (450)
  • Dentistry and Oral Medicine (82)
  • Dermatology (55)
  • Emergency Medicine (157)
  • Endocrinology (including Diabetes Mellitus and Metabolic Disease) (191)
  • Epidemiology (5248)
  • Forensic Medicine (3)
  • Gastroenterology (195)
  • Genetic and Genomic Medicine (755)
  • Geriatric Medicine (80)
  • Health Economics (212)
  • Health Informatics (696)
  • Health Policy (356)
  • Health Systems and Quality Improvement (223)
  • Hematology (99)
  • HIV/AIDS (162)
  • Infectious Diseases (except HIV/AIDS) (5848)
  • Intensive Care and Critical Care Medicine (358)
  • Medical Education (103)
  • Medical Ethics (25)
  • Nephrology (80)
  • Neurology (762)
  • Nursing (43)
  • Nutrition (130)
  • Obstetrics and Gynecology (141)
  • Occupational and Environmental Health (231)
  • Oncology (476)
  • Ophthalmology (150)
  • Orthopedics (38)
  • Otolaryngology (95)
  • Pain Medicine (39)
  • Palliative Medicine (20)
  • Pathology (140)
  • Pediatrics (223)
  • Pharmacology and Therapeutics (136)
  • Primary Care Research (96)
  • Psychiatry and Clinical Psychology (859)
  • Public and Global Health (2004)
  • Radiology and Imaging (348)
  • Rehabilitation Medicine and Physical Therapy (158)
  • Respiratory Medicine (283)
  • Rheumatology (94)
  • Sexual and Reproductive Health (73)
  • Sports Medicine (76)
  • Surgery (109)
  • Toxicology (25)
  • Transplantation (29)
  • Urology (39)