TY - JOUR T1 - Description and clinical validation of a real-time AI diagnostic companion for fetal ultrasound examination JF - medRxiv DO - 10.1101/2021.05.25.21257630 SP - 2021.05.25.21257630 AU - Julien J. Stirnemann AU - Remi Besson AU - Emmanuel Spaggiari AU - Sandra Rojo AU - Frederic Loge AU - Helene Peyro-Saint-Paul AU - Stephanie Allassonniere AU - Erwan Le Pennec AU - Yves Ville Y1 - 2021/01/01 UR - http://medrxiv.org/content/early/2021/07/10/2021.05.25.21257630.abstract N2 - Objective To describe a real-time decision support system (DSS), named SONIO, to assist ultrasound-based prenatal diagnosis and to assess its performance using a clinical database of precisely phenotyped postmortem examinations.Population and Methods This DSS is knowledge-based and comprises a dedicated thesaurus of 294 syndromes and diseases. It operates by suggesting, at each step of the ultrasound examination, the best next symptom to check for in order to optimize the diagnostic pathway to the smallest number of possible diagnoses. This assistant was tested on a single-center database of 251 cases of postmortem phenotypes with a definite diagnosis. Adjudication of discordant diagnoses was made by a panel of external experts. The primary outcome was a target concordance rate >90% between the postmortem diagnosis and the top-7 diagnoses given by SONIO when providing the full phenotype as input. Secondary outcomes included concordance for the top-5 and top-3 diagnoses; We also assessed a “1-by-1” model, providing only the anomalies sequentially prompted by the system, mimicking the use of the software in a real-life clinical setting.Results The validation database covered 96 of the 294 (32.65%) syndromes and 79% of their overall prevalence in the SONIO thesaurus. The adjudicators discarded 42/251 cases as they were not amenable to ultrasound based diagnosis. SONIO failed to make the diagnosis on 7/209 cases. On average, each case displayed 6 anomalies, 3 of which were considered atypical for the condition. Using the ‘full-phenotype’ model, the success rate of the top-7 output of Sonio was 96.7% (202/209). This was 91.9% and 87.1% for the top-5 and top-3 outputs respectively. Using the “1-by-1” model, the correct diagnosis was within the top-7, top-5 and top-3 of SONIO’s output in 72.4%, 69.3% and 63.1%.Conclusion Sonio is a robust DSS with a success-rate >95% for top-7 ranking diagnoses when the full phenotype is provided, using a large database of noisy real data. The success rate over 70% using the ‘1-by-1’ model was understandably lower, given that SONIO’s sequential queries may not systematically cover the full phenotype.Competing Interest StatementJS, RB, ES, SA, ELP and YV are co-founders of the SONIO company. SR, FLM and HPSP are employed by the SONIO company.Clinical TrialThis was a retrospective observational study from clinical recordsFunding StatementNo fundingAuthor DeclarationsI confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.YesThe details of the IRB/oversight body that provided approval or exemption for the research described are given below:This study was approved by the local ethics committee of the Necker hospital (IRB registration : CERPAHP.5 #00011928)All necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived.YesI understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).YesI have followed all appropriate research reporting guidelines and uploaded the relevant EQUATOR Network research reporting checklist(s) and other pertinent material as supplementary files, if applicable.YesThe validation data will be made partly available, upon request. The algorithms and thesaurus of the SONIO DSS are the property of the SONIO company ER -