Structured Abstract
Background Lack of anatomy recognition represents a clinically relevant risk in abdominal surgery. Machine learning (ML) methods can help identify visible patterns and risk structures, however, their practical value remains largely unclear.
Materials and Methods Based on a novel dataset of 13195 laparoscopic images with pixel-wise segmentations of eleven anatomical structures, we developed specialized segmentation models for each structure and a combined model for all anatomical structures, and compared segmentation performance of both algorithms to a cohort of 28 physicians, medical students, and medical laypersons using the example of pancreas segmentation.
Results Mean Intersection-over-Union for segmentation of intraabdominal structures ranged from 0.28 to 0.83 and from 0.23 to 0.77 for the structure-specific and the combined semantic segmentation model, respectively. At average inference times per model of 28 ms and 71 ms, respectively, both variants are capable of near-real-time operation. Both models outperformed 26 out of 28 human participants in pancreas segmentation.
Conclusions These results demonstrate that ML methods have the potential to provide relevant assistance in anatomy recognition in minimally-invasive surgery in near-real-time. Future research should investigate the educational value and subsequent clinical impact of respective assistance systems.
Highlights
Based on a novel large-scale dataset of 13195 laparoscopic images, two machine learning models were developed for automated identification and delineation of 11 anatomical structures: One model for each structure (abdominal wall, colon, intestinal vessels (inferior mesenteric artery and inferior mesenteric vein with their subsidiary vessels), liver, pancreas, small intestine, spleen, stomach, ureter and vesicular glands) and a combined model for all structures.
The structure-specific and combined segmentation models demonstrated similar performance in identifying intraabdominal structures and can operate in near-real-time.
Both models outperformed 26 out of 28 human participants in pancreas segmentation, demonstrating their potential for real-time assistance in recognizing anatomical landmarks during minimally-invasive surgery.
Competing Interest Statement
The authors have declared no competing interest.
Funding Statement
This work has been funded by the project funding within the Else Kroener Fresenius Center for Digital Health (EKFZ), Dresden, Germany (project CoBot) and by the German Research Foundation DFG within the Cluster of Excellence EXC 2050: Center for Tactile Internet with Human-in-the-Loop (CeTI) (project number 390696704). FRK received funding from the Medical Faculty of the Technical University Dresden within the MedDrive Start program (grant number 60487) and from the Joachim Herz Foundation (Add-On Fellowship for Interdisciplinary Life Science). FMR received a doctoral student scholarship from the Carus Promotionskolleg Dresden.
Author Declarations
I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.
Yes
The details of the IRB/oversight body that provided approval or exemption for the research described are given below:
All experiments were performed in accordance with the ethical standards of the Declaration of Helsinki and its later amendments. The local Institutional Review Board (ethics committee at the Technical University Dresden) reviewed and approved this study (approval number: BO-EK-137042018). The trial was registered on clinicaltrials.gov (trial registration ID: NCT05268432). Written informed consent to laparoscopic image data acquisition, data annotation, data analysis, and anonymized data publication was obtained from all participants. Before publication, all data was anonymized according to the general data protection regulation of the European Union.
I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.
Yes
I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).
Yes
I have followed all appropriate research reporting guidelines and uploaded the relevant EQUATOR Network research reporting checklist(s) and other pertinent material as supplementary files, if applicable.
Yes
Footnotes
Revisions to figures and manuscript text, updated title for improved clarity, updated supplemental files
Data Availability
The Dresden Surgical Anatomy Dataset is publicly available via the following link: https://www.nature.com/articles/s41597-022-01719-2. All other data generated and analyzed during the current study are available from the corresponding authors on reasonable request. To gain access, data requestors will need to sign a data access agreement.
Abbreviations
- AI
- Artificial Intelligence
- IoU
- Intersection-over-Union
- SD
- Standard deviation
