-
The Endoscapes Dataset for Surgical Scene Segmentation, Object Detection, and Critical View of Safety Assessment: Official Splits and Benchmark
Authors:
Aditya Murali,
Deepak Alapatt,
Pietro Mascagni,
Armine Vardazaryan,
Alain Garcia,
Nariaki Okamoto,
Guido Costamagna,
Didier Mutter,
Jacques Marescaux,
Bernard Dallemagne,
Nicolas Padoy
Abstract:
This technical report provides a detailed overview of Endoscapes, a dataset of laparoscopic cholecystectomy (LC) videos with highly intricate annotations targeted at automated assessment of the Critical View of Safety (CVS). Endoscapes comprises 201 LC videos with frames annotated sparsely but regularly with segmentation masks, bounding boxes, and CVS assessment by three different clinical experts…
▽ More
This technical report provides a detailed overview of Endoscapes, a dataset of laparoscopic cholecystectomy (LC) videos with highly intricate annotations targeted at automated assessment of the Critical View of Safety (CVS). Endoscapes comprises 201 LC videos with frames annotated sparsely but regularly with segmentation masks, bounding boxes, and CVS assessment by three different clinical experts. Altogether, there are 11090 frames annotated with CVS and 1933 frames annotated with tool and anatomy bounding boxes from the 201 videos, as well as an additional 422 frames from 50 of the 201 videos annotated with tool and anatomy segmentation masks. In this report, we provide detailed dataset statistics (size, class distribution, dataset splits, etc.) and a comprehensive performance benchmark for instance segmentation, object detection, and CVS prediction. The dataset and model checkpoints are publically available at https://github.com/CAMMA-public/Endoscapes.
△ Less
Submitted 26 January, 2024; v1 submitted 19 December, 2023;
originally announced December 2023.
-
Real-Time Artificial Intelligence Assistance for Safe Laparoscopic Cholecystectomy: Early-Stage Clinical Evaluation
Authors:
Pietro Mascagni,
Deepak Alapatt,
Alfonso Lapergola,
Armine Vardazaryan,
Jean-Paul Mazellier,
Bernard Dallemagne,
Didier Mutter,
Nicolas Padoy
Abstract:
Artificial intelligence is set to be deployed in operating rooms to improve surgical care. This early-stage clinical evaluation shows the feasibility of concurrently attaining real-time, high-quality predictions from several deep neural networks for endoscopic video analysis deployed for assistance during three laparoscopic cholecystectomies.
Artificial intelligence is set to be deployed in operating rooms to improve surgical care. This early-stage clinical evaluation shows the feasibility of concurrently attaining real-time, high-quality predictions from several deep neural networks for endoscopic video analysis deployed for assistance during three laparoscopic cholecystectomies.
△ Less
Submitted 13 December, 2022;
originally announced December 2022.
-
Temporally Constrained Neural Networks (TCNN): A framework for semi-supervised video semantic segmentation
Authors:
Deepak Alapatt,
Pietro Mascagni,
Armine Vardazaryan,
Alain Garcia,
Nariaki Okamoto,
Didier Mutter,
Jacques Marescaux,
Guido Costamagna,
Bernard Dallemagne,
Nicolas Padoy
Abstract:
A major obstacle to building models for effective semantic segmentation, and particularly video semantic segmentation, is a lack of large and well annotated datasets. This bottleneck is particularly prohibitive in highly specialized and regulated fields such as medicine and surgery, where video semantic segmentation could have important applications but data and expert annotations are scarce. In t…
▽ More
A major obstacle to building models for effective semantic segmentation, and particularly video semantic segmentation, is a lack of large and well annotated datasets. This bottleneck is particularly prohibitive in highly specialized and regulated fields such as medicine and surgery, where video semantic segmentation could have important applications but data and expert annotations are scarce. In these settings, temporal clues and anatomical constraints could be leveraged during training to improve performance. Here, we present Temporally Constrained Neural Networks (TCNN), a semi-supervised framework used for video semantic segmentation of surgical videos. In this work, we show that autoencoder networks can be used to efficiently provide both spatial and temporal supervisory signals to train deep learning models. We test our method on a newly introduced video dataset of laparoscopic cholecystectomy procedures, Endoscapes, and an adaptation of a public dataset of cataract surgeries, CaDIS. We demonstrate that lower-dimensional representations of predicted masks can be leveraged to provide a consistent improvement on both sparsely labeled datasets with no additional computational cost at inference time. Further, the TCNN framework is model-agnostic and can be used in conjunction with other model design choices with minimal additional complexity.
△ Less
Submitted 27 December, 2021;
originally announced December 2021.
-
Surgical data science for safe cholecystectomy: a protocol for segmentation of hepatocystic anatomy and assessment of the critical view of safety
Authors:
Pietro Mascagni,
Deepak Alapatt,
Alain Garcia,
Nariaki Okamoto,
Armine Vardazaryan,
Guido Costamagna,
Bernard Dallemagne,
Nicolas Padoy
Abstract:
Minimally invasive image-guided surgery heavily relies on vision. Deep learning models for surgical video analysis could therefore support visual tasks such as assessing the critical view of safety (CVS) in laparoscopic cholecystectomy (LC), potentially contributing to surgical safety and efficiency. However, the performance, reliability and reproducibility of such models are deeply dependent on t…
▽ More
Minimally invasive image-guided surgery heavily relies on vision. Deep learning models for surgical video analysis could therefore support visual tasks such as assessing the critical view of safety (CVS) in laparoscopic cholecystectomy (LC), potentially contributing to surgical safety and efficiency. However, the performance, reliability and reproducibility of such models are deeply dependent on the quality of data and annotations used in their development. Here, we present a protocol, checklists, and visual examples to promote consistent annotation of hepatocystic anatomy and CVS criteria. We believe that sharing annotation guidelines can help build trustworthy multicentric datasets for assessing generalizability of performance, thus accelerating the clinical translation of deep learning models for surgical video analysis.
△ Less
Submitted 20 September, 2021; v1 submitted 21 June, 2021;
originally announced June 2021.
-
Intraoperative time out to promote the implementation of the critical view of safety in laparoscopic cholecystectomy: a video-based assessment of 343 procedures
Authors:
Pietro Mascagni,
Maria Rita Rodriguez-Luna,
Takeshi Urade,
Emanuele Felli,
Patrick Pessaux,
Didier Mutter,
Jacques Marescaux,
Guido Costamagna,
Bernard Dallemagne,
Nicolas Padoy
Abstract:
Background: The critical view of safety (CVS) is poorly adopted in surgical practices although it is ubiquitously recommended to prevent major bile duct injuries during laparoscopic cholecystectomy (LC). This study aims to determine whether performing a short intraoperative time out can improve CVS implementation. Methods: Surgeons performing LCs at an academic centre were invited to perform a 5-s…
▽ More
Background: The critical view of safety (CVS) is poorly adopted in surgical practices although it is ubiquitously recommended to prevent major bile duct injuries during laparoscopic cholecystectomy (LC). This study aims to determine whether performing a short intraoperative time out can improve CVS implementation. Methods: Surgeons performing LCs at an academic centre were invited to perform a 5-second long time out to verify CVS before dividing the cystic duct (5-second rule). The primary endpoint was to compare the rate of CVS achievement between LCs performed in the year before and the year after the 5-second rule. The CVS achievement rate was computed using the mediated video-based assessment of two independent reviewers. Clinical outcomes, LC workflows, and postoperative reports were also compared. Results: Three hundred and forty-three (171 before and 172 after the 5-second rule) of the 381 LCs performed over the 2-year study were analysed. After the implementation of the 5-second rule, the rate of CVS achievement increased significantly (15.9 vs 44.1 %; P<0.001) as well as the rate of bailout procedures (8.2 vs 15.7 %; P=0.045), the median time to clip the cystic duct or artery (17:26 [interquartile range: 16:46] vs 23:12 [17:16] minutes; P=0.007), and the rate of postoperative CVS reporting (1.3 vs 28.8 %; P<0.001). Morbidity was comparable (1.75 vs 2.33 % before and after the 5-second rule respectively; P=0.685). Conclusion: Performing a short intraoperative time out improves CVS implementation during LC. Systematic intraoperative cognitive aids should be studied to sustain the uptake of guidelines.
△ Less
Submitted 6 April, 2021;
originally announced April 2021.