-
Assessing generalisability of deep learning-based polyp detection and segmentation methods through a computer vision challenge
Authors:
Sharib Ali,
Noha Ghatwary,
Debesh Jha,
Ece Isik-Polat,
Gorkem Polat,
Chen Yang,
Wuyang Li,
Adrian Galdran,
Miguel-Ángel González Ballester,
Vajira Thambawita,
Steven Hicks,
Sahadev Poudel,
Sang-Woong Lee,
Ziyi **,
Tianyuan Gan,
ChengHui Yu,
JiangPeng Yan,
Doyeob Yeo,
Hyunseok Lee,
Nikhil Kumar Tomar,
Mahmood Haithmi,
Amr Ahmed,
Michael A. Riegler,
Christian Daul,
Pål Halvorsen
, et al. (7 additional authors not shown)
Abstract:
Polyps are well-known cancer precursors identified by colonoscopy. However, variability in their size, location, and surface largely affect identification, localisation, and characterisation. Moreover, colonoscopic surveillance and removal of polyps (referred to as polypectomy ) are highly operator-dependent procedures. There exist a high missed detection rate and incomplete removal of colonic pol…
▽ More
Polyps are well-known cancer precursors identified by colonoscopy. However, variability in their size, location, and surface largely affect identification, localisation, and characterisation. Moreover, colonoscopic surveillance and removal of polyps (referred to as polypectomy ) are highly operator-dependent procedures. There exist a high missed detection rate and incomplete removal of colonic polyps due to their variable nature, the difficulties to delineate the abnormality, the high recurrence rates, and the anatomical topography of the colon. There have been several developments in realising automated methods for both detection and segmentation of these polyps using machine learning. However, the major drawback in most of these methods is their ability to generalise to out-of-sample unseen datasets that come from different centres, modalities and acquisition systems. To test this hypothesis rigorously we curated a multi-centre and multi-population dataset acquired from multiple colonoscopy systems and challenged teams comprising machine learning experts to develop robust automated detection and segmentation methods as part of our crowd-sourcing Endoscopic computer vision challenge (EndoCV) 2021. In this paper, we analyse the detection results of the four top (among seven) teams and the segmentation results of the five top teams (among 16). Our analyses demonstrate that the top-ranking teams concentrated on accuracy (i.e., accuracy > 80% on overall Dice score on different validation sets) over real-time performance required for clinical applicability. We further dissect the methods and provide an experiment-based hypothesis that reveals the need for improved generalisability to tackle diversity present in multi-centre datasets.
△ Less
Submitted 24 February, 2022;
originally announced February 2022.
-
A multi-centre polyp detection and segmentation dataset for generalisability assessment
Authors:
Sharib Ali,
Debesh Jha,
Noha Ghatwary,
Stefano Realdon,
Renato Cannizzaro,
Osama E. Salem,
Dominique Lamarque,
Christian Daul,
Michael A. Riegler,
Kim V. Anonsen,
Andreas Petlund,
Pål Halvorsen,
Jens Rittscher,
Thomas de Lange,
James E. East
Abstract:
Polyps in the colon are widely known cancer precursors identified by colonoscopy. Whilst most polyps are benign, the polyp's number, size and surface structure are linked to the risk of colon cancer. Several methods have been developed to automate polyp detection and segmentation. However, the main issue is that they are not tested rigorously on a large multicentre purpose-built dataset, one reaso…
▽ More
Polyps in the colon are widely known cancer precursors identified by colonoscopy. Whilst most polyps are benign, the polyp's number, size and surface structure are linked to the risk of colon cancer. Several methods have been developed to automate polyp detection and segmentation. However, the main issue is that they are not tested rigorously on a large multicentre purpose-built dataset, one reason being the lack of a comprehensive public dataset. As a result, the developed methods may not generalise to different population datasets. To this extent, we have curated a dataset from six unique centres incorporating more than 300 patients. The dataset includes both single frame and sequence data with 3762 annotated polyp labels with precise delineation of polyp boundaries verified by six senior gastroenterologists. To our knowledge, this is the most comprehensive detection and pixel-level segmentation dataset (referred to as \textit{PolypGen}) curated by a team of computational scientists and expert gastroenterologists. The paper provides insight into data construction and annotation strategies, quality assurance, and technical validation. Our dataset can be downloaded from \url{ https://doi.org/10.7303/syn26376615}.
△ Less
Submitted 19 May, 2023; v1 submitted 8 June, 2021;
originally announced June 2021.
-
Deep learning for detection and segmentation of artefact and disease instances in gastrointestinal endoscopy
Authors:
Sharib Ali,
Mariia Dmitrieva,
Noha Ghatwary,
Sophia Bano,
Gorkem Polat,
Alptekin Temizel,
Adrian Krenzer,
Amar Hekalo,
Yun Bo Guo,
Bogdan Matuszewski,
Mourad Gridach,
Irina Voiculescu,
Vishnusai Yoganand,
Arnav Chavan,
Aryan Raj,
Nhan T. Nguyen,
Dat Q. Tran,
Le Duy Huynh,
Nicolas Boutry,
Shahadate Rezvy,
Haijian Chen,
Yoon Ho Choi,
Anand Subramanian,
Velmurugan Balasubramanian,
Xiaohong W. Gao
, et al. (12 additional authors not shown)
Abstract:
The Endoscopy Computer Vision Challenge (EndoCV) is a crowd-sourcing initiative to address eminent problems in develo** reliable computer aided detection and diagnosis endoscopy systems and suggest a pathway for clinical translation of technologies. Whilst endoscopy is a widely used diagnostic and treatment tool for hollow-organs, there are several core challenges often faced by endoscopists, ma…
▽ More
The Endoscopy Computer Vision Challenge (EndoCV) is a crowd-sourcing initiative to address eminent problems in develo** reliable computer aided detection and diagnosis endoscopy systems and suggest a pathway for clinical translation of technologies. Whilst endoscopy is a widely used diagnostic and treatment tool for hollow-organs, there are several core challenges often faced by endoscopists, mainly: 1) presence of multi-class artefacts that hinder their visual interpretation, and 2) difficulty in identifying subtle precancerous precursors and cancer abnormalities. Artefacts often affect the robustness of deep learning methods applied to the gastrointestinal tract organs as they can be confused with tissue of interest. EndoCV2020 challenges are designed to address research questions in these remits. In this paper, we present a summary of methods developed by the top 17 teams and provide an objective comparison of state-of-the-art methods and methods designed by the participants for two sub-challenges: i) artefact detection and segmentation (EAD2020), and ii) disease detection and segmentation (EDD2020). Multi-center, multi-organ, multi-class, and multi-modal clinical endoscopy datasets were compiled for both EAD2020 and EDD2020 sub-challenges. The out-of-sample generalization ability of detection algorithms was also evaluated. Whilst most teams focused on accuracy improvements, only a few methods hold credibility for clinical usability. The best performing teams provided solutions to tackle class imbalance, and variabilities in size, origin, modality and occurrences by exploring data augmentation, data fusion, and optimal class thresholding techniques.
△ Less
Submitted 17 February, 2021; v1 submitted 12 October, 2020;
originally announced October 2020.
-
Endoscopy disease detection challenge 2020
Authors:
Sharib Ali,
Noha Ghatwary,
Barbara Braden,
Dominique Lamarque,
Adam Bailey,
Stefano Realdon,
Renato Cannizzaro,
Jens Rittscher,
Christian Daul,
James East
Abstract:
Whilst many technologies are built around endoscopy, there is a need to have a comprehensive dataset collected from multiple centers to address the generalization issues with most deep learning frameworks. What could be more important than disease detection and localization? Through our extensive network of clinical and computational experts, we have collected, curated and annotated gastrointestin…
▽ More
Whilst many technologies are built around endoscopy, there is a need to have a comprehensive dataset collected from multiple centers to address the generalization issues with most deep learning frameworks. What could be more important than disease detection and localization? Through our extensive network of clinical and computational experts, we have collected, curated and annotated gastrointestinal endoscopy video frames. We have released this dataset and have launched disease detection and segmentation challenge EDD2020 https://edd2020.grand-challenge.org to address the limitations and explore new directions. EDD2020 is a crowd sourcing initiative to test the feasibility of recent deep learning methods and to promote research for building robust technologies. In this paper, we provide an overview of the EDD2020 dataset, challenge tasks, evaluation strategies and a short summary of results on test data. A detailed paper will be drafted after the challenge workshop with more detailed analysis of the results.
△ Less
Submitted 6 March, 2020;
originally announced March 2020.
-
Endoscopy artifact detection (EAD 2019) challenge dataset
Authors:
Sharib Ali,
Felix Zhou,
Christian Daul,
Barbara Braden,
Adam Bailey,
Stefano Realdon,
James East,
Georges Wagnières,
Victor Loschenov,
Enrico Grisan,
Walter Blondel,
Jens Rittscher
Abstract:
Endoscopic artifacts are a core challenge in facilitating the diagnosis and treatment of diseases in hollow organs. Precise detection of specific artifacts like pixel saturations, motion blur, specular reflections, bubbles and debris is essential for high-quality frame restoration and is crucial for realizing reliable computer-assisted tools for improved patient care. At present most videos in end…
▽ More
Endoscopic artifacts are a core challenge in facilitating the diagnosis and treatment of diseases in hollow organs. Precise detection of specific artifacts like pixel saturations, motion blur, specular reflections, bubbles and debris is essential for high-quality frame restoration and is crucial for realizing reliable computer-assisted tools for improved patient care. At present most videos in endoscopy are currently not analyzed due to the abundant presence of multi-class artifacts in video frames. Through the endoscopic artifact detection (EAD 2019) challenge, we address this key bottleneck problem by solving the accurate identification and localization of endoscopic frame artifacts to enable further key quantitative analysis of unusable video frames such as mosaicking and 3D reconstruction which is crucial for delivering improved patient care. This paper summarizes the challenge tasks and describes the dataset and evaluation criteria established in the EAD 2019 challenge.
△ Less
Submitted 8 May, 2019;
originally announced May 2019.
