Reproducibility analysis of automated deep learning based localisation of mandibular canals on a temporal CBCT dataset
Authors:
Jorma Järnstedt,
Jaakko Sahlsten,
Joel Jaskari,
Kimmo Kaski,
Helena Mehtonen,
Ari Hietanen,
Osku Sundqvist,
Vesa Varjonen,
Vesa Mattila,
Sangsom Prapayasotok,
Sakarat Nalampang
Abstract:
Preoperative radiological identification of mandibular canals is essential for maxillofacial surgery. This study demonstrates the reproducibility of a deep learning system (DLS) by evaluating its localisation performance on 165 heterogeneous cone beam computed tomography (CBCT) scans from 72 patients in comparison to an experienced radiologist's annotations. We evaluated the performance of the DLS…
▽ More
Preoperative radiological identification of mandibular canals is essential for maxillofacial surgery. This study demonstrates the reproducibility of a deep learning system (DLS) by evaluating its localisation performance on 165 heterogeneous cone beam computed tomography (CBCT) scans from 72 patients in comparison to an experienced radiologist's annotations. We evaluated the performance of the DLS using the symmetric mean curve distance (SMCD), the average symmetric surface distance (ASSD), and the Dice similarity coefficient (DSC). The reproducibility of the SMCD was assessed using the within-subject coefficient of repeatability (RC). Three other experts rated the diagnostic validity twice using a 0-4 Likert scale. The reproducibility of the Likert scoring was assessed using the repeatability measure (RM). The RC of SMCD was 0.969 mm, the median (interquartile range) SMCD and ASSD were 0.643 (0.186) mm and 0.351 (0.135) mm, respectively, and the mean (standard deviation) DSC was 0.548 (0.138). The DLS performance was most affected by postoperative changes. The RM of the Likert scoring was 0.923 for the radiologist and 0.877 for the DLS. The mean (standard deviation) Likert score was 3.94 (0.27) for the radiologist and 3.84 (0.65) for the DLS. The DLS demonstrated proficient qualitative and quantitative reproducibility, temporal generalisability, and clinical validity.
△ Less
Submitted 28 April, 2023;
originally announced May 2023.
Comparison of Deep Learning Segmentation and Multigrader-annotated Mandibular Canals of Multicenter CBCT scans
Authors:
Jorma Järnstedt,
Jaakko Sahlsten,
Joel Jaskari,
Kimmo Kaski,
Helena Mehtonen,
Ziyuan Lin,
Ari Hietanen,
Osku Sundqvist,
Vesa Varjonen,
Vesa Mattila,
Sangsom Prapayasotok,
Sakarat Nalampang
Abstract:
Deep learning approach has been demonstrated to automatically segment the bilateral mandibular canals from CBCT scans, yet systematic studies of its clinical and technical validation are scarce. To validate the mandibular canal localization accuracy of a deep learning system (DLS) we trained it with 982 CBCT scans and evaluated using 150 scans of five scanners from clinical workflow patients of Eu…
▽ More
Deep learning approach has been demonstrated to automatically segment the bilateral mandibular canals from CBCT scans, yet systematic studies of its clinical and technical validation are scarce. To validate the mandibular canal localization accuracy of a deep learning system (DLS) we trained it with 982 CBCT scans and evaluated using 150 scans of five scanners from clinical workflow patients of European and Southeast Asian Institutes, annotated by four radiologists. The interobserver variability was compared to the variability between the DLS and the radiologists. In addition, the generalization of DLS to CBCT scans from scanners not used in the training data was examined to evaluate the out-of-distribution generalization capability. The DLS had lower variability to the radiologists than the interobserver variability between them and it was able to generalize to three new devices. For the radiologists' consensus segmentation, used as gold standard, the DLS had a symmetric mean curve distance of 0.39 mm compared to those of the individual radiologists with 0.62 mm, 0.55 mm, 0.47 mm, and 0.42 mm. The DLS showed comparable or slightly better performance in the segmentation of the mandibular canal with the radiologists and generalization capability to new scanners.
△ Less
Submitted 27 May, 2022;
originally announced May 2022.
