-
Soft-tissue Driven Craniomaxillofacial Surgical Planning
Authors:
Xi Fang,
Daeseung Kim,
Xuanang Xu,
Tianshu Kuang,
Nathan Lampen,
Jungwook Lee,
Hannah H. Deng,
Jaime Gateno,
Michael A. K. Liebschner,
James J. Xia,
**kun Yan
Abstract:
In CMF surgery, the planning of bony movement to achieve a desired facial outcome is a challenging task. Current bone driven approaches focus on normalizing the bone with the expectation that the facial appearance will be corrected accordingly. However, due to the complex non-linear relationship between bony structure and facial soft-tissue, such bone-driven methods are insufficient to correct fac…
▽ More
In CMF surgery, the planning of bony movement to achieve a desired facial outcome is a challenging task. Current bone driven approaches focus on normalizing the bone with the expectation that the facial appearance will be corrected accordingly. However, due to the complex non-linear relationship between bony structure and facial soft-tissue, such bone-driven methods are insufficient to correct facial deformities. Despite efforts to simulate facial changes resulting from bony movement, surgical planning still relies on iterative revisions and educated guesses. To address these issues, we propose a soft-tissue driven framework that can automatically create and verify surgical plans. Our framework consists of a bony planner network that estimates the bony movements required to achieve the desired facial outcome and a facial simulator network that can simulate the possible facial changes resulting from the estimated bony movement plans. By combining these two models, we can verify and determine the final bony movement required for planning. The proposed framework was evaluated using a clinical dataset, and our experimental results demonstrate that the soft-tissue driven approach greatly improves the accuracy and efficacy of surgical planning when compared to the conventional bone-driven approach.
△ Less
Submitted 20 July, 2023;
originally announced July 2023.
-
Deep Learning-based Facial Appearance Simulation Driven by Surgically Planned Craniomaxillofacial Bony Movement
Authors:
Xi Fang,
Daeseung Kim,
Xuanang Xu,
Tianshu Kuang,
Hannah H. Deng,
Joshua C. Barber,
Nathan Lampen,
Jaime Gateno,
Michael A. K. Liebschner,
James J. Xia,
**kun Yan
Abstract:
Simulating facial appearance change following bony movement is a critical step in orthognathic surgical planning for patients with jaw deformities. Conventional biomechanics-based methods such as the finite-element method (FEM) are labor intensive and computationally inefficient. Deep learning-based approaches can be promising alternatives due to their high computational efficiency and strong mode…
▽ More
Simulating facial appearance change following bony movement is a critical step in orthognathic surgical planning for patients with jaw deformities. Conventional biomechanics-based methods such as the finite-element method (FEM) are labor intensive and computationally inefficient. Deep learning-based approaches can be promising alternatives due to their high computational efficiency and strong modeling capability. However, the existing deep learning-based method ignores the physical correspondence between facial soft tissue and bony segments and thus is significantly less accurate compared to FEM. In this work, we propose an Attentive Correspondence assisted Movement Transformation network (ACMT-Net) to estimate the facial appearance by transforming the bony movement to facial soft tissue through a point-to-point attentive correspondence matrix. Experimental results on patients with jaw deformity show that our proposed method can achieve comparable facial change prediction accuracy compared with the state-of-the-art FEM-based approach with significantly improved computational efficiency.
△ Less
Submitted 4 October, 2022;
originally announced October 2022.
-
Federated Multi-organ Segmentation with Inconsistent Labels
Authors:
Xuanang Xu,
Hannah H. Deng,
Jaime Gateno,
**kun Yan
Abstract:
Federated learning is an emerging paradigm allowing large-scale decentralized learning without sharing data across different data owners, which helps address the concern of data privacy in medical image analysis. However, the requirement for label consistency across clients by the existing methods largely narrows its application scope. In practice, each clinical site may only annotate certain orga…
▽ More
Federated learning is an emerging paradigm allowing large-scale decentralized learning without sharing data across different data owners, which helps address the concern of data privacy in medical image analysis. However, the requirement for label consistency across clients by the existing methods largely narrows its application scope. In practice, each clinical site may only annotate certain organs of interest with partial or no overlap with other sites. Incorporating such partially labeled data into a unified federation is an unexplored problem with clinical significance and urgency. This work tackles the challenge by using a novel federated multi-encoding U-Net (Fed-MENU) method for multi-organ segmentation. In our method, a multi-encoding U-Net (MENU-Net) is proposed to extract organ-specific features through different encoding sub-networks. Each sub-network can be seen as an expert of a specific organ and trained for that client. Moreover, to encourage the organ-specific features extracted by different sub-networks to be informative and distinctive, we regularize the training of the MENU-Net by designing an auxiliary generic decoder (AGD). Extensive experiments on six public abdominal CT datasets show that our Fed-MENU method can effectively obtain a federated learning model using the partially labeled datasets with superior performance to other models trained by either localized or centralized learning methods. Source code is publicly available at https://github.com/DIAL-RPI/Fed-MENU.
△ Less
Submitted 25 May, 2023; v1 submitted 14 June, 2022;
originally announced June 2022.
-
Federated Cross Learning for Medical Image Segmentation
Authors:
Xuanang Xu,
Hannah H. Deng,
Tianyi Chen,
Tianshu Kuang,
Joshua C. Barber,
Daeseung Kim,
Jaime Gateno,
James J. Xia,
**kun Yan
Abstract:
Federated learning (FL) can collaboratively train deep learning models using isolated patient data owned by different hospitals for various clinical applications, including medical image segmentation. However, a major problem of FL is its performance degradation when dealing with data that are not independently and identically distributed (non-iid), which is often the case in medical images. In th…
▽ More
Federated learning (FL) can collaboratively train deep learning models using isolated patient data owned by different hospitals for various clinical applications, including medical image segmentation. However, a major problem of FL is its performance degradation when dealing with data that are not independently and identically distributed (non-iid), which is often the case in medical images. In this paper, we first conduct a theoretical analysis on the FL algorithm to reveal the problem of model aggregation during training on non-iid data. With the insights gained through the analysis, we propose a simple yet effective method, federated cross learning (FedCross), to tackle this challenging problem. Unlike the conventional FL methods that combine multiple individually trained local models on a server node, our FedCross sequentially trains the global model across different clients in a round-robin manner, and thus the entire training procedure does not involve any model aggregation steps. To further improve its performance to be comparable with the centralized learning method, we combine the FedCross with an ensemble learning mechanism to compose a federated cross ensemble learning (FedCrossEns) method. Finally, we conduct extensive experiments using a set of public datasets. The experimental results show that the proposed FedCross training strategy outperforms the mainstream FL methods on non-iid data. In addition to improving the segmentation performance, our FedCrossEns can further provide a quantitative estimation of the model uncertainty, demonstrating the effectiveness and clinical significance of our designs. Source code is publicly available at https://github.com/DIAL-RPI/FedCross.
△ Less
Submitted 22 May, 2023; v1 submitted 5 April, 2022;
originally announced April 2022.
-
SkullEngine: A Multi-stage CNN Framework for Collaborative CBCT Image Segmentation and Landmark Detection
Authors:
Qin Liu,
Han Deng,
Chunfeng Lian,
Xiaoyang Chen,
Deqiang Xiao,
Lei Ma,
Xu Chen,
Tianshu Kuang,
Jaime Gateno,
Pew-Thian Yap,
James J. Xia
Abstract:
We propose a multi-stage coarse-to-fine CNN-based framework, called SkullEngine, for high-resolution segmentation and large-scale landmark detection through a collaborative, integrated, and scalable JSD model and three segmentation and landmark detection refinement models. We evaluated our framework on a clinical dataset consisting of 170 CBCT/CT images for the task of segmenting 2 bones (midface…
▽ More
We propose a multi-stage coarse-to-fine CNN-based framework, called SkullEngine, for high-resolution segmentation and large-scale landmark detection through a collaborative, integrated, and scalable JSD model and three segmentation and landmark detection refinement models. We evaluated our framework on a clinical dataset consisting of 170 CBCT/CT images for the task of segmenting 2 bones (midface and mandible) and detecting 175 clinically common landmarks on bones, teeth, and soft tissues.
△ Less
Submitted 21 December, 2021; v1 submitted 7 October, 2021;
originally announced October 2021.
-
A Self-Supervised Deep Framework for Reference Bony Shape Estimation in Orthognathic Surgical Planning
Authors:
Deqiang Xiao,
Hannah Deng,
Tianshu Kuang,
Lei Ma,
Qin Liu,
Xu Chen,
Chunfeng Lian,
Yankun Lang,
Daeseung Kim,
Jaime Gateno,
Steve Guofang Shen,
Dinggang Shen,
Pew-Thian Yap,
James J. Xia
Abstract:
Virtual orthognathic surgical planning involves simulating surgical corrections of jaw deformities on 3D facial bony shape models. Due to the lack of necessary guidance, the planning procedure is highly experience-dependent and the planning results are often suboptimal. A reference facial bony shape model representing normal anatomies can provide an objective guidance to improve planning accuracy.…
▽ More
Virtual orthognathic surgical planning involves simulating surgical corrections of jaw deformities on 3D facial bony shape models. Due to the lack of necessary guidance, the planning procedure is highly experience-dependent and the planning results are often suboptimal. A reference facial bony shape model representing normal anatomies can provide an objective guidance to improve planning accuracy. Therefore, we propose a self-supervised deep framework to automatically estimate reference facial bony shape models. Our framework is an end-to-end trainable network, consisting of a simulator and a corrector. In the training stage, the simulator maps jaw deformities of a patient bone to a normal bone to generate a simulated deformed bone. The corrector then restores the simulated deformed bone back to normal. In the inference stage, the trained corrector is applied to generate a patient-specific normal-looking reference bone from a real deformed bone. The proposed framework was evaluated using a clinical dataset and compared with a state-of-the-art method that is based on a supervised point-cloud network. Experimental results show that the estimated shape models given by our approach are clinically acceptable and significantly more accurate than that of the competing method.
△ Less
Submitted 11 September, 2021;
originally announced September 2021.
-
Calculating the Midsagittal Plane for Symmetrical Bilateral Shapes: Applications to Clinical Facial Surgical Planning
Authors:
Aarti Jajoo,
Matthew Nicol,
Jaime Gateno,
Ken-Chung Chen,
Zhen Tang,
Tasadduk Chowdhury,
Jainfu Li,
Steve Goufang Shen,
James J. Xia
Abstract:
It is difficult to estimate the midsagittal plane of human subjects with craniomaxillofacial (CMF) deformities. We have developed a LAndmark GEometric Routine (LAGER), which automatically estimates a midsagittal plane for such subjects. The LAGER algorithm was based on the assumption that the optimal midsagittal plane of a patient with a deformity is the premorbid midsagittal plane of the patient…
▽ More
It is difficult to estimate the midsagittal plane of human subjects with craniomaxillofacial (CMF) deformities. We have developed a LAndmark GEometric Routine (LAGER), which automatically estimates a midsagittal plane for such subjects. The LAGER algorithm was based on the assumption that the optimal midsagittal plane of a patient with a deformity is the premorbid midsagittal plane of the patient (i.e. hypothetically normal without deformity). The LAGER algorithm consists of three steps. The first step quantifies the asymmetry of the landmarks using a Euclidean distance matrix analysis and ranks the landmarks according to their degree of asymmetry. The second step uses a recursive algorithm to drop outlier landmarks. The third step inputs the remaining landmarks into an optimization algorithm to determine an optimal midsaggital plane. We validate LAGER on 20 synthetic models mimicking the skulls of real patients with CMF deformities. The results indicated that all the LAGER algorithm-generated midsagittal planes met clinical criteria. Thus it can be used clinically to determine the midsagittal plane for patients with CMF deformities.
△ Less
Submitted 10 March, 2018;
originally announced March 2018.
