-
Performance Deterioration of Deep Learning Models after Clinical Deployment: A Case Study with Auto-segmentation for Definitive Prostate Cancer Radiotherapy
Authors:
Biling Wang,
Michael Dohopolski,
Ti Bai,
Junjie Wu,
Raquibul Hannan,
Neil Desai,
Aurelie Garant,
Daniel Yang,
Dan Nguyen,
Mu-Han Lin,
Robert Timmerman,
Xinlei Wang,
Steve Jiang
Abstract:
We evaluated the temporal performance of a deep learning (DL) based artificial intelligence (AI) model for auto segmentation in prostate radiotherapy, seeking to correlate its efficacy with changes in clinical landscapes. Our study involved 1328 prostate cancer patients who underwent definitive radiotherapy from January 2006 to August 2022 at the University of Texas Southwestern Medical Center. We…
▽ More
We evaluated the temporal performance of a deep learning (DL) based artificial intelligence (AI) model for auto segmentation in prostate radiotherapy, seeking to correlate its efficacy with changes in clinical landscapes. Our study involved 1328 prostate cancer patients who underwent definitive radiotherapy from January 2006 to August 2022 at the University of Texas Southwestern Medical Center. We trained a UNet based segmentation model on data from 2006 to 2011 and tested it on data from 2012 to 2022 to simulate real world clinical deployment. We measured the model performance using the Dice similarity coefficient (DSC), visualized the trends in contour quality using exponentially weighted moving average (EMA) curves. Additionally, we performed Wilcoxon Rank Sum Test to analyze the differences in DSC distributions across distinct periods, and multiple linear regression to investigate the impact of various clinical factors. The model exhibited peak performance in the initial phase (from 2012 to 2014) for segmenting the prostate, rectum, and bladder. However, we observed a notable decline in performance for the prostate and rectum after 2015, while bladder contour quality remained stable. Key factors that impacted the prostate contour quality included physician contouring styles, the use of various hydrogel spacer, CT scan slice thickness, MRI-guided contouring, and using intravenous (IV) contrast. Rectum contour quality was influenced by factors such as slice thickness, physician contouring styles, and the use of various hydrogel spacers. The bladder contour quality was primarily affected by using IV contrast. This study highlights the challenges in maintaining AI model performance consistency in a dynamic clinical setting. It underscores the need for continuous monitoring and updating of AI models to ensure their ongoing effectiveness and relevance in patient care.
△ Less
Submitted 16 November, 2023; v1 submitted 10 October, 2022;
originally announced October 2022.
-
New vehicle-actuated access algorithms for intersections close to oversaturation
Authors:
Rik W. Timmerman,
Marko A. A. Boon
Abstract:
Optimal traffic-light settings are generally hard to obtain, certainly for actuated access control of an intersection. Typically, computationally expensive (microscopic) simulations or complicated optimization schemes are required to find those optimal settings. Based on recent developments regarding the Fixed-Cycle Traffic-Light (FCTL) queue, we propose a new allocation scheme for actuated access…
▽ More
Optimal traffic-light settings are generally hard to obtain, certainly for actuated access control of an intersection. Typically, computationally expensive (microscopic) simulations or complicated optimization schemes are required to find those optimal settings. Based on recent developments regarding the Fixed-Cycle Traffic-Light (FCTL) queue, we propose a new allocation scheme for actuated access control at isolated intersections. Our scheme allows for very easy computations and an intuitive explanation, which are both based on the Central Limit Theorem, a key tool in probability theory. The desirable properties observed for the FCTL queue can also be achieved in case of actuated access control of an intersection. We show this by means of simulation of the underlying queueing model, because the actuated control defies an exact analysis so far. We also observe the same type of results employing the microscopic traffic simulator SUMO. The intuition and insights obtained might also open up new possibilities for obtaining optimal actuated access control for a network of intersections.
△ Less
Submitted 16 July, 2020; v1 submitted 9 March, 2020;
originally announced April 2020.
-
Platoon Forming Algorithms for Intelligent Street Intersections
Authors:
R. W. Timmerman,
M. A. A. Boon
Abstract:
We study intersection access control for autonomous vehicles. Platoon forming algorithms, which aim to organize individual vehicles in platoons, are very promising. To create those platoons, we slow down vehicles before the actual arrival at the intersection in such a way that each vehicle can traverse the intersection at high speed. This increases the capacity of the intersection significantly, o…
▽ More
We study intersection access control for autonomous vehicles. Platoon forming algorithms, which aim to organize individual vehicles in platoons, are very promising. To create those platoons, we slow down vehicles before the actual arrival at the intersection in such a way that each vehicle can traverse the intersection at high speed. This increases the capacity of the intersection significantly, offering huge potential savings with respect to travel time compared to nowadays traffic.
We propose several new platoon forming algorithms and provide an approximate mean delay analysis for our algorithms. A comparison between the current day practice at intersections (through a case study in SUMO) and our proposed algorithms is provided. Simulation results for fairness are obtained as well, showing that platoon forming algorithms with a low mean delay sometimes are relatively unfair, indicating a potential need for balancing mean delay and fairness.
△ Less
Submitted 24 July, 2019; v1 submitted 2 January, 2019;
originally announced January 2019.
