-
Assessing the Performance of Deep Learning for Automated Gleason Grading in Prostate Cancer
Authors:
Dominik Müller,
Philip Meyer,
Lukas Rentschler,
Robin Manz,
Daniel Hieber,
Jonas Bäcker,
Samantha Cramer,
Christoph Wengenmayr,
Bruno Märkl,
Ralf Huss,
Frank Kramer,
Iñaki Soto-Rey,
Johannes Raffler
Abstract:
Prostate cancer is a dominant health concern calling for advanced diagnostic tools. Utilizing digital pathology and artificial intelligence, this study explores the potential of 11 deep neural network architectures for automated Gleason grading in prostate carcinoma focusing on comparing traditional and recent architectures. A standardized image classification pipeline, based on the AUCMEDI framew…
▽ More
Prostate cancer is a dominant health concern calling for advanced diagnostic tools. Utilizing digital pathology and artificial intelligence, this study explores the potential of 11 deep neural network architectures for automated Gleason grading in prostate carcinoma focusing on comparing traditional and recent architectures. A standardized image classification pipeline, based on the AUCMEDI framework, facilitated robust evaluation using an in-house dataset consisting of 34,264 annotated tissue tiles. The results indicated varying sensitivity across architectures, with ConvNeXt demonstrating the strongest performance. Notably, newer architectures achieved superior performance, even though with challenges in differentiating closely related Gleason grades. The ConvNeXt model was capable of learning a balance between complexity and generalizability. Overall, this study lays the groundwork for enhanced Gleason grading systems, potentially improving diagnostic efficiency for prostate cancer.
△ Less
Submitted 25 March, 2024;
originally announced March 2024.
-
DeepGleason: a System for Automated Gleason Grading of Prostate Cancer using Deep Neural Networks
Authors:
Dominik Müller,
Philip Meyer,
Lukas Rentschler,
Robin Manz,
Jonas Bäcker,
Samantha Cramer,
Christoph Wengenmayr,
Bruno Märkl,
Ralf Huss,
Iñaki Soto-Rey,
Johannes Raffler
Abstract:
Advances in digital pathology and artificial intelligence (AI) offer promising opportunities for clinical decision support and enhancing diagnostic workflows. Previous studies already demonstrated AI's potential for automated Gleason grading, but lack state-of-the-art methodology and model reusability. To address this issue, we propose DeepGleason: an open-source deep neural network based image cl…
▽ More
Advances in digital pathology and artificial intelligence (AI) offer promising opportunities for clinical decision support and enhancing diagnostic workflows. Previous studies already demonstrated AI's potential for automated Gleason grading, but lack state-of-the-art methodology and model reusability. To address this issue, we propose DeepGleason: an open-source deep neural network based image classification system for automated Gleason grading using whole-slide histopathology images from prostate tissue sections. Implemented with the standardized AUCMEDI framework, our tool employs a tile-wise classification approach utilizing fine-tuned image preprocessing techniques in combination with a ConvNeXt architecture which was compared to various state-of-the-art architectures. The neural network model was trained and validated on an in-house dataset of 34,264 annotated tiles from 369 prostate carcinoma slides. We demonstrated that DeepGleason is capable of highly accurate and reliable Gleason grading with a macro-averaged F1-score of 0.806, AUC of 0.991, and Accuracy of 0.974. The internal architecture comparison revealed that the ConvNeXt model was superior performance-wise on our dataset to established and other modern architectures like transformers. Furthermore, we were able to outperform the current state-of-the-art in tile-wise fine-classification with a sensitivity and specificity of 0.94 and 0.98 for benign vs malignant detection as well as of 0.91 and 0.75 for Gleason 3 vs Gleason 4 & 5 classification, respectively. Our tool contributes to the wider adoption of AI-based Gleason grading within the research community and paves the way for broader clinical application of deep learning models in digital pathology. DeepGleason is open-source and publicly available for research application in the following Git repository: https://github.com/frankkramer-lab/DeepGleason.
△ Less
Submitted 25 March, 2024;
originally announced March 2024.
-
Reachability Analysis of Neural Networks with Uncertain Parameters
Authors:
Pierre-Jean Meyer
Abstract:
The literature on reachability analysis methods for neural networks currently only focuses on uncertainties on the network's inputs. In this paper, we introduce two new approaches for the reachability analysis of neural networks with additional uncertainties on their internal parameters (weight matrices and bias vectors of each layer), which may open the field of formal methods on neural networks…
▽ More
The literature on reachability analysis methods for neural networks currently only focuses on uncertainties on the network's inputs. In this paper, we introduce two new approaches for the reachability analysis of neural networks with additional uncertainties on their internal parameters (weight matrices and bias vectors of each layer), which may open the field of formal methods on neural networks to new topics, such as safe training or network repair. The first and main method that we propose relies on existing reachability analysis approach based on mixed monotonicity (initially introduced for dynamical systems). The second proposed approach extends the ESIP (Error-based Symbolic Interval Propagation) approach which was first implemented in the verification tool Neurify, and first mentioned in the publication of the tool VeriNet. Although the ESIP approach has been shown to often outperform the mixed-monotonicity reachability analysis in the classical case with uncertainties only on the network's inputs, we show in this paper through numerical simulations that the situation is greatly reversed (in terms of precision, computation time, memory usage, and broader applicability) when dealing with uncertainties on the weights and biases.
△ Less
Submitted 14 March, 2023;
originally announced March 2023.
-
Reachability analysis of neural networks using mixed monotonicity
Authors:
Pierre-Jean Meyer
Abstract:
This paper presents a new reachability analysis approach to compute interval over-approximations of the output set of feedforward neural networks with input uncertainty. We adapt to neural networks an existing mixed-monotonicity method for the reachability analysis of dynamical systems and apply it to each partial network within the main network. This ensures that the intersection of the obtained…
▽ More
This paper presents a new reachability analysis approach to compute interval over-approximations of the output set of feedforward neural networks with input uncertainty. We adapt to neural networks an existing mixed-monotonicity method for the reachability analysis of dynamical systems and apply it to each partial network within the main network. This ensures that the intersection of the obtained results is the tightest interval over-approximation of the output of each layer that can be obtained using mixed-monotonicity on any partial network decomposition. Unlike other tools in the literature focusing on small classes of piecewise-affine or monotone activation functions, the main strength of our approach is its generality: it can handle neural networks with any Lipschitz-continuous activation function. In addition, the simplicity of our framework allows users to very easily add unimplemented activation functions, by simply providing the function, its derivative and the global argmin and argmax of the derivative. Our algorithm is compared to five other interval-based tools (Interval Bound Propagation, ReluVal, Neurify, VeriNet, CROWN) on both existing benchmarks and two sets of small and large randomly generated networks for four activation functions (ReLU, TanH, ELU, SiLU).
△ Less
Submitted 23 June, 2022; v1 submitted 15 November, 2021;
originally announced November 2021.
-
Convolutional neural network based deep-learning architecture for intraprostatic tumour contouring on PSMA PET images in patients with primary prostate cancer
Authors:
Dejan Kostyszyn,
Tobias Fechter,
Nico Bartl,
Anca L. Grosu,
Christian Gratzke,
August Sigle,
Michael Mix,
Juri Ruf,
Thomas F. Fassbender,
Selina Kiefer,
Alisa S. Bettermann,
Nils H. Nicolay,
Simon Spohn,
Maria U. Kramer,
Peter Bronsert,
Hongqian Guo,
Xuefeng Qiu,
Feng Wang,
Christoph Henkenberens,
Rudolf A. Werner,
Dimos Baltas,
Philipp T. Meyer,
Thorsten Derlin,
Mengxia Chen,
Constantinos Zamboglou
Abstract:
Accurate delineation of the intraprostatic gross tumour volume (GTV) is a prerequisite for treatment approaches in patients with primary prostate cancer (PCa). Prostate-specific membrane antigen positron emission tomography (PSMA-PET) may outperform MRI in GTV detection. However, visual GTV delineation underlies interobserver heterogeneity and is time consuming. The aim of this study was to develo…
▽ More
Accurate delineation of the intraprostatic gross tumour volume (GTV) is a prerequisite for treatment approaches in patients with primary prostate cancer (PCa). Prostate-specific membrane antigen positron emission tomography (PSMA-PET) may outperform MRI in GTV detection. However, visual GTV delineation underlies interobserver heterogeneity and is time consuming. The aim of this study was to develop a convolutional neural network (CNN) for automated segmentation of intraprostatic tumour (GTV-CNN) in PSMA-PET.
Methods: The CNN (3D U-Net) was trained on [68Ga]PSMA-PET images of 152 patients from two different institutions and the training labels were generated manually using a validated technique. The CNN was tested on two independent internal (cohort 1: [68Ga]PSMA-PET, n=18 and cohort 2: [18F]PSMA-PET, n=19) and one external (cohort 3: [68Ga]PSMA-PET, n=20) test-datasets. Accordance between manual contours and GTV-CNN was assessed with Dice-Sørensen coefficient (DSC). Sensitivity and specificity were calculated for the two internal test-datasets by using whole-mount histology.
Results: Median DSCs for cohorts 1-3 were 0.84 (range: 0.32-0.95), 0.81 (range: 0.28-0.93) and 0.83 (range: 0.32-0.93), respectively. Sensitivities and specificities for GTV-CNN were comparable with manual expert contours: 0.98 and 0.76 (cohort 1) and 1 and 0.57 (cohort 2), respectively. Computation time was around 6 seconds for a standard dataset.
Conclusion: The application of a CNN for automated contouring of intraprostatic GTV in [68Ga]PSMA- and [18F]PSMA-PET images resulted in a high concordance with expert contours and in high sensitivities and specificities in comparison with histology reference. This robust, accurate and fast technique may be implemented for treatment concepts in primary PCa. The trained model and the study's source code are available in an open source repository.
△ Less
Submitted 7 August, 2020;
originally announced August 2020.
-
MultiXNet: Multiclass Multistage Multimodal Motion Prediction
Authors:
Nemanja Djuric,
Henggang Cui,
Zhaoen Su,
Shangxuan Wu,
Huahua Wang,
Fang-Chieh Chou,
Luisa San Martin,
Song Feng,
Rui Hu,
Yang Xu,
Alyssa Dayan,
Sidney Zhang,
Brian C. Becker,
Gregory P. Meyer,
Carlos Vallespi-Gonzalez,
Carl K. Wellington
Abstract:
One of the critical pieces of the self-driving puzzle is understanding the surroundings of a self-driving vehicle (SDV) and predicting how these surroundings will change in the near future. To address this task we propose MultiXNet, an end-to-end approach for detection and motion prediction based directly on lidar sensor data. This approach builds on prior work by handling multiple classes of traf…
▽ More
One of the critical pieces of the self-driving puzzle is understanding the surroundings of a self-driving vehicle (SDV) and predicting how these surroundings will change in the near future. To address this task we propose MultiXNet, an end-to-end approach for detection and motion prediction based directly on lidar sensor data. This approach builds on prior work by handling multiple classes of traffic actors, adding a jointly trained second-stage trajectory refinement step, and producing a multimodal probability distribution over future actor motion that includes both multiple discrete traffic behaviors and calibrated continuous position uncertainties. The method was evaluated on large-scale, real-world data collected by a fleet of SDVs in several cities, with the results indicating that it outperforms existing state-of-the-art approaches.
△ Less
Submitted 24 May, 2021; v1 submitted 2 June, 2020;
originally announced June 2020.
-
Interval Reachability Analysis using Second-Order Sensitivity
Authors:
Pierre-Jean Meyer,
Murat Arcak
Abstract:
We propose a new approach to compute an interval over-approximation of the finite time reachable set for a large class of nonlinear systems. This approach relies on the notions of sensitivity matrices, which are the partial derivatives representing the variations of the system trajectories in response to variations of the initial states. Using interval arithmetics, we first over-approximate the po…
▽ More
We propose a new approach to compute an interval over-approximation of the finite time reachable set for a large class of nonlinear systems. This approach relies on the notions of sensitivity matrices, which are the partial derivatives representing the variations of the system trajectories in response to variations of the initial states. Using interval arithmetics, we first over-approximate the possible values of the second-order sensitivity at the final time of the reachability problem. Then we exploit these bounds and the evaluation of the first-order sensitivity matrices at a few sampled initial states to obtain an over-approximation of the first-order sensitivity, which is in turn used to over-approximate the reachable set of the initial system. Unlike existing methods relying only on the first-order sensitivity matrix, this new approach provides guaranteed over-approximations of the first-order sensitivity and can also provide such over-approximations with an arbitrary precision by increasing the number of samples.
△ Less
Submitted 4 May, 2020; v1 submitted 21 November, 2019;
originally announced November 2019.
-
Continuous and discrete abstractions for planning, applied to ship docking
Authors:
Pierre-Jean Meyer,
He Yin,
Astrid H. Brodtkorb,
Murat Arcak,
Asgeir J. Sørensen
Abstract:
We propose a hierarchical control framework for the synthesis of correct-by-construction controllers for nonlinear control-affine systems with respect to reach-avoid-stay specifications. We first create a low-dimensional continuous abstraction of the system and use Sum-of-Squares (SOS) programming to obtain a low-level controller ensuring a bounded error between the two models. We then create a di…
▽ More
We propose a hierarchical control framework for the synthesis of correct-by-construction controllers for nonlinear control-affine systems with respect to reach-avoid-stay specifications. We first create a low-dimensional continuous abstraction of the system and use Sum-of-Squares (SOS) programming to obtain a low-level controller ensuring a bounded error between the two models. We then create a discrete abstraction of the continuous abstraction and use formal methods to synthesize a controller satisfying the specifications shrunk by the obtained error bound. Combining both controllers finally solves the main control problem on the initial system. This two-step framework allows the discrete abstraction methods to deal with higher-dimensional systems which may be computationally expensive without the prior continuous abstraction. The main novelty of the proposed SOS continuous abstraction is that it allows the error between abstract and concrete models to explicitly depend on the control input of the abstract model, which offers more freedom in the choice of the continuous abstraction model and provides lower error bounds than when only the states of both models are considered. This approach is illustrated on the docking problem of a marine vessel.
△ Less
Submitted 21 November, 2019;
originally announced November 2019.
-
TIRA: Toolbox for Interval Reachability Analysis
Authors:
Pierre-Jean Meyer,
Alex Devonport,
Murat Arcak
Abstract:
This paper presents TIRA, a Matlab library gathering several methods for the computation of interval over-approximations of the reachable sets for both continuous- and discrete-time nonlinear systems. Unlike other existing tools, the main strength of interval-based reachability analysis is its simplicity and scalability, rather than the accuracy of the over-approximations. The current implementati…
▽ More
This paper presents TIRA, a Matlab library gathering several methods for the computation of interval over-approximations of the reachable sets for both continuous- and discrete-time nonlinear systems. Unlike other existing tools, the main strength of interval-based reachability analysis is its simplicity and scalability, rather than the accuracy of the over-approximations. The current implementation of TIRA contains four reachability methods covering wide classes of nonlinear systems, handled with recent results relying on contraction/growth bounds and monotonicity concepts. TIRA's architecture features a central function working as a hub between the user-defined reachability problem and the library of available reachability methods. This design choice offers increased extensibility of the library, where users can define their own method in a separate function and add the function call in the hub function.
△ Less
Submitted 13 February, 2019;
originally announced February 2019.
-
Robust Control of the Sit-to-Stand Movement for a Powered Lower Limb Orthosis
Authors:
Octavio Narvaez-Aroche,
Pierre-Jean Meyer,
Stephen Tu,
Andrew Packard,
Murat Arcak
Abstract:
The sit-to-stand movement is a key feature for wide adoption of powered lower limb orthoses for patients with complete paraplegia. In this paper we study the control of the ascending phase of the sit-to-stand movement for a minimally actuated powered lower limb orthosis at the hips. First, we generate a pool of finite horizon Linear Quadratic Regulator feedback gains, designed under the assumption…
▽ More
The sit-to-stand movement is a key feature for wide adoption of powered lower limb orthoses for patients with complete paraplegia. In this paper we study the control of the ascending phase of the sit-to-stand movement for a minimally actuated powered lower limb orthosis at the hips. First, we generate a pool of finite horizon Linear Quadratic Regulator feedback gains, designed under the assumption that we can control not only the torque at the hips but also the loads at the shoulders that in reality are applied by the user. Next we conduct reachability analysis to define a performance metric measuring the robustness of each controller against parameter uncertainty, and choose the best controller from the pool with respect to this metric. Then, we replace the presumed shoulder control with an Iterative Learning Control algorithm as a substitute for human experiments. Indeed this algorithm obtains torque and forces at the shoulders that result in successful simulations of the sit-to-stand movement, regardless of parameter uncertainty and factors deliberately introduced to hinder learning. Thus it is reasonable to expect that the superior cognitive skills of real users will enable them to cooperate with the hip torque controller through training.
△ Less
Submitted 16 November, 2018;
originally announced November 2018.
-
Reachability Analysis for Robustness Evaluation of the Sit-To-Stand Movement for Powered Lower Limb Orthoses
Authors:
Octavio Narvaez-Aroche,
Pierre-Jean Meyer,
Murat Arcak,
Andrew Packard
Abstract:
A sensitivity-based approach for computing over-approximations of reachable sets, in the presence of constant parameter uncertainties and a single initial state, is used to analyze a three-link planar robot modeling a Powered Lower Limb Orthosis and its user. Given the nature of the map**s relating the state and parameters of the system with the inputs, and outputs describing the trajectories of…
▽ More
A sensitivity-based approach for computing over-approximations of reachable sets, in the presence of constant parameter uncertainties and a single initial state, is used to analyze a three-link planar robot modeling a Powered Lower Limb Orthosis and its user. Given the nature of the map**s relating the state and parameters of the system with the inputs, and outputs describing the trajectories of its Center of Mass, reachable sets for their respective spaces can be obtained relying on the sensitivities of the nonlinear closed-loop dynamics in the state space. These over-approximations are used to evaluate the worst-case performances of a finite time horizon linear-quadratic regulator (LQR) for controlling the ascending phase of the Sit-To-Stand movement.
△ Less
Submitted 28 May, 2018;
originally announced May 2018.
-
Sampled-data reachability analysis using sensitivity and mixed-monotonicity
Authors:
Pierre-Jean Meyer,
Samuel Coogan,
Murat Arcak
Abstract:
This paper over-approximates the reachable sets of a continuous-time uncertain system using the sensitivity of its trajectories with respect to initial conditions and uncertain parameters. We first prove the equivalence between an existing over-approximation result based on the sign-stability of the sensitivity matrices and a discrete-time approach relying on a mixed-monotonicity property. We then…
▽ More
This paper over-approximates the reachable sets of a continuous-time uncertain system using the sensitivity of its trajectories with respect to initial conditions and uncertain parameters. We first prove the equivalence between an existing over-approximation result based on the sign-stability of the sensitivity matrices and a discrete-time approach relying on a mixed-monotonicity property. We then present a new over-approximation result which scales at worst linearly with the state dimension and is applicable to any continuous-time system with bounded sensitivity. Finally, we provide a simulation-based approach to estimate these bounds through sampling and falsification. The results are illustrated with numerical examples on traffic networks and satellite orbits.
△ Less
Submitted 13 June, 2018; v1 submitted 6 March, 2018;
originally announced March 2018.
-
Hierarchical decomposition of LTL synthesis problem for nonlinear control systems
Authors:
Pierre-Jean Meyer,
Dimos V. Dimarogonas
Abstract:
This paper deals with the control synthesis problem for a continuous nonlinear dynamical system under a Linear Temporal Logic (LTL) formula. The proposed solution is a top-down hierarchical decomposition of the control problem involving three abstraction layers of the problem, iteratively solved from the coarsest to the finest. The LTL planning is first solved on a small transition system only des…
▽ More
This paper deals with the control synthesis problem for a continuous nonlinear dynamical system under a Linear Temporal Logic (LTL) formula. The proposed solution is a top-down hierarchical decomposition of the control problem involving three abstraction layers of the problem, iteratively solved from the coarsest to the finest. The LTL planning is first solved on a small transition system only describing the regions of interest involved in the LTL formula. For each pair of consecutive regions of interest in the resulting accepting path satisfying the LTL formula, a discrete plan is then constructed in the partitioned workspace to connect these two regions while avoiding unsafe regions. Finally, an abstraction refinement approach is applied to synthesize a controller for the dynamical system to follow each discrete plan. The second main contribution, used in the third abstraction layer, is a new monotonicity-based method to over-approximate the finite-time reachable set of any continuously differentiable system. The proposed framework is demonstrated in simulation for a motion planning problem of a mobile robot modeled as a disturbed unicycle.
△ Less
Submitted 5 March, 2019; v1 submitted 16 December, 2017;
originally announced December 2017.
-
Compositional abstraction refinement for control synthesis
Authors:
Pierre-Jean Meyer,
Dimos V. Dimarogonas
Abstract:
This paper presents a compositional approach to specification-guided abstraction refinement for control synthesis of a nonlinear system associated with a method to over-approximate its reachable sets. Given an initial coarse partition of the state space, the control specification is given as a sequence of the cells of this partition to visit at each sampling time. The dynamics are decomposed into…
▽ More
This paper presents a compositional approach to specification-guided abstraction refinement for control synthesis of a nonlinear system associated with a method to over-approximate its reachable sets. Given an initial coarse partition of the state space, the control specification is given as a sequence of the cells of this partition to visit at each sampling time. The dynamics are decomposed into subsystems where some states and inputs are not observed, some states are observed but not controlled and where assume-guarantee obligations are used on the uncontrolled states of each subsystem. A finite abstraction is created for each subsystem through a refinement procedure starting from a coarse partition of the state space, then proceeding backwards on the specification sequence to iteratively split the elements of the partition whose coarseness prevents the satisfaction of the specification. Each refined abstraction is associated with a controller and it is proved that combining these local controllers can enforce the specification on the original system. The efficiency of the proposed approach compared to other abstraction methods is illustrated in a numerical example.
△ Less
Submitted 16 December, 2017;
originally announced December 2017.
-
Quantitative Implementation Strategies for Safety Controllers
Authors:
Philipp J. Meyer,
Matthias Rungger,
Michael Luttenberger,
Javier Esparza,
Majid Zamani
Abstract:
We consider the symbolic controller synthesis approach to enforce safety specifications on perturbed, nonlinear control systems. In general, in each state of the system several control values might be applicable to enforce the safety requirement and in the implementation one has the burden of picking a particular control value out of possibly many. We present a class of implementation strategies t…
▽ More
We consider the symbolic controller synthesis approach to enforce safety specifications on perturbed, nonlinear control systems. In general, in each state of the system several control values might be applicable to enforce the safety requirement and in the implementation one has the burden of picking a particular control value out of possibly many. We present a class of implementation strategies to obtain a controller with certain performance guarantees. This class includes two existing implementation strategies from the literature, based on discounted payoff and mean-payoff games. We unify both approaches by using games characterized by a single discount factor determining the implementation. We evaluate different implementations from our class experimentally on two case studies. We show that the choice of the discount factor has a significant influence on the average long-term costs, and the best performance guarantee for the symbolic model does not result in the best implementation. Comparing the optimal choice of the discount factor here with the previously proposed values, the costs differ by a factor of up to 50. Our approach therefore yields a method to choose systematically a good implementation for safety controllers with quantitative objectives.
△ Less
Submitted 14 December, 2017;
originally announced December 2017.
-
Compositional abstraction and safety synthesis using overlap** symbolic models
Authors:
Pierre-Jean Meyer,
Antoine Girard,
Emmanuel Witrant
Abstract:
In this paper, we develop a compositional approach to abstraction and safety synthesis for a general class of discrete time nonlinear systems. Our approach makes it possible to define a symbolic abstraction by composing a set of symbolic subsystems that are overlap** in the sense that they can share some common state variables. We develop compositional safety synthesis techniques using such over…
▽ More
In this paper, we develop a compositional approach to abstraction and safety synthesis for a general class of discrete time nonlinear systems. Our approach makes it possible to define a symbolic abstraction by composing a set of symbolic subsystems that are overlap** in the sense that they can share some common state variables. We develop compositional safety synthesis techniques using such overlap** symbolic subsystems. Comparisons, in terms of conservativeness and of computational complexity, between abstractions and controllers obtained from different system decompositions are provided. Numerical experiments show that the proposed approach for symbolic control synthesis enables a significant complexity reduction with respect to the centralized approach, while reducing the conservatism with respect to compositional approaches using non-overlap** subsystems.
△ Less
Submitted 19 July, 2017; v1 submitted 24 April, 2017;
originally announced April 2017.