-
Future Aware Safe Active Learning of Time Varying Systems using Gaussian Processes
Authors:
Markus Lange-Hegermann,
Christoph Zimmer
Abstract:
Experimental exploration of high-cost systems with safety constraints, common in engineering applications, is a challenging endeavor. Data-driven models offer a promising solution, but acquiring the requisite data remains expensive and is potentially unsafe. Safe active learning techniques prove essential, enabling the learning of high-quality models with minimal expensive data points and high saf…
▽ More
Experimental exploration of high-cost systems with safety constraints, common in engineering applications, is a challenging endeavor. Data-driven models offer a promising solution, but acquiring the requisite data remains expensive and is potentially unsafe. Safe active learning techniques prove essential, enabling the learning of high-quality models with minimal expensive data points and high safety. This paper introduces a safe active learning framework tailored for time-varying systems, addressing drift, seasonal changes, and complexities due to dynamic behavior. The proposed Time-aware Integrated Mean Squared Prediction Error (T-IMSPE) method minimizes posterior variance over current and future states, optimizing information gathering also in the time domain. Empirical results highlight T-IMSPE's advantages in model quality through toy and real-world examples. State of the art Gaussian processes are compatible with T-IMSPE. Our theoretical contributions include a clear delineation which Gaussian process kernels, domains, and weighting measures are suitable for T-IMSPE and even beyond for its non-time aware predecessor IMSPE.
△ Less
Submitted 17 May, 2024;
originally announced May 2024.
-
Efficiently Computable Safety Bounds for Gaussian Processes in Active Learning
Authors:
Jörn Tebbe,
Christoph Zimmer,
Ansgar Steland,
Markus Lange-Hegermann,
Fabian Mies
Abstract:
Active learning of physical systems must commonly respect practical safety constraints, which restricts the exploration of the design space. Gaussian Processes (GPs) and their calibrated uncertainty estimations are widely used for this purpose. In many technical applications the design space is explored via continuous trajectories, along which the safety needs to be assessed. This is particularly…
▽ More
Active learning of physical systems must commonly respect practical safety constraints, which restricts the exploration of the design space. Gaussian Processes (GPs) and their calibrated uncertainty estimations are widely used for this purpose. In many technical applications the design space is explored via continuous trajectories, along which the safety needs to be assessed. This is particularly challenging for strict safety requirements in GP methods, as it employs computationally expensive Monte-Carlo sampling of high quantiles. We address these challenges by providing provable safety bounds based on the adaptively sampled median of the supremum of the posterior GP. Our method significantly reduces the number of samples required for estimating high safety probabilities, resulting in faster evaluation without sacrificing accuracy and exploration speed. The effectiveness of our safe active learning approach is demonstrated through extensive simulations and validated using a real-world engine example.
△ Less
Submitted 15 April, 2024; v1 submitted 28 February, 2024;
originally announced February 2024.
-
Global Safe Sequential Learning via Efficient Knowledge Transfer
Authors:
Cen-You Li,
Olaf Duennbier,
Marc Toussaint,
Barbara Rakitsch,
Christoph Zimmer
Abstract:
Sequential learning methods such as active learning and Bayesian optimization select the most informative data to learn about a task. In many medical or engineering applications, the data selection is constrained by a priori unknown safety conditions. A promissing line of safe learning methods utilize Gaussian processes (GPs) to model the safety probability and perform data selection in areas with…
▽ More
Sequential learning methods such as active learning and Bayesian optimization select the most informative data to learn about a task. In many medical or engineering applications, the data selection is constrained by a priori unknown safety conditions. A promissing line of safe learning methods utilize Gaussian processes (GPs) to model the safety probability and perform data selection in areas with high safety confidence. However, accurate safety modeling requires prior knowledge or consumes data. In addition, the safety confidence centers around the given observations which leads to local exploration. As transferable source knowledge is often available in safety critical experiments, we propose to consider transfer safe sequential learning to accelerate the learning of safety. We further consider a pre-computation of source components to reduce the additional computational load that is introduced by incorporating source data. In this paper, we theoretically analyze the maximum explorable safe regions of conventional safe learning methods. Furthermore, we empirically demonstrate that our approach 1) learns a task with lower data consumption, 2) globally explores multiple disjoint safe regions under guidance of the source knowledge, and 3) operates with computation comparable to conventional safe learning methods.
△ Less
Submitted 15 April, 2024; v1 submitted 22 February, 2024;
originally announced February 2024.
-
Safe Active Learning for Time-Series Modeling with Gaussian Processes
Authors:
Christoph Zimmer,
Mona Meister,
Duy Nguyen-Tuong
Abstract:
Learning time-series models is useful for many applications, such as simulation and forecasting. In this study, we consider the problem of actively learning time-series models while taking given safety constraints into account. For time-series modeling we employ a Gaussian process with a nonlinear exogenous input structure. The proposed approach generates data appropriate for time series model lea…
▽ More
Learning time-series models is useful for many applications, such as simulation and forecasting. In this study, we consider the problem of actively learning time-series models while taking given safety constraints into account. For time-series modeling we employ a Gaussian process with a nonlinear exogenous input structure. The proposed approach generates data appropriate for time series model learning, i.e. input and output trajectories, by dynamically exploring the input space. The approach parametrizes the input trajectory as consecutive trajectory sections, which are determined stepwise given safety requirements and past observations. We analyze the proposed algorithm and evaluate it empirically on a technical application. The results show the effectiveness of our approach in a realistic technical use case.
△ Less
Submitted 9 February, 2024;
originally announced February 2024.
-
Amortized Inference for Gaussian Process Hyperparameters of Structured Kernels
Authors:
Matthias Bitzer,
Mona Meister,
Christoph Zimmer
Abstract:
Learning the kernel parameters for Gaussian processes is often the computational bottleneck in applications such as online learning, Bayesian optimization, or active learning. Amortizing parameter inference over different datasets is a promising approach to dramatically speed up training time. However, existing methods restrict the amortized inference procedure to a fixed kernel structure. The amo…
▽ More
Learning the kernel parameters for Gaussian processes is often the computational bottleneck in applications such as online learning, Bayesian optimization, or active learning. Amortizing parameter inference over different datasets is a promising approach to dramatically speed up training time. However, existing methods restrict the amortized inference procedure to a fixed kernel structure. The amortization network must be redesigned manually and trained again in case a different kernel is employed, which leads to a large overhead in design time and training time. We propose amortizing kernel parameter inference over a complete kernel-structure-family rather than a fixed kernel structure. We do that via defining an amortization network over pairs of datasets and kernel structures. This enables fast kernel inference for each element in the kernel family without retraining the amortization network. As a by-product, our amortization network is able to do fast ensembling over kernel structures. In our experiments, we show drastically reduced inference time combined with competitive test performance for a large set of kernels and datasets.
△ Less
Submitted 16 June, 2023;
originally announced June 2023.
-
Hierarchical-Hyperplane Kernels for Actively Learning Gaussian Process Models of Nonstationary Systems
Authors:
Matthias Bitzer,
Mona Meister,
Christoph Zimmer
Abstract:
Learning precise surrogate models of complex computer simulations and physical machines often require long-lasting or expensive experiments. Furthermore, the modeled physical dependencies exhibit nonlinear and nonstationary behavior. Machine learning methods that are used to produce the surrogate model should therefore address these problems by providing a scheme to keep the number of queries smal…
▽ More
Learning precise surrogate models of complex computer simulations and physical machines often require long-lasting or expensive experiments. Furthermore, the modeled physical dependencies exhibit nonlinear and nonstationary behavior. Machine learning methods that are used to produce the surrogate model should therefore address these problems by providing a scheme to keep the number of queries small, e.g. by using active learning and be able to capture the nonlinear and nonstationary properties of the system. One way of modeling the nonstationarity is to induce input-partitioning, a principle that has proven to be advantageous in active learning for Gaussian processes. However, these methods either assume a known partitioning, need to introduce complex sampling schemes or rely on very simple geometries. In this work, we present a simple, yet powerful kernel family that incorporates a partitioning that: i) is learnable via gradient-based methods, ii) uses a geometry that is more flexible than previous ones, while still being applicable in the low data regime. Thus, it provides a good prior for active learning procedures. We empirically demonstrate excellent performance on various active learning tasks.
△ Less
Submitted 17 March, 2023;
originally announced March 2023.
-
Approaching Peak Ground Truth
Authors:
Florian Kofler,
Johannes Wahle,
Ivan Ezhov,
Sophia Wagner,
Rami Al-Maskari,
Emilia Gryska,
Mihail Todorov,
Christina Bukas,
Felix Meissen,
Tingying Peng,
Ali Ertürk,
Daniel Rueckert,
Rolf Heckemann,
Jan Kirschke,
Claus Zimmer,
Benedikt Wiestler,
Bjoern Menze,
Marie Piraud
Abstract:
Machine learning models are typically evaluated by computing similarity with reference annotations and trained by maximizing similarity with such. Especially in the biomedical domain, annotations are subjective and suffer from low inter- and intra-rater reliability. Since annotations only reflect one interpretation of the real world, this can lead to sub-optimal predictions even though the model a…
▽ More
Machine learning models are typically evaluated by computing similarity with reference annotations and trained by maximizing similarity with such. Especially in the biomedical domain, annotations are subjective and suffer from low inter- and intra-rater reliability. Since annotations only reflect one interpretation of the real world, this can lead to sub-optimal predictions even though the model achieves high similarity scores. Here, the theoretical concept of PGT is introduced. PGT marks the point beyond which an increase in similarity with the \emph{reference annotation} stops translating to better RWMP. Additionally, a quantitative technique to approximate PGT by computing inter- and intra-rater reliability is proposed. Finally, four categories of PGT-aware strategies to evaluate and improve model performance are reviewed.
△ Less
Submitted 18 March, 2023; v1 submitted 31 December, 2022;
originally announced January 2023.
-
Structural Kernel Search via Bayesian Optimization and Symbolical Optimal Transport
Authors:
Matthias Bitzer,
Mona Meister,
Christoph Zimmer
Abstract:
Despite recent advances in automated machine learning, model selection is still a complex and computationally intensive process. For Gaussian processes (GPs), selecting the kernel is a crucial task, often done manually by the expert. Additionally, evaluating the model selection criteria for Gaussian processes typically scales cubically in the sample size, rendering kernel search particularly compu…
▽ More
Despite recent advances in automated machine learning, model selection is still a complex and computationally intensive process. For Gaussian processes (GPs), selecting the kernel is a crucial task, often done manually by the expert. Additionally, evaluating the model selection criteria for Gaussian processes typically scales cubically in the sample size, rendering kernel search particularly computationally expensive. We propose a novel, efficient search method through a general, structured kernel space. Previous methods solved this task via Bayesian optimization and relied on measuring the distance between GP's directly in function space to construct a kernel-kernel. We present an alternative approach by defining a kernel-kernel over the symbolic representation of the statistical hypothesis that is associated with a kernel. We empirically show that this leads to a computationally more efficient way of searching through a discrete kernel space.
△ Less
Submitted 21 October, 2022;
originally announced October 2022.
-
ISLES 2022: A multi-center magnetic resonance imaging stroke lesion segmentation dataset
Authors:
Moritz Roman Hernandez Petzsche,
Ezequiel de la Rosa,
Uta Hanning,
Roland Wiest,
Waldo Enrique Valenzuela Pinilla,
Mauricio Reyes,
Maria Ines Meyer,
Sook-Lei Liew,
Florian Kofler,
Ivan Ezhov,
David Robben,
Alexander Hutton,
Tassilo Friedrich,
Teresa Zarth,
Johannes Bürkle,
The Anh Baran,
Bjoern Menze,
Gabriel Broocks,
Lukas Meyer,
Claus Zimmer,
Tobias Boeckh-Behrens,
Maria Berndt,
Benno Ikenberg,
Benedikt Wiestler,
Jan S. Kirschke
Abstract:
Magnetic resonance imaging (MRI) is a central modality for stroke imaging. It is used upon patient admission to make treatment decisions such as selecting patients for intravenous thrombolysis or endovascular therapy. MRI is later used in the duration of hospital stay to predict outcome by visualizing infarct core size and location. Furthermore, it may be used to characterize stroke etiology, e.g.…
▽ More
Magnetic resonance imaging (MRI) is a central modality for stroke imaging. It is used upon patient admission to make treatment decisions such as selecting patients for intravenous thrombolysis or endovascular therapy. MRI is later used in the duration of hospital stay to predict outcome by visualizing infarct core size and location. Furthermore, it may be used to characterize stroke etiology, e.g. differentiation between (cardio)-embolic and non-embolic stroke. Computer based automated medical image processing is increasingly finding its way into clinical routine. Previous iterations of the Ischemic Stroke Lesion Segmentation (ISLES) challenge have aided in the generation of identifying benchmark methods for acute and sub-acute ischemic stroke lesion segmentation. Here we introduce an expert-annotated, multicenter MRI dataset for segmentation of acute to subacute stroke lesions. This dataset comprises 400 multi-vendor MRI cases with high variability in stroke lesion size, quantity and location. It is split into a training dataset of n=250 and a test dataset of n=150. All training data will be made publicly available. The test dataset will be used for model validation only and will not be released to the public. This dataset serves as the foundation of the ISLES 2022 challenge with the goal of finding algorithmic methods to enable the development and benchmarking of robust and accurate segmentation algorithms for ischemic stroke.
△ Less
Submitted 14 June, 2022;
originally announced June 2022.
-
Deep Quality Estimation: Creating Surrogate Models for Human Quality Ratings
Authors:
Florian Kofler,
Ivan Ezhov,
Lucas Fidon,
Izabela Horvath,
Ezequiel de la Rosa,
John LaMaster,
Hongwei Li,
Tom Finck,
Suprosanna Shit,
Johannes Paetzold,
Spyridon Bakas,
Marie Piraud,
Jan Kirschke,
Tom Vercauteren,
Claus Zimmer,
Benedikt Wiestler,
Bjoern Menze
Abstract:
Human ratings are abstract representations of segmentation quality. To approximate human quality ratings on scarce expert data, we train surrogate quality estimation models. We evaluate on a complex multi-class segmentation problem, specifically glioma segmentation, following the BraTS annotation protocol. The training data features quality ratings from 15 expert neuroradiologists on a scale rangi…
▽ More
Human ratings are abstract representations of segmentation quality. To approximate human quality ratings on scarce expert data, we train surrogate quality estimation models. We evaluate on a complex multi-class segmentation problem, specifically glioma segmentation, following the BraTS annotation protocol. The training data features quality ratings from 15 expert neuroradiologists on a scale ranging from 1 to 6 stars for various computer-generated and manual 3D annotations. Even though the networks operate on 2D images and with scarce training data, we can approximate segmentation quality within a margin of error comparable to human intra-rater reliability. Segmentation quality prediction has broad applications. While an understanding of segmentation quality is imperative for successful clinical translation of automatic segmentation quality algorithms, it can play an essential role in training new segmentation models. Due to the split-second inference times, it can be directly applied within a loss function or as a fully-automatic dataset curation mechanism in a federated learning setting.
△ Less
Submitted 30 August, 2022; v1 submitted 17 May, 2022;
originally announced May 2022.
-
blob loss: instance imbalance aware loss functions for semantic segmentation
Authors:
Florian Kofler,
Suprosanna Shit,
Ivan Ezhov,
Lucas Fidon,
Izabela Horvath,
Rami Al-Maskari,
Hongwei Li,
Harsharan Bhatia,
Timo Loehr,
Marie Piraud,
Ali Erturk,
Jan Kirschke,
Jan C. Peeken,
Tom Vercauteren,
Claus Zimmer,
Benedikt Wiestler,
Bjoern Menze
Abstract:
Deep convolutional neural networks (CNN) have proven to be remarkably effective in semantic segmentation tasks. Most popular loss functions were introduced targeting improved volumetric scores, such as the Dice coefficient (DSC). By design, DSC can tackle class imbalance, however, it does not recognize instance imbalance within a class. As a result, a large foreground instance can dominate minor i…
▽ More
Deep convolutional neural networks (CNN) have proven to be remarkably effective in semantic segmentation tasks. Most popular loss functions were introduced targeting improved volumetric scores, such as the Dice coefficient (DSC). By design, DSC can tackle class imbalance, however, it does not recognize instance imbalance within a class. As a result, a large foreground instance can dominate minor instances and still produce a satisfactory DSC. Nevertheless, detecting tiny instances is crucial for many applications, such as disease monitoring. For example, it is imperative to locate and surveil small-scale lesions in the follow-up of multiple sclerosis patients. We propose a novel family of loss functions, \emph{blob loss}, primarily aimed at maximizing instance-level detection metrics, such as F1 score and sensitivity. \emph{Blob loss} is designed for semantic segmentation problems where detecting multiple instances matters. We extensively evaluate a DSC-based \emph{blob loss} in five complex 3D semantic segmentation tasks featuring pronounced instance heterogeneity in terms of texture and morphology. Compared to soft Dice loss, we achieve 5% improvement for MS lesions, 3% improvement for liver tumor, and an average 2% improvement for microscopy segmentation tasks considering F1 score.
△ Less
Submitted 6 June, 2023; v1 submitted 17 May, 2022;
originally announced May 2022.
-
Safe Active Learning for Multi-Output Gaussian Processes
Authors:
Cen-You Li,
Barbara Rakitsch,
Christoph Zimmer
Abstract:
Multi-output regression problems are commonly encountered in science and engineering. In particular, multi-output Gaussian processes have been emerged as a promising tool for modeling these complex systems since they can exploit the inherent correlations and provide reliable uncertainty estimates. In many applications, however, acquiring the data is expensive and safety concerns might arise (e.g.…
▽ More
Multi-output regression problems are commonly encountered in science and engineering. In particular, multi-output Gaussian processes have been emerged as a promising tool for modeling these complex systems since they can exploit the inherent correlations and provide reliable uncertainty estimates. In many applications, however, acquiring the data is expensive and safety concerns might arise (e.g. robotics, engineering). We propose a safe active learning approach for multi-output Gaussian process regression. This approach queries the most informative data or output taking the relatedness between the regressors and safety constraints into account. We prove the effectiveness of our approach by providing theoretical analysis and by demonstrating empirical results on simulated datasets and on a real-world engineering dataset. On all datasets, our approach shows improved convergence compared to its competitors.
△ Less
Submitted 28 March, 2022;
originally announced March 2022.
-
A Deep Learning Approach to Predicting Collateral Flow in Stroke Patients Using Radiomic Features from Perfusion Images
Authors:
Giles Tetteh,
Fernando Navarro,
Johannes Paetzold,
Jan Kirschke,
Claus Zimmer,
Bjoern H. Menze
Abstract:
Collateral circulation results from specialized anastomotic channels which are capable of providing oxygenated blood to regions with compromised blood flow caused by ischemic injuries. The quality of collateral circulation has been established as a key factor in determining the likelihood of a favorable clinical outcome and goes a long way to determine the choice of stroke care model - that is the…
▽ More
Collateral circulation results from specialized anastomotic channels which are capable of providing oxygenated blood to regions with compromised blood flow caused by ischemic injuries. The quality of collateral circulation has been established as a key factor in determining the likelihood of a favorable clinical outcome and goes a long way to determine the choice of stroke care model - that is the decision to transport or treat eligible patients immediately.
Though there exist several imaging methods and grading criteria for quantifying collateral blood flow, the actual grading is mostly done through manual inspection of the acquired images. This approach is associated with a number of challenges. First, it is time-consuming - the clinician needs to scan through several slices of images to ascertain the region of interest before deciding on what severity grade to assign to a patient. Second, there is a high tendency for bias and inconsistency in the final grade assigned to a patient depending on the experience level of the clinician.
We present a deep learning approach to predicting collateral flow grading in stroke patients based on radiomic features extracted from MR perfusion data. First, we formulate a region of interest detection task as a reinforcement learning problem and train a deep learning network to automatically detect the occluded region within the 3D MR perfusion volumes. Second, we extract radiomic features from the obtained region of interest through local image descriptors and denoising auto-encoders. Finally, we apply a convolutional neural network and other machine learning classifiers to the extracted radiomic features to automatically predict the collateral flow grading of the given patient volume as one of three severity classes - no flow (0), moderate flow (1), and good flow (2)...
△ Less
Submitted 24 October, 2021;
originally announced October 2021.
-
Active Learning in Gaussian Process State Space Model
Authors:
Hon Sum Alec Yu,
Dingling Yao,
Christoph Zimmer,
Marc Toussaint,
Duy Nguyen-Tuong
Abstract:
We investigate active learning in Gaussian Process state-space models (GPSSM). Our problem is to actively steer the system through latent states by determining its inputs such that the underlying dynamics can be optimally learned by a GPSSM. In order that the most informative inputs are selected, we employ mutual information as our active learning criterion. In particular, we present two approache…
▽ More
We investigate active learning in Gaussian Process state-space models (GPSSM). Our problem is to actively steer the system through latent states by determining its inputs such that the underlying dynamics can be optimally learned by a GPSSM. In order that the most informative inputs are selected, we employ mutual information as our active learning criterion. In particular, we present two approaches for the approximation of mutual information for the GPSSM given latent states. The proposed approaches are evaluated in several physical systems where we actively learn the underlying non-linear dynamics represented by the state-space model.
△ Less
Submitted 30 July, 2021;
originally announced August 2021.
-
A Computed Tomography Vertebral Segmentation Dataset with Anatomical Variations and Multi-Vendor Scanner Data
Authors:
Hans Liebl,
David Schinz,
Anjany Sekuboyina,
Luca Malagutti,
Maximilian T. Löffler,
Amirhossein Bayat,
Malek El Husseini,
Giles Tetteh,
Katharina Grau,
Eva Niederreiter,
Thomas Baum,
Benedikt Wiestler,
Bjoern Menze,
Rickmer Braren,
Claus Zimmer,
Jan S. Kirschke
Abstract:
With the advent of deep learning algorithms, fully automated radiological image analysis is within reach. In spine imaging, several atlas- and shape-based as well as deep learning segmentation algorithms have been proposed, allowing for subsequent automated analysis of morphology and pathology. The first Large Scale Vertebrae Segmentation Challenge (VerSe 2019) showed that these perform well on no…
▽ More
With the advent of deep learning algorithms, fully automated radiological image analysis is within reach. In spine imaging, several atlas- and shape-based as well as deep learning segmentation algorithms have been proposed, allowing for subsequent automated analysis of morphology and pathology. The first Large Scale Vertebrae Segmentation Challenge (VerSe 2019) showed that these perform well on normal anatomy, but fail in variants not frequently present in the training dataset. Building on that experience, we report on the largely increased VerSe 2020 dataset and results from the second iteration of the VerSe challenge (MICCAI 2020, Lima, Peru). VerSe 2020 comprises annotated spine computed tomography (CT) images from 300 subjects with 4142 fully visualized and annotated vertebrae, collected across multiple centres from four different scanner manufacturers, enriched with cases that exhibit anatomical variants such as enumeration abnormalities (n=77) and transitional vertebrae (n=161). Metadata includes vertebral labelling information, voxel-level segmentation masks obtained with a human-machine hybrid algorithm and anatomical ratings, to enable the development and benchmarking of robust and accurate segmentation algorithms.
△ Less
Submitted 10 March, 2021;
originally announced March 2021.
-
Are we using appropriate segmentation metrics? Identifying correlates of human expert perception for CNN training beyond rolling the DICE coefficient
Authors:
Florian Kofler,
Ivan Ezhov,
Fabian Isensee,
Fabian Balsiger,
Christoph Berger,
Maximilian Koerner,
Beatrice Demiray,
Julia Rackerseder,
Johannes Paetzold,
Hongwei Li,
Suprosanna Shit,
Richard McKinley,
Marie Piraud,
Spyridon Bakas,
Claus Zimmer,
Nassir Navab,
Jan Kirschke,
Benedikt Wiestler,
Bjoern Menze
Abstract:
Metrics optimized in complex machine learning tasks are often selected in an ad-hoc manner. It is unknown how they align with human expert perception. We explore the correlations between established quantitative segmentation quality metrics and qualitative evaluations by professionally trained human raters. Therefore, we conduct psychophysical experiments for two complex biomedical semantic segmen…
▽ More
Metrics optimized in complex machine learning tasks are often selected in an ad-hoc manner. It is unknown how they align with human expert perception. We explore the correlations between established quantitative segmentation quality metrics and qualitative evaluations by professionally trained human raters. Therefore, we conduct psychophysical experiments for two complex biomedical semantic segmentation problems. We discover that current standard metrics and loss functions correlate only moderately with the segmentation quality assessment of experts. Importantly, this effect is particularly pronounced for clinically relevant structures, such as the enhancing tumor compartment of glioma in brain magnetic resonance and grey matter in ultrasound imaging. It is often unclear how to optimize abstract metrics, such as human expert perception, in convolutional neural network (CNN) training. To cope with this challenge, we propose a novel strategy employing techniques of classical statistics to create complementary compound loss functions to better approximate human expert perception. Across all rating experiments, human experts consistently scored computer-generated segmentations better than the human-curated reference labels. Our results, therefore, strongly question many current practices in medical image segmentation and provide meaningful cues for future research.
△ Less
Submitted 2 May, 2023; v1 submitted 10 March, 2021;
originally announced March 2021.
-
ImJoy: an open-source computational platform for the deep learning era
Authors:
Wei Ouyang,
Florian Mueller,
Martin Hjelmare,
Emma Lundberg,
Christophe Zimmer
Abstract:
Deep learning methods have shown extraordinary potential for analyzing very diverse biomedical data, but their dissemination beyond developers is hindered by important computational hurdles. We introduce ImJoy (https://imjoy.io/), a flexible and open-source browser-based platform designed to facilitate widespread reuse of deep learning solutions in biomedical research. We highlight ImJoy's main fe…
▽ More
Deep learning methods have shown extraordinary potential for analyzing very diverse biomedical data, but their dissemination beyond developers is hindered by important computational hurdles. We introduce ImJoy (https://imjoy.io/), a flexible and open-source browser-based platform designed to facilitate widespread reuse of deep learning solutions in biomedical research. We highlight ImJoy's main features and illustrate its functionalities with deep learning plugins for mobile and interactive image analysis and genomics.
△ Less
Submitted 30 May, 2019;
originally announced May 2019.
-
DeepVesselNet: Vessel Segmentation, Centerline Prediction, and Bifurcation Detection in 3-D Angiographic Volumes
Authors:
Giles Tetteh,
Velizar Efremov,
Nils D. Forkert,
Matthias Schneider,
Jan Kirschke,
Bruno Weber,
Claus Zimmer,
Marie Piraud,
Bjoern H. Menze
Abstract:
We present DeepVesselNet, an architecture tailored to the challenges faced when extracting vessel networks or trees and corresponding features in 3-D angiographic volumes using deep learning. We discuss the problems of low execution speed and high memory requirements associated with full 3-D convolutional networks, high-class imbalance arising from the low percentage of vessel voxels, and unavaila…
▽ More
We present DeepVesselNet, an architecture tailored to the challenges faced when extracting vessel networks or trees and corresponding features in 3-D angiographic volumes using deep learning. We discuss the problems of low execution speed and high memory requirements associated with full 3-D convolutional networks, high-class imbalance arising from the low percentage of vessel voxels, and unavailability of accurately annotated training data - and offer solutions as the building blocks of DeepVesselNet.
First, we formulate 2-D orthogonal cross-hair filters which make use of 3-D context information at a reduced computational burden. Second, we introduce a class balancing cross-entropy loss function with false positive rate correction to handle the high-class imbalance and high false positive rate problems associated with existing loss functions. Finally, we generate synthetic dataset using a computational angiogenesis model capable of generating vascular trees under physiological constraints on local network structure and topology and use these data for transfer learning.
DeepVesselNet is optimized for segmenting and analyzing vessels, and we test the performance on a range of angiographic volumes including clinical MRA data of the human brain, as well as X-ray tomographic microscopy scans of the rat brain. Our experiments show that, by replacing 3-D filters with cross-hair filters in our network, we achieve over 23% improvement in speed, lower memory footprint, lower network complexity which prevents overfitting and comparable accuracy (with a Cox-Wilcoxon paired sample significance test p-value of 0.07 when compared to full 3-D filters). Our class balancing metric is crucial for training the network and transfer learning with synthetic data is an efficient, robust, and very generalizable approach leading to a network that excels in a variety of angiography segmentation tasks.
△ Less
Submitted 13 August, 2019; v1 submitted 25 March, 2018;
originally announced March 2018.
-
Deep-FExt: Deep Feature Extraction for Vessel Segmentation and Centerline Prediction
Authors:
Giles Tetteh,
Markus Rempfler,
Bjoern H. Menze,
Claus Zimmer
Abstract:
Feature extraction is a very crucial task in image and pixel (voxel) classification and regression in biomedical image modeling. In this work we present a machine learning based feature extraction scheme based on inception models for pixel classification tasks. We extract features under multi-scale and multi-layer schemes through convolutional operators. Layers of Fully Convolutional Network are l…
▽ More
Feature extraction is a very crucial task in image and pixel (voxel) classification and regression in biomedical image modeling. In this work we present a machine learning based feature extraction scheme based on inception models for pixel classification tasks. We extract features under multi-scale and multi-layer schemes through convolutional operators. Layers of Fully Convolutional Network are later stacked on this feature extraction layers and trained end-to-end for the purpose of classification. We test our model on the DRIVE and STARE public data sets for the purpose of segmentation and centerline detection and it out performs most existing hand crafted or deterministic feature schemes found in literature. We achieve an average maximum Dice of 0.85 on the DRIVE data set which out performs the scores from the second human annotator of this data set. We also achieve an average maximum Dice of 0.85 and kappa of 0.84 on the STARE data set. Though these datasets are mainly 2-D we also propose ways of extending this feature extraction scheme to handle 3-D datasets.
△ Less
Submitted 12 April, 2017;
originally announced April 2017.
-
Reducing local minima in fitness landscapes of parameter estimation by using piecewise evaluation and state estimation
Authors:
Christoph Zimmer,
Frank T. Bergmann,
Sven Sahle
Abstract:
Ordinary differential equations (ODE) are widely used for modeling in Systems Biology. As most commonly only some of the kinetic parameters are measurable or precisely known, parameter estimation techniques are applied to parametrize the model to experimental data. A main challenge for the parameter estimation is the complexity of the parameter space, especially its high dimensionality and local m…
▽ More
Ordinary differential equations (ODE) are widely used for modeling in Systems Biology. As most commonly only some of the kinetic parameters are measurable or precisely known, parameter estimation techniques are applied to parametrize the model to experimental data. A main challenge for the parameter estimation is the complexity of the parameter space, especially its high dimensionality and local minima.
Parameter estimation techniques consist of an objective function, measuring how well a certain parameter set describes the experimental data, and an optimization algorithm that optimizes this objective function. A lot of effort has been spent on develo** highly sophisticated optimization algorithms to cope with the complexity in the parameter space, but surprisingly few articles address the influence of the objective function on the computational complexity in finding global optima. We extend a recently developed multiple shooting for stochastic systems (MSS) objective function for parameter estimation of stochastic models and apply it to parameter estimation of ODE models. This MSS objective function treats the intervals between measurement points separately. This separate treatment allows the ODE trajectory to stay closer to the data and we show that it reduces the complexity of the parameter space.
We use examples from Systems Biology, namely a Lotka-Volterra model, a FitzHugh-Nagumo oscillator and a Calcium oscillation model, to demonstrate the power of the MSS approach for reducing the complexity and the number of local minima in the parameter space. The approach is fully implemented in the COPASI software package and, therefore, easily accessible for a wide community of researchers.
△ Less
Submitted 18 January, 2016;
originally announced January 2016.
-
The Estimation of Subjective Probabilities via Categorical Judgments of Uncertainty
Authors:
Alf C. Zimmer
Abstract:
Theoretically as well as experimentally it is investigated how people represent their knowledge in order to make decisions or to share their knowledge with others. Experiment 1 probes into the ways how people 6ather information about the frequencies of events and how the requested response mode, that is, numerical vs. verbal estimates interferes with this knowledge. The least interference occurs i…
▽ More
Theoretically as well as experimentally it is investigated how people represent their knowledge in order to make decisions or to share their knowledge with others. Experiment 1 probes into the ways how people 6ather information about the frequencies of events and how the requested response mode, that is, numerical vs. verbal estimates interferes with this knowledge. The least interference occurs if the subjects are allowed to give verbal responses. From this it is concluded that processing knowledge about uncertainty categorically, that is, by means of verbal expressions, imposes less mental work load on the decision matter than numerical processing. Possibility theory is used as a framework for modeling the individual usage of verbal categories for grades of uncertainty. The 'elastic' constraints on the verbal expressions for every sing1e subject are determined in Experiment 2 by means of sequential calibration. In further experiments it is shown that the superiority of the verbal processing of knowledge about uncertainty guise generally reduces persistent biases reported in the literature: conservatism (Experiment 3) and neg1igence of regression (Experiment 4). The reanalysis of Hormann's data reveal that in verbal Judgments people exhibit sensitivity for base rates and are not prone to the conjunction fallacy. In a final experiment (5) about predictions in a real-life situation it turns out that in a numerical forecasting task subjects restricted themselves to those parts of their knowledge which are numerical. On the other hand subjects in a verbal forecasting task accessed verbally as well as numerically stated knowledge. Forecasting is structurally related to the estimation of probabilities for rare events insofar as supporting and contradicting arguments have to be evaluated and the choice of the final Judgment has to be Justified according to the evidence brought forward. In order to assist people in such choice situations a formal model for the interactive checking of arguments has been developed. The model transforms the normal-language quantifiers used in the arguments into fuzzy numbers and evaluates the given train of arguments by means of fuzzy numerica1 operations. Ambiguities in the meanings of quantifiers are resolved interactively.
△ Less
Submitted 27 March, 2013;
originally announced April 2013.
