-
Global Safe Sequential Learning via Efficient Knowledge Transfer
Authors:
Cen-You Li,
Olaf Duennbier,
Marc Toussaint,
Barbara Rakitsch,
Christoph Zimmer
Abstract:
Sequential learning methods such as active learning and Bayesian optimization select the most informative data to learn about a task. In many medical or engineering applications, the data selection is constrained by a priori unknown safety conditions. A promissing line of safe learning methods utilize Gaussian processes (GPs) to model the safety probability and perform data selection in areas with…
▽ More
Sequential learning methods such as active learning and Bayesian optimization select the most informative data to learn about a task. In many medical or engineering applications, the data selection is constrained by a priori unknown safety conditions. A promissing line of safe learning methods utilize Gaussian processes (GPs) to model the safety probability and perform data selection in areas with high safety confidence. However, accurate safety modeling requires prior knowledge or consumes data. In addition, the safety confidence centers around the given observations which leads to local exploration. As transferable source knowledge is often available in safety critical experiments, we propose to consider transfer safe sequential learning to accelerate the learning of safety. We further consider a pre-computation of source components to reduce the additional computational load that is introduced by incorporating source data. In this paper, we theoretically analyze the maximum explorable safe regions of conventional safe learning methods. Furthermore, we empirically demonstrate that our approach 1) learns a task with lower data consumption, 2) globally explores multiple disjoint safe regions under guidance of the source knowledge, and 3) operates with computation comparable to conventional safe learning methods.
△ Less
Submitted 15 April, 2024; v1 submitted 22 February, 2024;
originally announced February 2024.
-
Safe Active Learning for Time-Series Modeling with Gaussian Processes
Authors:
Christoph Zimmer,
Mona Meister,
Duy Nguyen-Tuong
Abstract:
Learning time-series models is useful for many applications, such as simulation and forecasting. In this study, we consider the problem of actively learning time-series models while taking given safety constraints into account. For time-series modeling we employ a Gaussian process with a nonlinear exogenous input structure. The proposed approach generates data appropriate for time series model lea…
▽ More
Learning time-series models is useful for many applications, such as simulation and forecasting. In this study, we consider the problem of actively learning time-series models while taking given safety constraints into account. For time-series modeling we employ a Gaussian process with a nonlinear exogenous input structure. The proposed approach generates data appropriate for time series model learning, i.e. input and output trajectories, by dynamically exploring the input space. The approach parametrizes the input trajectory as consecutive trajectory sections, which are determined stepwise given safety requirements and past observations. We analyze the proposed algorithm and evaluate it empirically on a technical application. The results show the effectiveness of our approach in a realistic technical use case.
△ Less
Submitted 9 February, 2024;
originally announced February 2024.
-
Structural Kernel Search via Bayesian Optimization and Symbolical Optimal Transport
Authors:
Matthias Bitzer,
Mona Meister,
Christoph Zimmer
Abstract:
Despite recent advances in automated machine learning, model selection is still a complex and computationally intensive process. For Gaussian processes (GPs), selecting the kernel is a crucial task, often done manually by the expert. Additionally, evaluating the model selection criteria for Gaussian processes typically scales cubically in the sample size, rendering kernel search particularly compu…
▽ More
Despite recent advances in automated machine learning, model selection is still a complex and computationally intensive process. For Gaussian processes (GPs), selecting the kernel is a crucial task, often done manually by the expert. Additionally, evaluating the model selection criteria for Gaussian processes typically scales cubically in the sample size, rendering kernel search particularly computationally expensive. We propose a novel, efficient search method through a general, structured kernel space. Previous methods solved this task via Bayesian optimization and relied on measuring the distance between GP's directly in function space to construct a kernel-kernel. We present an alternative approach by defining a kernel-kernel over the symbolic representation of the statistical hypothesis that is associated with a kernel. We empirically show that this leads to a computationally more efficient way of searching through a discrete kernel space.
△ Less
Submitted 21 October, 2022;
originally announced October 2022.
-
From Small Scales to Large Scales: Distance-to-Measure Density based Geometric Analysis of Complex Data
Authors:
Katharina Proksch,
Christoph Alexander Weitkamp,
Thomas Staudt,
Benoît Lelandais,
Christophe Zimmer
Abstract:
How can we tell complex point clouds with different small scale characteristics apart, while disregarding global features? Can we find a suitable transformation of such data in a way that allows to discriminate between differences in this sense with statistical guarantees? In this paper, we consider the analysis and classification of complex point clouds as they are obtained, e.g., via single mole…
▽ More
How can we tell complex point clouds with different small scale characteristics apart, while disregarding global features? Can we find a suitable transformation of such data in a way that allows to discriminate between differences in this sense with statistical guarantees? In this paper, we consider the analysis and classification of complex point clouds as they are obtained, e.g., via single molecule localization microscopy. We focus on the task of identifying differences between noisy point clouds based on small scale characteristics, while disregarding large scale information such as overall size. We propose an approach based on a transformation of the data via the so-called Distance-to-Measure (DTM) function, a transformation which is based on the average of nearest neighbor distances. For each data set, we estimate the probability density of average local distances of all data points and use the estimated densities for classification. While the applicability is immediate and the practical performance of the proposed methodology is very good, the theoretical study of the density estimators is quite challenging, as they are based on i.i.d. observations that have been obtained via a complicated transformation. In fact, the transformed data are stochastically dependent in a non-local way that is not captured by commonly considered dependence measures. Nonetheless, we show that the asymptotic behaviour of the density estimator is driven by a kernel density estimator of certain i.i.d. random variables by using theoretical properties of U-statistics, which allows to handle the dependencies via a Hoeffding decomposition. We show via a numerical study and in an application to simulated single molecule localization microscopy data of chromatin fibers that unsupervised classification tasks based on estimated DTM-densities achieve excellent separation results.
△ Less
Submitted 18 May, 2022; v1 submitted 16 May, 2022;
originally announced May 2022.
-
Safe Active Learning for Multi-Output Gaussian Processes
Authors:
Cen-You Li,
Barbara Rakitsch,
Christoph Zimmer
Abstract:
Multi-output regression problems are commonly encountered in science and engineering. In particular, multi-output Gaussian processes have been emerged as a promising tool for modeling these complex systems since they can exploit the inherent correlations and provide reliable uncertainty estimates. In many applications, however, acquiring the data is expensive and safety concerns might arise (e.g.…
▽ More
Multi-output regression problems are commonly encountered in science and engineering. In particular, multi-output Gaussian processes have been emerged as a promising tool for modeling these complex systems since they can exploit the inherent correlations and provide reliable uncertainty estimates. In many applications, however, acquiring the data is expensive and safety concerns might arise (e.g. robotics, engineering). We propose a safe active learning approach for multi-output Gaussian process regression. This approach queries the most informative data or output taking the relatedness between the regressors and safety constraints into account. We prove the effectiveness of our approach by providing theoretical analysis and by demonstrating empirical results on simulated datasets and on a real-world engineering dataset. On all datasets, our approach shows improved convergence compared to its competitors.
△ Less
Submitted 28 March, 2022;
originally announced March 2022.
-
Active Learning in Gaussian Process State Space Model
Authors:
Hon Sum Alec Yu,
Dingling Yao,
Christoph Zimmer,
Marc Toussaint,
Duy Nguyen-Tuong
Abstract:
We investigate active learning in Gaussian Process state-space models (GPSSM). Our problem is to actively steer the system through latent states by determining its inputs such that the underlying dynamics can be optimally learned by a GPSSM. In order that the most informative inputs are selected, we employ mutual information as our active learning criterion. In particular, we present two approache…
▽ More
We investigate active learning in Gaussian Process state-space models (GPSSM). Our problem is to actively steer the system through latent states by determining its inputs such that the underlying dynamics can be optimally learned by a GPSSM. In order that the most informative inputs are selected, we employ mutual information as our active learning criterion. In particular, we present two approaches for the approximation of mutual information for the GPSSM given latent states. The proposed approaches are evaluated in several physical systems where we actively learn the underlying non-linear dynamics represented by the state-space model.
△ Less
Submitted 30 July, 2021;
originally announced August 2021.
-
ImJoy: an open-source computational platform for the deep learning era
Authors:
Wei Ouyang,
Florian Mueller,
Martin Hjelmare,
Emma Lundberg,
Christophe Zimmer
Abstract:
Deep learning methods have shown extraordinary potential for analyzing very diverse biomedical data, but their dissemination beyond developers is hindered by important computational hurdles. We introduce ImJoy (https://imjoy.io/), a flexible and open-source browser-based platform designed to facilitate widespread reuse of deep learning solutions in biomedical research. We highlight ImJoy's main fe…
▽ More
Deep learning methods have shown extraordinary potential for analyzing very diverse biomedical data, but their dissemination beyond developers is hindered by important computational hurdles. We introduce ImJoy (https://imjoy.io/), a flexible and open-source browser-based platform designed to facilitate widespread reuse of deep learning solutions in biomedical research. We highlight ImJoy's main features and illustrate its functionalities with deep learning plugins for mobile and interactive image analysis and genomics.
△ Less
Submitted 30 May, 2019;
originally announced May 2019.
-
Deep-FExt: Deep Feature Extraction for Vessel Segmentation and Centerline Prediction
Authors:
Giles Tetteh,
Markus Rempfler,
Bjoern H. Menze,
Claus Zimmer
Abstract:
Feature extraction is a very crucial task in image and pixel (voxel) classification and regression in biomedical image modeling. In this work we present a machine learning based feature extraction scheme based on inception models for pixel classification tasks. We extract features under multi-scale and multi-layer schemes through convolutional operators. Layers of Fully Convolutional Network are l…
▽ More
Feature extraction is a very crucial task in image and pixel (voxel) classification and regression in biomedical image modeling. In this work we present a machine learning based feature extraction scheme based on inception models for pixel classification tasks. We extract features under multi-scale and multi-layer schemes through convolutional operators. Layers of Fully Convolutional Network are later stacked on this feature extraction layers and trained end-to-end for the purpose of classification. We test our model on the DRIVE and STARE public data sets for the purpose of segmentation and centerline detection and it out performs most existing hand crafted or deterministic feature schemes found in literature. We achieve an average maximum Dice of 0.85 on the DRIVE data set which out performs the scores from the second human annotator of this data set. We also achieve an average maximum Dice of 0.85 and kappa of 0.84 on the STARE data set. Though these datasets are mainly 2-D we also propose ways of extending this feature extraction scheme to handle 3-D datasets.
△ Less
Submitted 12 April, 2017;
originally announced April 2017.
