Showing 151–200 of 342 results for author: Krause, A

Deep Learning for Climate Model Output Statistics

Authors: Michael Steininger, Daniel Abel, Katrin Ziegler, Anna Krause, Heiko Paeth, Andreas Hotho

Abstract: Climate models are an important tool for the assessment of prospective climate change effects but they suffer from systematic and representation errors, especially for precipitation. Model output statistics (MOS) reduce these errors by fitting the model output to observational data with machine learning. In this work, we explore the feasibility and potential of deep learning with convolutional neu… ▽ More Climate models are an important tool for the assessment of prospective climate change effects but they suffer from systematic and representation errors, especially for precipitation. Model output statistics (MOS) reduce these errors by fitting the model output to observational data with machine learning. In this work, we explore the feasibility and potential of deep learning with convolutional neural networks (CNNs) for MOS. We propose the CNN architecture ConvMOS specifically designed for reducing errors in climate model outputs and apply it to the climate model REMO. Our results show a considerable reduction of errors and mostly improved performance compared to three commonly used MOS approaches. △ Less

Submitted 9 December, 2020; originally announced December 2020.

Comments: Accepted for the Tackling Climate Change with Machine Learning Workshop at NeurIPS 2020

Map** spaces in homotopy coherent nerves

Authors: Fabian Hebestreit, Achim Krause

Abstract: We give a direct proof that middle map** spaces in coherent nerves of Kan enriched categories have the same homotopy type as the original map** spaces. We give a direct proof that middle map** spaces in coherent nerves of Kan enriched categories have the same homotopy type as the original map** spaces. △ Less

Submitted 18 November, 2020; originally announced November 2020.

Comments: 14 pages

MSC Class: 18N60; 55U10

Logistic Q-Learning

Authors: Joan Bas-Serrano, Sebastian Curi, Andreas Krause, Gergely Neu

Abstract: We propose a new reinforcement learning algorithm derived from a regularized linear-programming formulation of optimal control in MDPs. The method is closely related to the classic Relative Entropy Policy Search (REPS) algorithm of Peters et al. (2010), with the key difference that our method introduces a Q-function that enables efficient exact model-free implementation. The main feature of our al… ▽ More We propose a new reinforcement learning algorithm derived from a regularized linear-programming formulation of optimal control in MDPs. The method is closely related to the classic Relative Entropy Policy Search (REPS) algorithm of Peters et al. (2010), with the key difference that our method introduces a Q-function that enables efficient exact model-free implementation. The main feature of our algorithm (called QREPS) is a convex loss function for policy evaluation that serves as a theoretically sound alternative to the widely used squared Bellman error. We provide a practical saddle-point optimization method for minimizing this loss function and provide an error-propagation analysis that relates the quality of the individual updates to the performance of the output policy. Finally, we demonstrate the effectiveness of our method on a range of benchmark problems. △ Less

Submitted 25 February, 2021; v1 submitted 21 October, 2020; originally announced October 2020.

Online Active Model Selection for Pre-trained Classifiers

Authors: Mohammad Reza Karimi, Nezihe Merve Gürel, Bojan Karlaš, Johannes Rausch, Ce Zhang, Andreas Krause

Abstract: Given $k$ pre-trained classifiers and a stream of unlabeled data examples, how can we actively decide when to query a label so that we can distinguish the best model from the rest while making a small number of queries? Answering this question has a profound impact on a range of practical scenarios. In this work, we design an online selective sampling approach that actively selects informative exa… ▽ More Given $k$ pre-trained classifiers and a stream of unlabeled data examples, how can we actively decide when to query a label so that we can distinguish the best model from the rest while making a small number of queries? Answering this question has a profound impact on a range of practical scenarios. In this work, we design an online selective sampling approach that actively selects informative examples to label and outputs the best model with high probability at any round. Our algorithm can be used for online prediction tasks for both adversarial and stochastic streams. We establish several theoretical guarantees for our algorithm and extensively demonstrate its effectiveness in our experimental studies. △ Less

Submitted 17 April, 2021; v1 submitted 19 October, 2020; originally announced October 2020.

Semi-supervised Batch Active Learning via Bilevel Optimization

Authors: Zalán Borsos, Marco Tagliasacchi, Andreas Krause

Abstract: Active learning is an effective technique for reducing the labeling cost by improving data efficiency. In this work, we propose a novel batch acquisition strategy for active learning in the setting where the model training is performed in a semi-supervised manner. We formulate our approach as a data summarization problem via bilevel optimization, where the queried batch consists of the points that… ▽ More Active learning is an effective technique for reducing the labeling cost by improving data efficiency. In this work, we propose a novel batch acquisition strategy for active learning in the setting where the model training is performed in a semi-supervised manner. We formulate our approach as a data summarization problem via bilevel optimization, where the queried batch consists of the points that best summarize the unlabeled data pool. We show that our method is highly effective in keyword detection tasks in the regime when only few labeled samples are available. △ Less

Submitted 19 October, 2020; originally announced October 2020.

Rao-Blackwellizing the Straight-Through Gumbel-Softmax Gradient Estimator

Authors: Max B. Paulus, Chris J. Maddison, Andreas Krause

Abstract: Gradient estimation in models with discrete latent variables is a challenging problem, because the simplest unbiased estimators tend to have high variance. To counteract this, modern estimators either introduce bias, rely on multiple function evaluations, or use learned, input-dependent baselines. Thus, there is a need for estimators that require minimal tuning, are computationally cheap, and have… ▽ More Gradient estimation in models with discrete latent variables is a challenging problem, because the simplest unbiased estimators tend to have high variance. To counteract this, modern estimators either introduce bias, rely on multiple function evaluations, or use learned, input-dependent baselines. Thus, there is a need for estimators that require minimal tuning, are computationally cheap, and have low mean squared error. In this paper, we show that the variance of the straight-through variant of the popular Gumbel-Softmax estimator can be reduced through Rao-Blackwellization without increasing the number of function evaluations. This provably reduces the mean squared error. We empirically demonstrate that this leads to variance reduction, faster convergence, and generally improved performance in two unsupervised latent variable models. △ Less

Submitted 9 October, 2020; originally announced October 2020.

Learning Set Functions that are Sparse in Non-Orthogonal Fourier Bases

Authors: Chris Wendler, Andisheh Amrollahi, Bastian Seifert, Andreas Krause, Markus Püschel

Abstract: Many applications of machine learning on discrete domains, such as learning preference functions in recommender systems or auctions, can be reduced to estimating a set function that is sparse in the Fourier domain. In this work, we present a new family of algorithms for learning Fourier-sparse set functions. They require at most $nk - k \log_2 k + k$ queries (set function evaluations), under mild… ▽ More Many applications of machine learning on discrete domains, such as learning preference functions in recommender systems or auctions, can be reduced to estimating a set function that is sparse in the Fourier domain. In this work, we present a new family of algorithms for learning Fourier-sparse set functions. They require at most $nk - k \log_2 k + k$ queries (set function evaluations), under mild conditions on the Fourier coefficients, where $n$ is the size of the ground set and $k$ the number of non-zero Fourier coefficients. In contrast to other work that focused on the orthogonal Walsh-Hadamard transform, our novel algorithms operate with recently introduced non-orthogonal Fourier transforms that offer different notions of Fourier-sparsity. These naturally arise when modeling, e.g., sets of items forming substitutes and complements. We demonstrate effectiveness on several real-world applications. △ Less

Submitted 29 March, 2021; v1 submitted 1 October, 2020; originally announced October 2020.

Journal ref: Proc. AAAI, 2021

Isolating Patterns in Open Reaction-Diffusion Systems

Authors: Andrew L. Krause, Václav Klika, Philip K. Maini, Denis Headon, Eamonn A. Gaffney

Abstract: Realistic examples of reaction-diffusion phenomena governing spatial and spatiotemporal pattern formation are rarely isolated systems, either chemically or thermodynamically. However, even formulations of `open' reaction-diffusion systems often neglect the role of domain boundaries. Most idealizations of closed reaction-diffusion systems employ no-flux boundary conditions, and often patterns will… ▽ More Realistic examples of reaction-diffusion phenomena governing spatial and spatiotemporal pattern formation are rarely isolated systems, either chemically or thermodynamically. However, even formulations of `open' reaction-diffusion systems often neglect the role of domain boundaries. Most idealizations of closed reaction-diffusion systems employ no-flux boundary conditions, and often patterns will form up to, or along, these boundaries. Motivated by boundaries of patterning fields related to the emergence of spatial form in embryonic development, we propose a set of mixed boundary conditions for a two-species reaction-diffusion system which forms inhomogeneous solutions away from the boundary of the domain for a variety of different reaction kinetics, with a prescribed uniform state near the boundary. We show that these boundary conditions can be derived from a larger heterogeneous field, indicating that these conditions can arise naturally if cell signalling or other properties of the medium vary in space. We explain the basic mechanisms behind this pattern localization, and demonstrate that it can capture a large range of localized patterning in one, two, and three dimensions, and that this framework can be applied to systems involving more than two species. Furthermore, the boundary conditions proposed lead to more symmetrical patterns on the interior of the domain, and plausibly capture more realistic boundaries in developmental systems. Finally, we show that these isolated patterns are more robust to fluctuations in initial conditions, and that they allow intriguing possibilities of pattern selection via geometry, distinct from known selection mechanisms. △ Less

Submitted 12 May, 2021; v1 submitted 28 September, 2020; originally announced September 2020.

Comments: 25 pages, 11 figures

MSC Class: 92C15 (primary) 35B36; 35K57 (secondary)

A Deep Learning Approach to Geographical Candidate Selection through Toponym Matching

Authors: Mariona Coll Ardanuy, Kasra Hosseini, Katherine McDonough, Amrey Krause, Daniel van Strien, Federico Nanni

Abstract: Recognizing toponyms and resolving them to their real-world referents is required for providing advanced semantic access to textual data. This process is often hindered by the high degree of variation in toponyms. Candidate selection is the task of identifying the potential entities that can be referred to by a toponym previously recognized. While it has traditionally received little attention in… ▽ More Recognizing toponyms and resolving them to their real-world referents is required for providing advanced semantic access to textual data. This process is often hindered by the high degree of variation in toponyms. Candidate selection is the task of identifying the potential entities that can be referred to by a toponym previously recognized. While it has traditionally received little attention in the research community, it has been shown that candidate selection has a significant impact on downstream tasks (i.e. entity resolution), especially in noisy or non-standard text. In this paper, we introduce a flexible deep learning method for candidate selection through toponym matching, using state-of-the-art neural network architectures. We perform an intrinsic toponym matching evaluation based on several new realistic datasets, which cover various challenging scenarios (cross-lingual and regional variations, as well as OCR errors). We report its performance on candidate selection in the context of the downstream task of toponym resolution, both on existing datasets and on a new manually-annotated resource of nineteenth-century English OCR'd text. △ Less

Submitted 22 September, 2020; v1 submitted 17 September, 2020; originally announced September 2020.

Comments: 10 pages, 1 figure

The Picard group in equivariant homotopy theory via stable module categories

Authors: Achim Krause

Abstract: We develop a mechanism of "isotropy separation for compact objects" that explicitly describes an invertible $G$-spectrum through its collection of geometric fixed points and gluing data located in certain variants of the stable module category. As an application, we carry out a complete analysis of invertible G-spectra in the case $G=A_5$. A further application is given by showing that the Picard… ▽ More We develop a mechanism of "isotropy separation for compact objects" that explicitly describes an invertible $G$-spectrum through its collection of geometric fixed points and gluing data located in certain variants of the stable module category. As an application, we carry out a complete analysis of invertible G-spectra in the case $G=A_5$. A further application is given by showing that the Picard groups of $\mathrm{Sp}^G$ and a category of derived Mackey functors agree. △ Less

Submitted 12 August, 2020; originally announced August 2020.

Comments: 37 pages

Learning to Play Sequential Games versus Unknown Opponents

Authors: Pier Giuseppe Sessa, Ilija Bogunovic, Maryam Kamgarpour, Andreas Krause

Abstract: We consider a repeated sequential game between a learner, who plays first, and an opponent who responds to the chosen action. We seek to design strategies for the learner to successfully interact with the opponent. While most previous approaches consider known opponent models, we focus on the setting in which the opponent's model is unknown. To this end, we use kernel-based regularity assumptions… ▽ More We consider a repeated sequential game between a learner, who plays first, and an opponent who responds to the chosen action. We seek to design strategies for the learner to successfully interact with the opponent. While most previous approaches consider known opponent models, we focus on the setting in which the opponent's model is unknown. To this end, we use kernel-based regularity assumptions to capture and exploit the structure in the opponent's response. We propose a novel algorithm for the learner when playing against an adversarial sequence of opponents. The algorithm combines ideas from bilevel optimization and online learning to effectively balance between exploration (learning about the opponent's model) and exploitation (selecting highly rewarding actions for the learner). Our results include algorithm's regret guarantees that depend on the regularity of the opponent's response and scale sublinearly with the number of game rounds. Moreover, we specialize our approach to repeated Stackelberg games, and empirically demonstrate its effectiveness in a traffic routing and wildlife conservation task △ Less

Submitted 10 July, 2020; originally announced July 2020.

Stochastic Linear Bandits Robust to Adversarial Attacks

Authors: Ilija Bogunovic, Arpan Losalka, Andreas Krause, Jonathan Scarlett

Abstract: We consider a stochastic linear bandit problem in which the rewards are not only subject to random noise, but also adversarial attacks subject to a suitable budget $C$ (i.e., an upper bound on the sum of corruption magnitudes across the time horizon). We provide two variants of a Robust Phased Elimination algorithm, one that knows $C$ and one that does not. Both variants are shown to attain near-o… ▽ More We consider a stochastic linear bandit problem in which the rewards are not only subject to random noise, but also adversarial attacks subject to a suitable budget $C$ (i.e., an upper bound on the sum of corruption magnitudes across the time horizon). We provide two variants of a Robust Phased Elimination algorithm, one that knows $C$ and one that does not. Both variants are shown to attain near-optimal regret in the non-corrupted case $C = 0$, while incurring additional additive terms respectively having a linear and quadratic dependency on $C$ in general. We present algorithm independent lower bounds showing that these additive terms are near-optimal. In addition, in a contextual setting, we revisit a setup of diverse contexts, and show that a simple greedy algorithm is provably robust with a near-optimal additive regret term, despite performing no explicit exploration and not knowing $C$. △ Less

Submitted 27 October, 2020; v1 submitted 7 July, 2020; originally announced July 2020.

Continuous Submodular Function Maximization

Authors: Yatao Bian, Joachim M. Buhmann, Andreas Krause

Abstract: Continuous submodular functions are a category of generally non-convex/non-concave functions with a wide spectrum of applications. The celebrated property of this class of functions - continuous submodularity - enables both exact minimization and approximate maximization in poly. time. Continuous submodularity is obtained by generalizing the notion of submodularity from discrete domains to continu… ▽ More Continuous submodular functions are a category of generally non-convex/non-concave functions with a wide spectrum of applications. The celebrated property of this class of functions - continuous submodularity - enables both exact minimization and approximate maximization in poly. time. Continuous submodularity is obtained by generalizing the notion of submodularity from discrete domains to continuous domains. It intuitively captures a repulsive effect amongst different dimensions of the defined multivariate function. In this paper, we systematically study continuous submodularity and a class of non-convex optimization problems: continuous submodular function maximization. We start by a thorough characterization of the class of continuous submodular functions, and show that continuous submodularity is equivalent to a weak version of the diminishing returns (DR) property. Thus we also derive a subclass of continuous submodular functions, termed continuous DR-submodular functions, which enjoys the full DR property. Then we present operations that preserve continuous (DR-)submodularity, thus yielding general rules for composing new submodular functions. We establish intriguing properties for the problem of constrained DR-submodular maximization, such as the local-global relation. We identify several applications of continuous submodular optimization, ranging from influence maximization, MAP inference for DPPs to provable mean field inference. For these applications, continuous submodularity formalizes valuable domain knowledge relevant for optimizing this class of objectives. We present inapproximability results and provable algorithms for two problem settings: constrained monotone DR-submodular maximization and constrained non-monotone DR-submodular maximization. Finally, we extensively evaluate the effectiveness of the proposed algorithms. △ Less

Submitted 24 June, 2020; originally announced June 2020.

Comments: 64 pages

Safe Reinforcement Learning via Curriculum Induction

Authors: Matteo Turchetta, Andrey Kolobov, Shital Shah, Andreas Krause, Alekh Agarwal

Abstract: In safety-critical applications, autonomous agents may need to learn in an environment where mistakes can be very costly. In such settings, the agent needs to behave safely not only after but also while learning. To achieve this, existing safe reinforcement learning methods make an agent rely on priors that let it avoid dangerous situations during exploration with high probability, but both the pr… ▽ More In safety-critical applications, autonomous agents may need to learn in an environment where mistakes can be very costly. In such settings, the agent needs to behave safely not only after but also while learning. To achieve this, existing safe reinforcement learning methods make an agent rely on priors that let it avoid dangerous situations during exploration with high probability, but both the probabilistic guarantees and the smoothness assumptions inherent in the priors are not viable in many scenarios of interest such as autonomous driving. This paper presents an alternative approach inspired by human teaching, where an agent learns under the supervision of an automatic instructor that saves the agent from violating constraints during learning. In this model, we introduce the monitor that neither needs to know how to do well at the task the agent is learning nor needs to know how the environment works. Instead, it has a library of reset controllers that it activates when the agent starts behaving dangerously, preventing it from doing damage. Crucially, the choices of which reset controller to apply in which situation affect the speed of agent learning. Based on observing agents' progress, the teacher itself learns a policy for choosing the reset controllers, a curriculum, to optimize the agent's final policy reward. Our experiments use this framework in two environments to induce curricula for safe and efficient learning. △ Less

Submitted 21 January, 2021; v1 submitted 22 June, 2020; originally announced June 2020.

Learning Stabilizing Controllers for Unstable Linear Quadratic Regulators from a Single Trajectory

Authors: Lenart Treven, Sebastian Curi, Mojmir Mutny, Andreas Krause

Abstract: The principal task to control dynamical systems is to ensure their stability. When the system is unknown, robust approaches are promising since they aim to stabilize a large set of plausible systems simultaneously. We study linear controllers under quadratic costs model also known as linear quadratic regulators (LQR). We present two different semi-definite programs (SDP) which results in a control… ▽ More The principal task to control dynamical systems is to ensure their stability. When the system is unknown, robust approaches are promising since they aim to stabilize a large set of plausible systems simultaneously. We study linear controllers under quadratic costs model also known as linear quadratic regulators (LQR). We present two different semi-definite programs (SDP) which results in a controller that stabilizes all systems within an ellipsoid uncertainty set. We further show that the feasibility conditions of the proposed SDPs are \emph{equivalent}. Using the derived robust controller syntheses, we propose an efficient data dependent algorithm -- \textsc{eXploration} -- that with high probability quickly identifies a stabilizing controller. Our approach can be used to initialize existing algorithms that require a stabilizing controller as an input while adding constant to the regret. We further propose different heuristics which empirically reduce the number of steps taken by \textsc{eXploration} and reduce the suffered cost while searching for a stabilizing controller. △ Less

Submitted 23 November, 2020; v1 submitted 19 June, 2020; originally announced June 2020.

Efficient Model-Based Reinforcement Learning through Optimistic Policy Search and Planning

Authors: Sebastian Curi, Felix Berkenkamp, Andreas Krause

Abstract: Model-based reinforcement learning algorithms with probabilistic dynamical models are amongst the most data-efficient learning methods. This is often attributed to their ability to distinguish between epistemic and aleatoric uncertainty. However, while most algorithms distinguish these two uncertainties for learning the model, they ignore it when optimizing the policy, which leads to greedy and in… ▽ More Model-based reinforcement learning algorithms with probabilistic dynamical models are amongst the most data-efficient learning methods. This is often attributed to their ability to distinguish between epistemic and aleatoric uncertainty. However, while most algorithms distinguish these two uncertainties for learning the model, they ignore it when optimizing the policy, which leads to greedy and insufficient exploration. At the same time, there are no practical solvers for optimistic exploration algorithms. In this paper, we propose a practical optimistic exploration algorithm (H-UCRL). H-UCRL reparameterizes the set of plausible models and hallucinates control directly on the epistemic uncertainty. By augmenting the input space with the hallucinated inputs, H-UCRL can be solved using standard greedy planners. Furthermore, we analyze H-UCRL and construct a general regret bound for well-calibrated models, which is provably sublinear in the case of Gaussian Process models. Based on this theoretical foundation, we show how optimistic exploration can be easily combined with state-of-the-art reinforcement learning algorithms and different probabilistic models. Our experiments demonstrate that optimistic exploration significantly speeds-up learning when there are penalties on actions, a setting that is notoriously difficult for existing model-based reinforcement learning algorithms. △ Less

Submitted 1 December, 2020; v1 submitted 15 June, 2020; originally announced June 2020.

Gradient Estimation with Stochastic Softmax Tricks

Authors: Max B. Paulus, Dami Choi, Daniel Tarlow, Andreas Krause, Chris J. Maddison

Abstract: The Gumbel-Max trick is the basis of many relaxed gradient estimators. These estimators are easy to implement and low variance, but the goal of scaling them comprehensively to large combinatorial distributions is still outstanding. Working within the perturbation model framework, we introduce stochastic softmax tricks, which generalize the Gumbel-Softmax trick to combinatorial spaces. Our framewor… ▽ More The Gumbel-Max trick is the basis of many relaxed gradient estimators. These estimators are easy to implement and low variance, but the goal of scaling them comprehensively to large combinatorial distributions is still outstanding. Working within the perturbation model framework, we introduce stochastic softmax tricks, which generalize the Gumbel-Softmax trick to combinatorial spaces. Our framework is a unified perspective on existing relaxed estimators for perturbation models, and it contains many novel relaxations. We design structured relaxations for subset selection, spanning trees, arborescences, and others. When compared to less structured baselines, we find that stochastic softmax tricks can be used to train latent variable models that perform better and discover more latent structure. △ Less

Submitted 28 February, 2021; v1 submitted 14 June, 2020; originally announced June 2020.

Comments: NeurIPS 2020, final copy

Learning Graph Models for Retrosynthesis Prediction

Authors: Vignesh Ram Somnath, Charlotte Bunne, Connor W. Coley, Andreas Krause, Regina Barzilay

Abstract: Retrosynthesis prediction is a fundamental problem in organic synthesis, where the task is to identify precursor molecules that can be used to synthesize a target molecule. A key consideration in building neural models for this task is aligning model design with strategies adopted by chemists. Building on this viewpoint, this paper introduces a graph-based approach that capitalizes on the idea tha… ▽ More Retrosynthesis prediction is a fundamental problem in organic synthesis, where the task is to identify precursor molecules that can be used to synthesize a target molecule. A key consideration in building neural models for this task is aligning model design with strategies adopted by chemists. Building on this viewpoint, this paper introduces a graph-based approach that capitalizes on the idea that the graph topology of precursor molecules is largely unaltered during a chemical reaction. The model first predicts the set of graph edits transforming the target into incomplete molecules called synthons. Next, the model learns to expand synthons into complete molecules by attaching relevant leaving groups. This decomposition simplifies the architecture, making its predictions more interpretable, and also amenable to manual correction. Our model achieves a top-1 accuracy of $53.7\%$, outperforming previous template-free and semi-template-based methods. △ Less

Submitted 4 June, 2021; v1 submitted 12 June, 2020; originally announced June 2020.

Turing Patterning in Stratified Domains

Authors: Andrew L. Krause, Václav Klika, Jacob Halatek, Paul K. Grant, Thomas E. Woolley, Neil Dalchau, Eamonn A. Gaffney

Abstract: Reaction-diffusion processes across layered media arise in several scientific domains such as pattern-forming E. coli on agar substrates, epidermal-mesenchymal coupling in development, and symmetry-breaking in cell polarisation. We develop a modelling framework for bi-layer reaction-diffusion systems and relate it to a range of existing models. We derive conditions for diffusion-driven instability… ▽ More Reaction-diffusion processes across layered media arise in several scientific domains such as pattern-forming E. coli on agar substrates, epidermal-mesenchymal coupling in development, and symmetry-breaking in cell polarisation. We develop a modelling framework for bi-layer reaction-diffusion systems and relate it to a range of existing models. We derive conditions for diffusion-driven instability of a spatially homogeneous equilibrium analogous to the classical conditions for a Turing instability in the simplest nontrivial setting where one domain has a standard reaction-diffusion system, and the other permits only diffusion. Due to the transverse coupling between these two regions, standard techniques for computing eigenfunctions of the Laplacian cannot be applied, and so we propose an alternative method to compute the dispersion relation directly. We compare instability conditions with full numerical simulations to demonstrate impacts of the geometry and coupling parameters on patterning, and explore various experimentally-relevant asymptotic regimes. In the regime where the first domain is suitably thin, we recover a simple modulation of the standard Turing conditions, and find that often the broad impact of the diffusion-only domain is to reduce the ability of the system to form patterns. We also demonstrate complex impacts of this coupling on pattern formation. For instance, we exhibit non-monotonicity of pattern-forming instabilities with respect to geometric and coupling parameters, and highlight an instability from a nontrivial interaction between kinetics in one domain and diffusion in the other. These results are valuable for informing design choices in applications such as synthetic engineering of Turing patterns, but also for understanding the role of stratified media in modulating pattern-forming processes in developmental biology and beyond. △ Less

Submitted 17 September, 2020; v1 submitted 11 June, 2020; originally announced June 2020.

Comments: 25 pages, 7 figures

MSC Class: 92C15 (primary) 35B36; 35K57 (secondary)

Coresets via Bilevel Optimization for Continual Learning and Streaming

Authors: Zalán Borsos, Mojmír Mutný, Andreas Krause

Abstract: Coresets are small data summaries that are sufficient for model training. They can be maintained online, enabling efficient handling of large data streams under resource constraints. However, existing constructions are limited to simple models such as k-means and logistic regression. In this work, we propose a novel coreset construction via cardinality-constrained bilevel optimization. We show how… ▽ More Coresets are small data summaries that are sufficient for model training. They can be maintained online, enabling efficient handling of large data streams under resource constraints. However, existing constructions are limited to simple models such as k-means and logistic regression. In this work, we propose a novel coreset construction via cardinality-constrained bilevel optimization. We show how our framework can efficiently generate coresets for deep neural networks, and demonstrate its empirical benefits in continual learning and in streaming settings. △ Less

Submitted 22 October, 2020; v1 submitted 6 June, 2020; originally announced June 2020.

Comments: NeurIPS 2020

From Sets to Multisets: Provable Variational Inference for Probabilistic Integer Submodular Models

Authors: Aytunc Sahin, Yatao Bian, Joachim M. Buhmann, Andreas Krause

Abstract: Submodular functions have been studied extensively in machine learning and data mining. In particular, the optimization of submodular functions over the integer lattice (integer submodular functions) has recently attracted much interest, because this domain relates naturally to many practical problem settings, such as multilabel graph cut, budget allocation and revenue maximization with discrete a… ▽ More Submodular functions have been studied extensively in machine learning and data mining. In particular, the optimization of submodular functions over the integer lattice (integer submodular functions) has recently attracted much interest, because this domain relates naturally to many practical problem settings, such as multilabel graph cut, budget allocation and revenue maximization with discrete assignments. In contrast, the use of these functions for probabilistic modeling has received surprisingly little attention so far. In this work, we firstly propose the Generalized Multilinear Extension, a continuous DR-submodular extension for integer submodular functions. We study central properties of this extension and formulate a new probabilistic model which is defined through integer submodular functions. Then, we introduce a block-coordinate ascent algorithm to perform approximate inference for those class of models. Finally, we demonstrate its effectiveness and viability on several real-world social connection graph datasets with integer submodular objectives. △ Less

Submitted 1 June, 2020; originally announced June 2020.

MorphStore: Analytical Query Engine with a Holistic Compression-Enabled Processing Model

Authors: Patrick Damme, Annett Ungethüm, Johannes Pietrzyk, Alexander Krause, Dirk Habich, Wolfgang Lehner

Abstract: In this paper, we present MorphStore, an open-source in-memory columnar analytical query engine with a novel holistic compression-enabled processing model. Basically, compression using lightweight integer compression algorithms already plays an important role in existing in-memory column-store database systems, but mainly for base data. In particular, during query processing, these systems only ke… ▽ More In this paper, we present MorphStore, an open-source in-memory columnar analytical query engine with a novel holistic compression-enabled processing model. Basically, compression using lightweight integer compression algorithms already plays an important role in existing in-memory column-store database systems, but mainly for base data. In particular, during query processing, these systems only keep the data compressed until an operator cannot process the compressed data directly, whereupon the data is decompressed, but not recompressed. Thus, the full potential of compression during query processing is not exploited. To overcome that, we developed a novel compression-enabled processing model as presented in this paper. As we are going to show, the continuous usage of compression for all base data and all intermediates is very beneficial to reduce the overall memory footprint as well as to improve the query performance. △ Less

Submitted 20 April, 2020; originally announced April 2020.

Comments: Submitted to PVLDB

Hierarchical Image Classification using Entailment Cone Embeddings

Authors: Ankit Dhall, Anastasia Makarova, Octavian Ganea, Dario Pavllo, Michael Greeff, Andreas Krause

Abstract: Image classification has been studied extensively, but there has been limited work in using unconventional, external guidance other than traditional image-label pairs for training. We present a set of methods for leveraging information about the semantic hierarchy embedded in class labels. We first inject label-hierarchy knowledge into an arbitrary CNN-based classifier and empirically show that av… ▽ More Image classification has been studied extensively, but there has been limited work in using unconventional, external guidance other than traditional image-label pairs for training. We present a set of methods for leveraging information about the semantic hierarchy embedded in class labels. We first inject label-hierarchy knowledge into an arbitrary CNN-based classifier and empirically show that availability of such external semantic information in conjunction with the visual semantics from images boosts overall performance. Taking a step further in this direction, we model more explicitly the label-label and label-image interactions using order-preserving embeddings governed by both Euclidean and hyperbolic geometries, prevalent in natural language, and tailor them to hierarchical image classification and representation learning. We empirically validate all the models on the hierarchical ETHEC dataset. △ Less

Submitted 25 April, 2020; v1 submitted 2 April, 2020; originally announced April 2020.

Comments: Accepted in the CVPR 2020 Workshop on Differential Geometry in Computer Vision and Machine Learning

Anomaly Detection in Beehives using Deep Recurrent Autoencoders

Authors: Padraig Davidson, Michael Steininger, Florian Lautenschlager, Konstantin Kobs, Anna Krause, Andreas Hotho

Abstract: Precision beekee** allows to monitor bees' living conditions by equip** beehives with sensors. The data recorded by these hives can be analyzed by machine learning models to learn behavioral patterns of or search for unusual events in bee colonies. One typical target is the early detection of bee swarming as apiarists want to avoid this due to economical reasons. Advanced methods should be abl… ▽ More Precision beekee** allows to monitor bees' living conditions by equip** beehives with sensors. The data recorded by these hives can be analyzed by machine learning models to learn behavioral patterns of or search for unusual events in bee colonies. One typical target is the early detection of bee swarming as apiarists want to avoid this due to economical reasons. Advanced methods should be able to detect any other unusual or abnormal behavior arising from illness of bees or from technical reasons, e.g. sensor failure. In this position paper we present an autoencoder, a deep learning model, which detects any type of anomaly in data independent of its origin. Our model is able to reveal the same swarms as a simple rule-based swarm detection algorithm but is also triggered by any other anomaly. We evaluated our model on real world data sets that were collected on different hives and with different sensor setups. △ Less

Submitted 10 March, 2020; originally announced March 2020.

Journal ref: Proceedings of the 9th International Conference on Sensor Networks (SENSORNETS 2020), 2020, 142-149

SLEIPNIR: Deterministic and Provably Accurate Feature Expansion for Gaussian Process Regression with Derivatives

Authors: Emmanouil Angelis, Philippe Wenk, Bernhard Schölkopf, Stefan Bauer, Andreas Krause

Abstract: Gaussian processes are an important regression tool with excellent analytic properties which allow for direct integration of derivative observations. However, vanilla GP methods scale cubically in the amount of observations. In this work, we propose a novel approach for scaling GP regression with derivatives based on quadrature Fourier features. We then prove deterministic, non-asymptotic and expo… ▽ More Gaussian processes are an important regression tool with excellent analytic properties which allow for direct integration of derivative observations. However, vanilla GP methods scale cubically in the amount of observations. In this work, we propose a novel approach for scaling GP regression with derivatives based on quadrature Fourier features. We then prove deterministic, non-asymptotic and exponentially fast decaying error bounds which apply for both the approximated kernel as well as the approximated posterior. To furthermore illustrate the practical applicability of our method, we then apply it to ODIN, a recently developed algorithm for ODE parameter inference. In an extensive experiments section, all results are empirically validated, demonstrating the speed, accuracy, and practical applicability of this approach. △ Less

Submitted 5 March, 2020; originally announced March 2020.

Microservice Decomposition via Static and Dynamic Analysis of the Monolith

Authors: Alexander Krause, Christian Zirkelbach, Wilhelm Hasselbring, Stephan Lenga, Dan Kröger

Abstract: Migrating monolithic software systems into microservices requires the application of decomposition techniquesto find and select appropriate service boundaries. These techniques are often based on domain knowledge, static code analysis, and non-functional requirements such as maintainability. In this paper, we present our experience with an approach that extends static analysis with dynamic analysi… ▽ More Migrating monolithic software systems into microservices requires the application of decomposition techniquesto find and select appropriate service boundaries. These techniques are often based on domain knowledge, static code analysis, and non-functional requirements such as maintainability. In this paper, we present our experience with an approach that extends static analysis with dynamic analysis of a legacy software system's runtime behavior, including the live trace visualization to support the decomposition into microservices. Overall, our approach combines established analysis techniques for microservice decomposition, such as the bounded context pattern of domain-driven design, and enriches the collected information via dynamic software visualization to identify appropriate microservice boundaries. In collaboration with the German IT service provider adesso SE, we applied our approach to their real-word, legacy lottery application in|FOCUS to identify good microservice decompositions for this layered monolithic Enterprise Java system. △ Less

Submitted 5 March, 2020; originally announced March 2020.

Corruption-Tolerant Gaussian Process Bandit Optimization

Authors: Ilija Bogunovic, Andreas Krause, Jonathan Scarlett

Abstract: We consider the problem of optimizing an unknown (typically non-convex) function with a bounded norm in some Reproducing Kernel Hilbert Space (RKHS), based on noisy bandit feedback. We consider a novel variant of this problem in which the point evaluations are not only corrupted by random noise, but also adversarial corruptions. We introduce an algorithm Fast-Slow GP-UCB based on Gaussian process… ▽ More We consider the problem of optimizing an unknown (typically non-convex) function with a bounded norm in some Reproducing Kernel Hilbert Space (RKHS), based on noisy bandit feedback. We consider a novel variant of this problem in which the point evaluations are not only corrupted by random noise, but also adversarial corruptions. We introduce an algorithm Fast-Slow GP-UCB based on Gaussian process methods, randomized selection between two instances labeled "fast" (but non-robust) and "slow" (but robust), enlarged confidence bounds, and the principle of optimism under uncertainty. We present a novel theoretical analysis upper bounding the cumulative regret in terms of the corruption level, the time horizon, and the underlying kernel, and we argue that certain dependencies cannot be improved. We observe that distinct algorithmic ideas are required depending on whether one is required to perform well in both the corrupted and non-corrupted settings, and whether the corruption level is known or not. △ Less

Submitted 4 March, 2020; originally announced March 2020.

Comments: Accepted to AISTATS 2020

Unstaggered-staggered solitons on one- and two-dimensional two-component discrete nonlinear Schrödinger lattices

Authors: Robert A. Van Gorder, Andrew L. Krause, Boris A. Malomed, D. J. Kaup

Abstract: We study coupled unstaggered-staggered soliton pairs emergent from a system of two coupled discrete nonlinear Schrödinger (DNLS) equations with the self-attractive on-site self-phase-modulation nonlinearity, coupled by the repulsive cross-phase-modulation interaction, on 1D and 2D lattice domains. These mixed modes are of a "symbiotic" type, as each component in isolation may only carry ordinary u… ▽ More We study coupled unstaggered-staggered soliton pairs emergent from a system of two coupled discrete nonlinear Schrödinger (DNLS) equations with the self-attractive on-site self-phase-modulation nonlinearity, coupled by the repulsive cross-phase-modulation interaction, on 1D and 2D lattice domains. These mixed modes are of a "symbiotic" type, as each component in isolation may only carry ordinary unstaggered solitons. While most work on DNLS systems addressed symmetric on-site-centered fundamental solitons, these models give rise to a variety of other excited states, which may also be stable. The simplest among them are antisymmetric states in the form of discrete twisted solitons, which have no counterparts in the continuum limit. In the extension to 2D lattice domains, a natural counterpart of the twisted states are vortical solitons. We first introduce a variational approximation (VA) for the solitons, and then correct it numerically to construct exact stationary solutions, which are then used as initial conditions for simulations to check if the stationary states persist under time evolution. Two-component solutions obtained include (i) 1D fundamental-twisted and twisted-twisted soliton pairs, (ii) 2D fundamental-fundamental soliton pairs, and (iii) 2D vortical-vortical soliton pairs. We also highlight a variety of other transient dynamical regimes, such as breathers and amplitude death. The findings apply to modeling binary Bose-Einstein condensates, loaded in a deep lattice potential, with identical or different atomic masses of the two components, and arrays of bimodal optical waveguides. △ Less

Submitted 1 March, 2020; originally announced March 2020.

Comments: to be published in Communications in Nonlinear Science and Numerical Simulation

Journal ref: Communications in Nonlinear Science and Numerical Simulation 85 (2020) 105244

Mixed Strategies for Robust Optimization of Unknown Objectives

Authors: Pier Giuseppe Sessa, Ilija Bogunovic, Maryam Kamgarpour, Andreas Krause

Abstract: We consider robust optimization problems, where the goal is to optimize an unknown objective function against the worst-case realization of an uncertain parameter. For this setting, we design a novel sample-efficient algorithm GP-MRO, which sequentially learns about the unknown objective from noisy point evaluations. GP-MRO seeks to discover a robust and randomized mixed strategy, that maximizes t… ▽ More We consider robust optimization problems, where the goal is to optimize an unknown objective function against the worst-case realization of an uncertain parameter. For this setting, we design a novel sample-efficient algorithm GP-MRO, which sequentially learns about the unknown objective from noisy point evaluations. GP-MRO seeks to discover a robust and randomized mixed strategy, that maximizes the worst-case expected objective value. To achieve this, it combines techniques from online learning with nonparametric confidence bounds from Gaussian processes. Our theoretical results characterize the number of samples required by GP-MRO to discover a robust near-optimal mixed strategy for different GP kernels of interest. We experimentally demonstrate the performance of our algorithm on synthetic datasets and on human-assisted trajectory planning tasks for autonomous vehicles. In our simulations, we show that robust deterministic strategies can be overly conservative, while the mixed strategies found by GP-MRO significantly improve the overall performance. △ Less

Submitted 2 March, 2020; v1 submitted 28 February, 2020; originally announced February 2020.

Information Directed Sampling for Linear Partial Monitoring

Authors: Johannes Kirschner, Tor Lattimore, Andreas Krause

Abstract: Partial monitoring is a rich framework for sequential decision making under uncertainty that generalizes many well known bandit models, including linear, combinatorial and dueling bandits. We introduce information directed sampling (IDS) for stochastic partial monitoring with a linear reward and observation structure. IDS achieves adaptive worst-case regret rates that depend on precise observabili… ▽ More Partial monitoring is a rich framework for sequential decision making under uncertainty that generalizes many well known bandit models, including linear, combinatorial and dueling bandits. We introduce information directed sampling (IDS) for stochastic partial monitoring with a linear reward and observation structure. IDS achieves adaptive worst-case regret rates that depend on precise observability conditions of the game. Moreover, we prove lower bounds that classify the minimax regret of all finite games into four possible regimes. IDS achieves the optimal rate in all cases up to logarithmic factors, without tuning any hyper-parameters. We further extend our results to the contextual and the kernelized setting, which significantly increases the range of possible applications. △ Less

Submitted 25 February, 2020; originally announced February 2020.

Distributionally Robust Bayesian Optimization

Authors: Johannes Kirschner, Ilija Bogunovic, Stefanie Jegelka, Andreas Krause

Abstract: Robustness to distributional shift is one of the key challenges of contemporary machine learning. Attaining such robustness is the goal of distributionally robust optimization, which seeks a solution to an optimization problem that is worst-case robust under a specified distributional shift of an uncontrolled covariate. In this paper, we study such a problem when the distributional shift is measur… ▽ More Robustness to distributional shift is one of the key challenges of contemporary machine learning. Attaining such robustness is the goal of distributionally robust optimization, which seeks a solution to an optimization problem that is worst-case robust under a specified distributional shift of an uncontrolled covariate. In this paper, we study such a problem when the distributional shift is measured via the maximum mean discrepancy (MMD). For the setting of zeroth-order, noisy optimization, we present a novel distributionally robust Bayesian optimization algorithm (DRBO). Our algorithm provably obtains sub-linear robust regret in various settings that differ in how the uncertain covariate is observed. We demonstrate the robust performance of our method on both synthetic and real-world benchmarks. △ Less

Submitted 22 March, 2020; v1 submitted 20 February, 2020; originally announced February 2020.

Comments: Accepted at AISTATS 2020

PACOH: Bayes-Optimal Meta-Learning with PAC-Guarantees

Authors: Jonas Rothfuss, Vincent Fortuin, Martin Josifoski, Andreas Krause

Abstract: Meta-learning can successfully acquire useful inductive biases from data. Yet, its generalization properties to unseen learning tasks are poorly understood. Particularly if the number of meta-training tasks is small, this raises concerns about overfitting. We provide a theoretical analysis using the PAC-Bayesian framework and derive novel generalization bounds for meta-learning. Using these bounds… ▽ More Meta-learning can successfully acquire useful inductive biases from data. Yet, its generalization properties to unseen learning tasks are poorly understood. Particularly if the number of meta-training tasks is small, this raises concerns about overfitting. We provide a theoretical analysis using the PAC-Bayesian framework and derive novel generalization bounds for meta-learning. Using these bounds, we develop a class of PAC-optimal meta-learning algorithms with performance guarantees and a principled meta-level regularization. Unlike previous PAC-Bayesian meta-learners, our method results in a standard stochastic optimization problem which can be solved efficiently and scales well. When instantiating our PAC-optimal hyper-posterior (PACOH) with Gaussian processes and Bayesian Neural Networks as base learners, the resulting methods yield state-of-the-art performance, both in terms of predictive accuracy and the quality of uncertainty estimates. Thanks to their principled treatment of uncertainty, our meta-learners can also be successfully employed for sequential decision problems. △ Less

Submitted 18 June, 2021; v1 submitted 13 February, 2020; originally announced February 2020.

Comments: International Conference on Machine Learning (ICML) 2021

MSC Class: 68Q32

Witt vectors with coefficients and characteristic polynomials over non-commutative rings

Authors: Emanuele Dotto, Achim Krause, Thomas Nikolaus, Irakli Patchkoria

Abstract: For a not-necessarily commutative ring R we define an abelian group W(R;M) of Witt vectors with coefficients in an R-bimodule M. These groups generalize the usual big Witt vectors of commutative rings and we prove that they have analogous formal properties and structure. One main result is that W(R) := W(R;R) is Morita invariant in R. For an R-linear endomorphism f of a finitely generated projec… ▽ More For a not-necessarily commutative ring R we define an abelian group W(R;M) of Witt vectors with coefficients in an R-bimodule M. These groups generalize the usual big Witt vectors of commutative rings and we prove that they have analogous formal properties and structure. One main result is that W(R) := W(R;R) is Morita invariant in R. For an R-linear endomorphism f of a finitely generated projective R-module we define a characteristic element $χ_f \in W(R)$. This element is a non-commutative analogue of the classical characteristic polynomial and we show that it has similar properties. The assignment $f \mapsto χ_f$ induces an isomorphism between a suitable completion of cyclic K-theory and W(R). △ Less

Submitted 4 February, 2020; originally announced February 2020.

Comments: 39 pages

Log Barriers for Safe Non-convex Black-box Optimization

Authors: Ilnura Usmanova, Andreas Krause, Maryam Kamgarpour

Abstract: We address the problem of minimizing a smooth function $f^0(x)$ over a compact set $D$ defined by smooth functional constraints $f^i(x)\leq 0,~ i = 1,\ldots, m$ given noisy value measurements of $f^i(x)$. This problem arises in safety-critical applications, where certain parameters need to be adapted online in a data-driven fashion, such as in personalized medicine, robotics, manufacturing, etc. I… ▽ More We address the problem of minimizing a smooth function $f^0(x)$ over a compact set $D$ defined by smooth functional constraints $f^i(x)\leq 0,~ i = 1,\ldots, m$ given noisy value measurements of $f^i(x)$. This problem arises in safety-critical applications, where certain parameters need to be adapted online in a data-driven fashion, such as in personalized medicine, robotics, manufacturing, etc. In such cases, it is important to ensure constraints are not violated while taking measurements and seeking the minimum of the cost function. We propose a new algorithm s0-LBM, which provides provably feasible iterates with high probability and applies to the challenging case of uncertain zero-th order oracle. We also analyze the convergence rate of the algorithm, and empirically demonstrate its effectiveness. △ Less

Submitted 19 December, 2019; originally announced December 2019.

Comments: under review

Safe non-smooth black-box optimization with application to policy search

Authors: Ilnura Usmanova, Andreas Krause, Maryam Kamgarpour

Abstract: For safety-critical black-box optimization tasks, observations of the constraints and the objective are often noisy and available only for the feasible points. We propose an approach based on log barriers to find a local solution of a non-convex non-smooth black-box optimization problem $\min f^0(x)$ subject to $f^i(x)\leq 0,~ i = 1,\ldots, m$, at the same time, guaranteeing constraint satisfactio… ▽ More For safety-critical black-box optimization tasks, observations of the constraints and the objective are often noisy and available only for the feasible points. We propose an approach based on log barriers to find a local solution of a non-convex non-smooth black-box optimization problem $\min f^0(x)$ subject to $f^i(x)\leq 0,~ i = 1,\ldots, m$, at the same time, guaranteeing constraint satisfaction while learning an optimal solution with high probability. Our proposed algorithm exploits noisy observations to iteratively improve on an initial safe point until convergence. We derive the convergence rate and prove safety of our algorithm. We demonstrate its performance in an application to an iterative control design problem. △ Less

Submitted 23 February, 2021; v1 submitted 19 December, 2019; originally announced December 2019.

A Human-in-the-loop Framework to Construct Context-aware Mathematical Notions of Outcome Fairness

Authors: Mohammad Yaghini, Andreas Krause, Hoda Heidari

Abstract: Existing mathematical notions of fairness fail to account for the context of decision-making. We argue that moral consideration of contextual factors is an inherently human task. So we present a framework to learn context-aware mathematical formulations of fairness by eliciting people's situated fairness assessments. Our family of fairness notions corresponds to a new interpretation of economic mo… ▽ More Existing mathematical notions of fairness fail to account for the context of decision-making. We argue that moral consideration of contextual factors is an inherently human task. So we present a framework to learn context-aware mathematical formulations of fairness by eliciting people's situated fairness assessments. Our family of fairness notions corresponds to a new interpretation of economic models of Equality of Opportunity (EOP), and it includes most existing notions of fairness as special cases. Our human-in-the-loop approach is designed to learn the appropriate parameters of the EOP family by utilizing human responses to pair-wise questions about decision subjects' circumstance and deservingness, and the harm/benefit imposed on them. We illustrate our framework in a hypothetical criminal risk assessment scenario by conducting a series of human-subject experiments on Amazon Mechanical Turk. Our work takes an important initial step toward empowering stakeholders to have a voice in the formulation of fairness for Machine Learning. △ Less

Submitted 18 May, 2021; v1 submitted 7 November, 2019; originally announced November 2019.

Comments: In the forth AAAI/ACM Conference on Artificial Intelligence, Ethics, and Society (AIES-2021)

Safe Exploration for Interactive Machine Learning

Authors: Matteo Turchetta, Felix Berkenkamp, Andreas Krause

Abstract: In Interactive Machine Learning (IML), we iteratively make decisions and obtain noisy observations of an unknown function. While IML methods, e.g., Bayesian optimization and active learning, have been successful in applications, on real-world systems they must provably avoid unsafe decisions. To this end, safe IML algorithms must carefully learn about a priori unknown constraints without making un… ▽ More In Interactive Machine Learning (IML), we iteratively make decisions and obtain noisy observations of an unknown function. While IML methods, e.g., Bayesian optimization and active learning, have been successful in applications, on real-world systems they must provably avoid unsafe decisions. To this end, safe IML algorithms must carefully learn about a priori unknown constraints without making unsafe decisions. Existing algorithms for this problem learn about the safety of all decisions to ensure convergence. This is sample-inefficient, as it explores decisions that are not relevant for the original IML objective. In this paper, we introduce a novel framework that renders any existing unsafe IML algorithm safe. Our method works as an add-on that takes suggested decisions as input and exploits regularity assumptions in terms of a Gaussian process prior in order to efficiently learn about their safety. As a result, we only explore the safe set when necessary for the IML problem. We apply our framework to safe Bayesian optimization and to safe exploration in deterministic Markov Decision Processes (MDP), which have been analyzed separately before. Our method outperforms other algorithms empirically. △ Less

Submitted 30 October, 2019; originally announced October 2019.

Comments: Accepted at NeurIPS 2019

Robust Model-free Reinforcement Learning with Multi-objective Bayesian Optimization

Authors: Matteo Turchetta, Andreas Krause, Sebastian Trimpe

Abstract: In reinforcement learning (RL), an autonomous agent learns to perform complex tasks by maximizing an exogenous reward signal while interacting with its environment. In real-world applications, test conditions may differ substantially from the training scenario and, therefore, focusing on pure reward maximization during training may lead to poor results at test time. In these cases, it is important… ▽ More In reinforcement learning (RL), an autonomous agent learns to perform complex tasks by maximizing an exogenous reward signal while interacting with its environment. In real-world applications, test conditions may differ substantially from the training scenario and, therefore, focusing on pure reward maximization during training may lead to poor results at test time. In these cases, it is important to trade-off between performance and robustness while learning a policy. While several results exist for robust, model-based RL, the model-free case has not been widely investigated. In this paper, we cast the robust, model-free RL problem as a multi-objective optimization problem. To quantify the robustness of a policy, we use delay margin and gain margin, two robustness indicators that are common in control theory. We show how these metrics can be estimated from data in the model-free setting. We use multi-objective Bayesian optimization (MOBO) to solve efficiently this expensive-to-evaluate, multi-objective optimization problem. We show the benefits of our robust formulation both in sim-to-real and pure hardware experiments to balance a Furuta pendulum. △ Less

Submitted 29 October, 2019; originally announced October 2019.

Comments: Submitted to IEEE Conference on Robotics and Automation 2020 (ICRA)

Adaptive Sampling for Stochastic Risk-Averse Learning

Authors: Sebastian Curi, Kfir. Y. Levy, Stefanie Jegelka, Andreas Krause

Abstract: In high-stakes machine learning applications, it is crucial to not only perform well on average, but also when restricted to difficult examples. To address this, we consider the problem of training models in a risk-averse manner. We propose an adaptive sampling algorithm for stochastically optimizing the Conditional Value-at-Risk (CVaR) of a loss distribution, which measures its performance on the… ▽ More In high-stakes machine learning applications, it is crucial to not only perform well on average, but also when restricted to difficult examples. To address this, we consider the problem of training models in a risk-averse manner. We propose an adaptive sampling algorithm for stochastically optimizing the Conditional Value-at-Risk (CVaR) of a loss distribution, which measures its performance on the $α$ fraction of most difficult examples. We use a distributionally robust formulation of the CVaR to phrase the problem as a zero-sum game between two players, and solve it efficiently using regret minimization. Our approach relies on sampling from structured Determinantal Point Processes (DPPs), which enables scaling it to large data sets. Finally, we empirically demonstrate its effectiveness on large-scale convex and non-convex learning tasks. △ Less

Submitted 6 November, 2020; v1 submitted 28 October, 2019; originally announced October 2019.

Convergence Analysis of Block Coordinate Algorithms with Determinantal Sampling

Authors: Mojmír Mutný, Michał Dereziński, Andreas Krause

Abstract: We analyze the convergence rate of the randomized Newton-like method introduced by Qu et. al. (2016) for smooth and convex objectives, which uses random coordinate blocks of a Hessian-over-approximation matrix $\bM$ instead of the true Hessian. The convergence analysis of the algorithm is challenging because of its complex dependence on the structure of $\bM$. However, we show that when the coordi… ▽ More We analyze the convergence rate of the randomized Newton-like method introduced by Qu et. al. (2016) for smooth and convex objectives, which uses random coordinate blocks of a Hessian-over-approximation matrix $\bM$ instead of the true Hessian. The convergence analysis of the algorithm is challenging because of its complex dependence on the structure of $\bM$. However, we show that when the coordinate blocks are sampled with probability proportional to their determinant, the convergence rate depends solely on the eigenvalue distribution of matrix $\bM$, and has an analytically tractable form. To do so, we derive a fundamental new expectation formula for determinantal point processes. We show that determinantal sampling allows us to reason about the optimal subset size of blocks in terms of the spectrum of $\bM$. Additionally, we provide a numerical evaluation of our analysis, demonstrating cases where determinantal sampling is superior or on par with uniform sampling. △ Less

Submitted 12 February, 2020; v1 submitted 25 October, 2019; originally announced October 2019.

Journal ref: AISTATS 2020

No-Regret Learning in Unknown Games with Correlated Payoffs

Authors: Pier Giuseppe Sessa, Ilija Bogunovic, Maryam Kamgarpour, Andreas Krause

Abstract: We consider the problem of learning to play a repeated multi-agent game with an unknown reward function. Single player online learning algorithms attain strong regret bounds when provided with full information feedback, which unfortunately is unavailable in many real-world scenarios. Bandit feedback alone, i.e., observing outcomes only for the selected action, yields substantially worse performanc… ▽ More We consider the problem of learning to play a repeated multi-agent game with an unknown reward function. Single player online learning algorithms attain strong regret bounds when provided with full information feedback, which unfortunately is unavailable in many real-world scenarios. Bandit feedback alone, i.e., observing outcomes only for the selected action, yields substantially worse performance. In this paper, we consider a natural model where, besides a noisy measurement of the obtained reward, the player can also observe the opponents' actions. This feedback model, together with a regularity assumption on the reward function, allows us to exploit the correlations among different game outcomes by means of Gaussian processes (GPs). We propose a novel confidence-bound based bandit algorithm GP-MW, which utilizes the GP model for the reward function and runs a multiplicative weight (MW) method. We obtain novel kernel-dependent regret bounds that are comparable to the known bounds in the full information setting, while substantially improving upon the existing bandit results. We experimentally demonstrate the effectiveness of GP-MW in random matrix games, as well as real-world problems of traffic routing and movie recommendation. In our experiments, GP-MW consistently outperforms several baselines, while its performance is often comparable to methods that have access to full information feedback. △ Less

Submitted 28 October, 2019; v1 submitted 18 September, 2019; originally announced September 2019.

From One Pattern into Another: Analysis of Turing Patterns in Heterogeneous Domains via WKBJ

Authors: Andrew L. Krause, Václav Klika, Thomas E. Woolley, Eamonn A. Gaffney

Abstract: Pattern formation from homogeneity is well-studied, but less is known concerning symmetry-breaking instabilities in heterogeneous media. It is nontrivial to separate observed spatial patterning due to inherent spatial heterogeneity from emergent patterning due to nonlinear instability. We employ WKBJ asymptotics to investigate Turing instabilities for a spatially heterogeneous reaction-diffusion s… ▽ More Pattern formation from homogeneity is well-studied, but less is known concerning symmetry-breaking instabilities in heterogeneous media. It is nontrivial to separate observed spatial patterning due to inherent spatial heterogeneity from emergent patterning due to nonlinear instability. We employ WKBJ asymptotics to investigate Turing instabilities for a spatially heterogeneous reaction-diffusion system, and derive conditions for instability which are local versions of the classical Turing conditions We find that the structure of unstable modes differs substantially from the typical trigonometric functions seen in the spatially homogeneous setting. Modes of different growth rates are localized to different spatial regions. This localization helps explain common amplitude modulations observed in simulations of Turing systems in heterogeneous settings. We numerically demonstrate this theory, giving an illustrative example of the emergent instabilities and the striking complexity arising from spatially heterogeneous reaction-diffusion systems. Our results give insight both into systems driven by exogenous heterogeneity, as well as successive pattern forming processes, noting that most scenarios in biology do not involve symmetry breaking from homogeneity, but instead consist of sequential evolutions of heterogeneous states. The instability mechanism reported here precisely captures such evolution, and extends Turing's original thesis to a far wider and more realistic class of systems. △ Less

Submitted 7 December, 2019; v1 submitted 20 August, 2019; originally announced August 2019.

Comments: 23 pages, 7 Figures

Schauder Bases Having Many Good Block Basic Sequences

Authors: Cory A. Krause

Abstract: In the study of asymptotic geometry in Banach spaces, a basic sequence which gives rise to a spreading model has been called a good sequence. It is well known that every normalized basic sequence in a Banach space has a subsequence which is good. We investigate the assumption that every normalized block tree relative to a basis has a branch which is good. This combinatorial property turns out to b… ▽ More In the study of asymptotic geometry in Banach spaces, a basic sequence which gives rise to a spreading model has been called a good sequence. It is well known that every normalized basic sequence in a Banach space has a subsequence which is good. We investigate the assumption that every normalized block tree relative to a basis has a branch which is good. This combinatorial property turns out to be very strong and is equivalent to the space being $1$-asymptotic $\ell_p$ for some $1\leq p\leq\infty$. We also investigate the even stronger assumption that every block basic sequence of a basis is good. Finally, using the Hindman-Milliken-Taylor theorem, we prove a stabilization theorem which produces a basic sequence all of whose normalized constant coefficient block basic sequences are good, and we present an application of this stabilization. △ Less

Submitted 8 January, 2020; v1 submitted 27 July, 2019; originally announced July 2019.

Comments: 21 pages

MSC Class: Primary 46B03; 46B06; 46B25; 46B45; Secondary 05D10

Noise Regularization for Conditional Density Estimation

Authors: Jonas Rothfuss, Fabio Ferreira, Simon Boehm, Simon Walther, Maxim Ulrich, Tamim Asfour, Andreas Krause

Abstract: Modelling statistical relationships beyond the conditional mean is crucial in many settings. Conditional density estimation (CDE) aims to learn the full conditional probability density from data. Though highly expressive, neural network based CDE models can suffer from severe over-fitting when trained with the maximum likelihood objective. Due to the inherent structure of such models, classical re… ▽ More Modelling statistical relationships beyond the conditional mean is crucial in many settings. Conditional density estimation (CDE) aims to learn the full conditional probability density from data. Though highly expressive, neural network based CDE models can suffer from severe over-fitting when trained with the maximum likelihood objective. Due to the inherent structure of such models, classical regularization approaches in the parameter space are rendered ineffective. To address this issue, we develop a model-agnostic noise regularization method for CDE that adds random perturbations to the data during training. We demonstrate that the proposed approach corresponds to a smoothness regularization and prove its asymptotic consistency. In our experiments, noise regularization significantly and consistently outperforms other regularization methods across seven data sets and three CDE models. The effectiveness of noise regularization makes neural network based CDE the preferable method over previous non- and semi-parametric approaches, even when training data is scarce. △ Less

Submitted 14 February, 2020; v1 submitted 21 July, 2019; originally announced July 2019.

Structured Variational Inference in Unstable Gaussian Process State Space Models

Authors: Silvan Melchior, Sebastian Curi, Felix Berkenkamp, Andreas Krause

Abstract: We propose a new variational inference algorithm for learning in Gaussian Process State-Space Models (GPSSMs). Our algorithm enables learning of unstable and partially observable systems, where previous algorithms fail. Our main algorithmic contribution is a novel approximate posterior that can be calculated efficiently using a single forward and backward pass along the training trajectories. The… ▽ More We propose a new variational inference algorithm for learning in Gaussian Process State-Space Models (GPSSMs). Our algorithm enables learning of unstable and partially observable systems, where previous algorithms fail. Our main algorithmic contribution is a novel approximate posterior that can be calculated efficiently using a single forward and backward pass along the training trajectories. The forward-backward pass is inspired on Kalman smoothing for linear dynamical systems but generalizes to GPSSMs. Our second contribution is a modification of the conditioning step that effectively lowers the Kalman gain. This modification is crucial to attaining good test performance where no measurements are available. Finally, we show experimentally that our learning algorithm performs well in stable and unstable real systems with hidden states. △ Less

Submitted 10 June, 2020; v1 submitted 16 July, 2019; originally announced July 2019.

Modularization of Research Software for Collaborative Open Source Development

Authors: Christian Zirkelbach, Alexander Krause, Wilhelm Hasselbring

Abstract: Software systems evolve over their lifetime. Changing conditions, such as requirements or customer requests make it inevitable for developers to perform adjustments to the underlying code base. Especially in the context of open source software where everybody can contribute, requirements can change over time and new user groups may be addressed. In particular, research software is often not struct… ▽ More Software systems evolve over their lifetime. Changing conditions, such as requirements or customer requests make it inevitable for developers to perform adjustments to the underlying code base. Especially in the context of open source software where everybody can contribute, requirements can change over time and new user groups may be addressed. In particular, research software is often not structured with a maintainable and extensible architecture. In combination with obsolescent technologies, this is a challenging task for new developers, especially, when students are involved. In this paper, we report on the modularization process and architecture of our open source research project ExplorViz towards a microservice architecture. The new architecture facilitates a collaborative development process for both researchers and students. We describe the modularization measures and present how we solved occurring issues and enhanced our development process. Afterwards, we illustrate our modularization approach with our modernized, extensible software system architecture and highlight the improved collaborative development process. Finally, we present a proof-of-concept implementation featuring several developed extensions in terms of architecture and extensibility. △ Less

Submitted 12 July, 2019; originally announced July 2019.

Comments: 8 pages

Journal ref: COLLA 2019: The Ninth International Conference on Advanced Collaborative Networks, Systems and Applications

Bökstedt periodicity and quotients of DVRs

Authors: Achim Krause, Thomas Nikolaus

Abstract: In this note we compute the topological Hochschild homology of quotients of DVRs. Along the way we give a short argument for Bökstedt periodicity and generalizations over various other bases. Our strategy also gives a very efficient way to redo the computations of THH (resp. logarithmic THH) of complete DVRs originally due to Lindenstrauss-Madsen (resp. Hesselholt-Madsen). In this note we compute the topological Hochschild homology of quotients of DVRs. Along the way we give a short argument for Bökstedt periodicity and generalizations over various other bases. Our strategy also gives a very efficient way to redo the computations of THH (resp. logarithmic THH) of complete DVRs originally due to Lindenstrauss-Madsen (resp. Hesselholt-Madsen). △ Less

Submitted 8 July, 2019; originally announced July 2019.

Comments: 35 pages

Mixed-Variable Bayesian Optimization

Authors: Erik Daxberger, Anastasia Makarova, Matteo Turchetta, Andreas Krause

Abstract: The optimization of expensive to evaluate, black-box, mixed-variable functions, i.e. functions that have continuous and discrete inputs, is a difficult and yet pervasive problem in science and engineering. In Bayesian optimization (BO), special cases of this problem that consider fully continuous or fully discrete domains have been widely studied. However, few methods exist for mixed-variable doma… ▽ More The optimization of expensive to evaluate, black-box, mixed-variable functions, i.e. functions that have continuous and discrete inputs, is a difficult and yet pervasive problem in science and engineering. In Bayesian optimization (BO), special cases of this problem that consider fully continuous or fully discrete domains have been widely studied. However, few methods exist for mixed-variable domains and none of them can handle discrete constraints that arise in many real-world applications. In this paper, we introduce MiVaBo, a novel BO algorithm for the efficient optimization of mixed-variable functions combining a linear surrogate model based on expressive feature representations with Thompson sampling. We propose an effective method to optimize its acquisition function, a challenging problem for mixed-variable domains, making MiVaBo the first BO method that can handle complex constraints over the discrete variables. Moreover, we provide the first convergence analysis of a mixed-variable BO algorithm. Finally, we show that MiVaBo is significantly more sample efficient than state-of-the-art mixed-variable BO algorithms on several hyperparameter tuning tasks, including the tuning of deep generative models. △ Less

Submitted 4 August, 2020; v1 submitted 2 July, 2019; originally announced July 2019.

Comments: IJCAI 2020 camera-ready; 17 pages, extended version with supplementary material

Journal ref: Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence (IJCAI-20), 2020, pages 2633-2639

Learning-based Model Predictive Control for Safe Exploration and Reinforcement Learning

Authors: Torsten Koller, Felix Berkenkamp, Matteo Turchetta, Joschka Boedecker, Andreas Krause

Abstract: Reinforcement learning has been successfully used to solve difficult tasks in complex unknown environments. However, these methods typically do not provide any safety guarantees during the learning process. This is particularly problematic, since reinforcement learning agent actively explore their environment. This prevents their use in safety-critical, real-world applications. In this paper, we p… ▽ More Reinforcement learning has been successfully used to solve difficult tasks in complex unknown environments. However, these methods typically do not provide any safety guarantees during the learning process. This is particularly problematic, since reinforcement learning agent actively explore their environment. This prevents their use in safety-critical, real-world applications. In this paper, we present a learning-based model predictive control scheme that provides high-probability safety guarantees throughout the learning process. Based on a reliable statistical model, we construct provably accurate confidence intervals on predicted trajectories. Unlike previous approaches, we allow for input-dependent uncertainties. Based on these reliable predictions, we guarantee that trajectories satisfy safety constraints. Moreover, we use a terminal set constraint to recursively guarantee the existence of safe control actions at every iteration. We evaluate the resulting algorithm to safely explore the dynamics of an inverted pendulum and to solve a reinforcement learning task on a cart-pole system with safety constraints. △ Less

Submitted 27 June, 2019; originally announced June 2019.

Comments: 14 pages, 7 figures. arXiv admin note: substantial text overlap with arXiv:1803.08287

Safe Contextual Bayesian Optimization for Sustainable Room Temperature PID Control Tuning

Authors: Marcello Fiducioso, Sebastian Curi, Benedikt Schumacher, Markus Gwerder, Andreas Krause

Abstract: We tune one of the most common heating, ventilation, and air conditioning (HVAC) control loops, namely the temperature control of a room. For economical and environmental reasons, it is of prime importance to optimize the performance of this system. Buildings account from 20 to 40% of a country energy consumption, and almost 50% of it comes from HVAC systems. Scenario projections predict a 30% dec… ▽ More We tune one of the most common heating, ventilation, and air conditioning (HVAC) control loops, namely the temperature control of a room. For economical and environmental reasons, it is of prime importance to optimize the performance of this system. Buildings account from 20 to 40% of a country energy consumption, and almost 50% of it comes from HVAC systems. Scenario projections predict a 30% decrease in heating consumption by 2050 due to efficiency increase. Advanced control techniques can improve performance; however, the proportional-integral-derivative (PID) control is typically used due to its simplicity and overall performance. We use Safe Contextual Bayesian Optimization to optimize the PID parameters without human intervention. We reduce costs by 32% compared to the current PID controller setting while assuring safety and comfort to people in the room. The results of this work have an immediate impact on the room control loop performances and its related commissioning costs. Furthermore, this successful attempt paves the way for further use at different levels of HVAC systems, with promising energy, operational, and commissioning costs savings, and it is a practical demonstration of the positive effects that Artificial Intelligence can have on environmental sustainability. △ Less

Submitted 28 June, 2019; originally announced June 2019.

Comments: 7 pages; Published in IJCAI 19