-
Turing and wave instabilities in hyperbolic reaction-diffusion systems: The role of second-order time derivatives and cross-diffusion terms on pattern formation
Authors:
Joshua Ritchie,
Andrew L. Krause,
Robert A. Van Gorder
Abstract:
Hyperbolic reaction-diffusion equations have recently attracted attention both for their application to a variety of biological and chemical phenomena, and for their distinct features in terms of propagation speed and novel instabilities not present in classical two-species reaction-diffusion systems. We explore the onset of diffusive instabilities and resulting pattern formation for such systems.…
▽ More
Hyperbolic reaction-diffusion equations have recently attracted attention both for their application to a variety of biological and chemical phenomena, and for their distinct features in terms of propagation speed and novel instabilities not present in classical two-species reaction-diffusion systems. We explore the onset of diffusive instabilities and resulting pattern formation for such systems. Starting with a rather general formulation of the problem, we obtain necessary and sufficient conditions for the Turing and wave instabilities in such systems, thereby classifying parameter spaces for which these diffusive instabilities occur. We find that the additional temporal terms do not strongly modify the Turing patterns which form or parameters which admit them, but only their regions of existence. This is in contrast to the case of additional space derivatives, where past work has shown that resulting patterned structures are sensitive to second-order cross-diffusion and first-order advection. We also show that additional temporal terms are necessary for the emergence of spatiotemporal patterns under the wave instability. We find that such wave instabilities exist for parameters which are mutually exclusive to those parameters leading to stationary Turing patterns. This implies that wave instabilities may occur in cases where the activator diffuses faster than the inhibitor, leading to routes to spatial symmetry breaking in reaction-diffusion systems which are distinct from the well studied Turing case.
△ Less
Submitted 28 April, 2022;
originally announced April 2022.
-
Gradient-Based Trajectory Optimization With Learned Dynamics
Authors:
Bhavya Sukhija,
Nathanael Köhler,
Miguel Zamora,
Simon Zimmermann,
Sebastian Curi,
Andreas Krause,
Stelian Coros
Abstract:
Trajectory optimization methods have achieved an exceptional level of performance on real-world robots in recent years. These methods heavily rely on accurate analytical models of the dynamics, yet some aspects of the physical world can only be captured to a limited extent. An alternative approach is to leverage machine learning techniques to learn a differentiable dynamics model of the system fro…
▽ More
Trajectory optimization methods have achieved an exceptional level of performance on real-world robots in recent years. These methods heavily rely on accurate analytical models of the dynamics, yet some aspects of the physical world can only be captured to a limited extent. An alternative approach is to leverage machine learning techniques to learn a differentiable dynamics model of the system from data. In this work, we use trajectory optimization and model learning for performing highly dynamic and complex tasks with robotic systems in absence of accurate analytical models of the dynamics. We show that a neural network can model highly nonlinear behaviors accurately for large time horizons, from data collected in only 25 minutes of interactions on two distinct robots: (i) the Boston Dynamics Spot and an (ii) RC car. Furthermore, we use the gradients of the neural network to perform gradient-based trajectory optimization. In our hardware experiments, we demonstrate that our learned model can represent complex dynamics for both the Spot and Radio-controlled (RC) car, and gives good performance in combination with trajectory optimization methods.
△ Less
Submitted 25 June, 2023; v1 submitted 9 April, 2022;
originally announced April 2022.
-
On the K-theory of $\mathbb{Z}/p^n$ -- announcement
Authors:
Benjamin Antieau,
Achim Krause,
Thomas Nikolaus
Abstract:
We announce new methods for using prismatic cohomology to compute the K-groups of $\mathbb{Z}/p^n$ and related rings. We use computer algebra methods to compute these K-groups through a large range in specific cases and also obtain explicit formulas for their orders in large degrees.
We announce new methods for using prismatic cohomology to compute the K-groups of $\mathbb{Z}/p^n$ and related rings. We use computer algebra methods to compute these K-groups through a large range in specific cases and also obtain explicit formulas for their orders in large degrees.
△ Less
Submitted 7 April, 2022;
originally announced April 2022.
-
Multi-Scale Representation Learning on Proteins
Authors:
Vignesh Ram Somnath,
Charlotte Bunne,
Andreas Krause
Abstract:
Proteins are fundamental biological entities mediating key roles in cellular function and disease. This paper introduces a multi-scale graph construction of a protein -- HoloProt -- connecting surface to structure and sequence. The surface captures coarser details of the protein, while sequence as primary component and structure -- comprising secondary and tertiary components -- capture finer deta…
▽ More
Proteins are fundamental biological entities mediating key roles in cellular function and disease. This paper introduces a multi-scale graph construction of a protein -- HoloProt -- connecting surface to structure and sequence. The surface captures coarser details of the protein, while sequence as primary component and structure -- comprising secondary and tertiary components -- capture finer details. Our graph encoder then learns a multi-scale representation by allowing each level to integrate the encoding from level(s) below with the graph at that level. We test the learned representation on different tasks, (i.) ligand binding affinity (regression), and (ii.) protein function prediction (classification). On the regression task, contrary to previous methods, our model performs consistently and reliably across different dataset splits, outperforming all baselines on most splits. On the classification task, it achieves a performance close to the top-performing model while using 10x fewer parameters. To improve the memory efficiency of our construction, we segment the multiplex protein surface manifold into molecular superpixels and substitute the surface with these superpixels at little to no performance loss.
△ Less
Submitted 4 April, 2022;
originally announced April 2022.
-
Tuning Particle Accelerators with Safety Constraints using Bayesian Optimization
Authors:
Johannes Kirschner,
Mojmir Mutný,
Andreas Krause,
Jaime Coello de Portugal,
Nicole Hiller,
Jochem Snuverink
Abstract:
Tuning machine parameters of particle accelerators is a repetitive and time-consuming task that is challenging to automate. While many off-the-shelf optimization algorithms are available, in practice their use is limited because most methods do not account for safety-critical constraints in each iteration, such as loss signals or step-size limitations. One notable exception is safe Bayesian optimi…
▽ More
Tuning machine parameters of particle accelerators is a repetitive and time-consuming task that is challenging to automate. While many off-the-shelf optimization algorithms are available, in practice their use is limited because most methods do not account for safety-critical constraints in each iteration, such as loss signals or step-size limitations. One notable exception is safe Bayesian optimization, which is a data-driven tuning approach for global optimization with noisy feedback. We propose and evaluate a step-size limited variant of safe Bayesian optimization on two research facilities of the Paul Scherrer Institut (PSI): a) the Swiss Free Electron Laser (SwissFEL) and b) the High-Intensity Proton Accelerator (HIPA). We report promising experimental results on both machines, tuning up to 16 parameters subject to 224 constraints.
△ Less
Submitted 30 June, 2022; v1 submitted 25 March, 2022;
originally announced March 2022.
-
Efficient Model-based Multi-agent Reinforcement Learning via Optimistic Equilibrium Computation
Authors:
Pier Giuseppe Sessa,
Maryam Kamgarpour,
Andreas Krause
Abstract:
We consider model-based multi-agent reinforcement learning, where the environment transition model is unknown and can only be learned via expensive interactions with the environment. We propose H-MARL (Hallucinated Multi-Agent Reinforcement Learning), a novel sample-efficient algorithm that can efficiently balance exploration, i.e., learning about the environment, and exploitation, i.e., achieve g…
▽ More
We consider model-based multi-agent reinforcement learning, where the environment transition model is unknown and can only be learned via expensive interactions with the environment. We propose H-MARL (Hallucinated Multi-Agent Reinforcement Learning), a novel sample-efficient algorithm that can efficiently balance exploration, i.e., learning about the environment, and exploitation, i.e., achieve good equilibrium performance in the underlying general-sum Markov game. H-MARL builds high-probability confidence intervals around the unknown transition model and sequentially updates them based on newly observed data. Using these, it constructs an optimistic hallucinated game for the agents for which equilibrium policies are computed at each round. We consider general statistical models (e.g., Gaussian processes, deep ensembles, etc.) and policy classes (e.g., deep neural networks), and theoretically analyze our approach by bounding the agents' dynamic regret. Moreover, we provide a convergence rate to the equilibria of the underlying Markov game. We demonstrate our approach experimentally on an autonomous driving simulation benchmark. H-MARL learns successful equilibrium policies after a few interactions with the environment and can significantly improve the performance compared to non-optimistic exploration methods.
△ Less
Submitted 10 July, 2022; v1 submitted 14 March, 2022;
originally announced March 2022.
-
A Novel Physics-Regularized Interpretable Machine Learning Model for Grain Growth
Authors:
Weishi Yan,
Joseph Melville,
Vishal Yadav,
Kristien Everett,
Lin Yang,
Michael S. Kesler,
Amanda R. Krause,
Michael R. Tonks,
Joel B. Harley
Abstract:
Experimental grain growth observations often deviate from grain growth simulations, revealing that the governing rules for grain boundary motion are not fully understood. A novel deep learning model was developed to capture grain growth behavior from training data without making assumptions about the underlying physics. The Physics-Regularized Interpretable Machine Learning Microstructure Evolutio…
▽ More
Experimental grain growth observations often deviate from grain growth simulations, revealing that the governing rules for grain boundary motion are not fully understood. A novel deep learning model was developed to capture grain growth behavior from training data without making assumptions about the underlying physics. The Physics-Regularized Interpretable Machine Learning Microstructure Evolution (PRIMME) model consists of a multi-layer neural network that predicts the likelihood of a point changing to a neighboring grain. Here, we demonstrate PRIMME's ability to replicate two-dimensional normal grain growth by training it with Monte Carlo Potts simulations. The trained PRIMME model's grain growth predictions in several test cases show good agreement with analytical models, phase-field simulations, Monte Carlo Potts simulations, and results from the literature. Additionally, PRIMME's adaptability to investigate irregular grain growth behavior is shown. Important aspects of PRIMME like interpretability, regularization, extrapolation, and overfitting are also discussed.
△ Less
Submitted 17 August, 2022; v1 submitted 7 March, 2022;
originally announced March 2022.
-
The Schrödinger Bridge between Gaussian Measures has a Closed Form
Authors:
Charlotte Bunne,
Ya-** Hsieh,
Marco Cuturi,
Andreas Krause
Abstract:
The static optimal transport $(\mathrm{OT})$ problem between Gaussians seeks to recover an optimal map, or more generally a coupling, to morph a Gaussian into another. It has been well studied and applied to a wide variety of tasks. Here we focus on the dynamic formulation of OT, also known as the Schrödinger bridge (SB) problem, which has recently seen a surge of interest in machine learning due…
▽ More
The static optimal transport $(\mathrm{OT})$ problem between Gaussians seeks to recover an optimal map, or more generally a coupling, to morph a Gaussian into another. It has been well studied and applied to a wide variety of tasks. Here we focus on the dynamic formulation of OT, also known as the Schrödinger bridge (SB) problem, which has recently seen a surge of interest in machine learning due to its connections with diffusion-based generative models. In contrast to the static setting, much less is known about the dynamic setting, even for Gaussian distributions. In this paper, we provide closed-form expressions for SBs between Gaussian measures. In contrast to the static Gaussian OT problem, which can be simply reduced to studying convex programs, our framework for solving SBs requires significantly more involved tools such as Riemannian geometry and generator theory. Notably, we establish that the solutions of SBs between Gaussian measures are themselves Gaussian processes with explicit mean and covariance kernels, and thus are readily amenable for many downstream applications such as generative modeling or interpolation. To demonstrate the utility, we devise a new method for modeling the evolution of single-cell genomics data and report significantly improved numerical stability compared to existing SB-based approaches.
△ Less
Submitted 31 March, 2023; v1 submitted 11 February, 2022;
originally announced February 2022.
-
A Robust Phased Elimination Algorithm for Corruption-Tolerant Gaussian Process Bandits
Authors:
Ilija Bogunovic,
Zihan Li,
Andreas Krause,
Jonathan Scarlett
Abstract:
We consider the sequential optimization of an unknown, continuous, and expensive to evaluate reward function, from noisy and adversarially corrupted observed rewards. When the corruption attacks are subject to a suitable budget $C$ and the function lives in a Reproducing Kernel Hilbert Space (RKHS), the problem can be posed as corrupted Gaussian process (GP) bandit optimization. We propose a novel…
▽ More
We consider the sequential optimization of an unknown, continuous, and expensive to evaluate reward function, from noisy and adversarially corrupted observed rewards. When the corruption attacks are subject to a suitable budget $C$ and the function lives in a Reproducing Kernel Hilbert Space (RKHS), the problem can be posed as corrupted Gaussian process (GP) bandit optimization. We propose a novel robust elimination-type algorithm that runs in epochs, combines exploration with infrequent switching to select a small subset of actions, and plays each action for multiple time instants. Our algorithm, Robust GP Phased Elimination (RGP-PE), successfully balances robustness to corruptions with exploration and exploitation such that its performance degrades minimally in the presence (or absence) of adversarial corruptions. When $T$ is the number of samples and $γ_T$ is the maximal information gain, the corruption-dependent term in our regret bound is $O(C γ_T^{3/2})$, which is significantly tighter than the existing $O(C \sqrt{T γ_T})$ for several commonly-considered kernels. We perform the first empirical study of robustness in the corrupted GP bandit setting, and show that our algorithm is robust against a variety of adversarial attacks.
△ Less
Submitted 28 March, 2022; v1 submitted 3 February, 2022;
originally announced February 2022.
-
Meta-Learning Hypothesis Spaces for Sequential Decision-making
Authors:
Parnian Kassraie,
Jonas Rothfuss,
Andreas Krause
Abstract:
Obtaining reliable, adaptive confidence sets for prediction functions (hypotheses) is a central challenge in sequential decision-making tasks, such as bandits and model-based reinforcement learning. These confidence sets typically rely on prior assumptions on the hypothesis space, e.g., the known kernel of a Reproducing Kernel Hilbert Space (RKHS). Hand-designing such kernels is error prone, and m…
▽ More
Obtaining reliable, adaptive confidence sets for prediction functions (hypotheses) is a central challenge in sequential decision-making tasks, such as bandits and model-based reinforcement learning. These confidence sets typically rely on prior assumptions on the hypothesis space, e.g., the known kernel of a Reproducing Kernel Hilbert Space (RKHS). Hand-designing such kernels is error prone, and misspecification may lead to poor or unsafe performance. In this work, we propose to meta-learn a kernel from offline data (Meta-KeL). For the case where the unknown kernel is a combination of known base kernels, we develop an estimator based on structured sparsity. Under mild conditions, we guarantee that our estimated RKHS yields valid confidence sets that, with increasing amounts of offline data, become as tight as those given the true unknown kernel. We demonstrate our approach on the kernelized bandit problem (a.k.a.~Bayesian optimization), where we establish regret bounds competitive with those given the true kernel. We also empirically evaluate the effectiveness of our approach on a Bayesian optimization task.
△ Less
Submitted 17 June, 2022; v1 submitted 1 February, 2022;
originally announced February 2022.
-
Constrained Policy Optimization via Bayesian World Models
Authors:
Yarden As,
Ilnura Usmanova,
Sebastian Curi,
Andreas Krause
Abstract:
Improving sample-efficiency and safety are crucial challenges when deploying reinforcement learning in high-stakes real world applications. We propose LAMBDA, a novel model-based approach for policy optimization in safety critical tasks modeled via constrained Markov decision processes. Our approach utilizes Bayesian world models, and harnesses the resulting uncertainty to maximize optimistic uppe…
▽ More
Improving sample-efficiency and safety are crucial challenges when deploying reinforcement learning in high-stakes real world applications. We propose LAMBDA, a novel model-based approach for policy optimization in safety critical tasks modeled via constrained Markov decision processes. Our approach utilizes Bayesian world models, and harnesses the resulting uncertainty to maximize optimistic upper bounds on the task objective, as well as pessimistic upper bounds on the safety constraints. We demonstrate LAMBDA's state of the art performance on the Safety-Gym benchmark suite in terms of sample efficiency and constraint violation.
△ Less
Submitted 6 February, 2022; v1 submitted 24 January, 2022;
originally announced January 2022.
-
GoSafeOpt: Scalable Safe Exploration for Global Optimization of Dynamical Systems
Authors:
Bhavya Sukhija,
Matteo Turchetta,
David Lindner,
Andreas Krause,
Sebastian Trimpe,
Dominik Baumann
Abstract:
Learning optimal control policies directly on physical systems is challenging since even a single failure can lead to costly hardware damage. Most existing model-free learning methods that guarantee safety, i.e., no failures, during exploration are limited to local optima. A notable exception is the GoSafe algorithm, which, unfortunately, cannot handle high-dimensional systems and hence cannot be…
▽ More
Learning optimal control policies directly on physical systems is challenging since even a single failure can lead to costly hardware damage. Most existing model-free learning methods that guarantee safety, i.e., no failures, during exploration are limited to local optima. A notable exception is the GoSafe algorithm, which, unfortunately, cannot handle high-dimensional systems and hence cannot be applied to most real-world dynamical systems. This work proposes GoSafeOpt as the first algorithm that can safely discover globally optimal policies for high-dimensional systems while giving safety and optimality guarantees. We demonstrate the superiority of GoSafeOpt over competing model-free safe learning methods on a robot arm that would be prohibitive for GoSafe.
△ Less
Submitted 12 June, 2023; v1 submitted 24 January, 2022;
originally announced January 2022.
-
Independent SE(3)-Equivariant Models for End-to-End Rigid Protein Docking
Authors:
Octavian-Eugen Ganea,
Xinyuan Huang,
Charlotte Bunne,
Yatao Bian,
Regina Barzilay,
Tommi Jaakkola,
Andreas Krause
Abstract:
Protein complex formation is a central problem in biology, being involved in most of the cell's processes, and essential for applications, e.g. drug design or protein engineering. We tackle rigid body protein-protein docking, i.e., computationally predicting the 3D structure of a protein-protein complex from the individual unbound structures, assuming no conformational change within the proteins h…
▽ More
Protein complex formation is a central problem in biology, being involved in most of the cell's processes, and essential for applications, e.g. drug design or protein engineering. We tackle rigid body protein-protein docking, i.e., computationally predicting the 3D structure of a protein-protein complex from the individual unbound structures, assuming no conformational change within the proteins happens during binding. We design a novel pairwise-independent SE(3)-equivariant graph matching network to predict the rotation and translation to place one of the proteins at the right docked position relative to the second protein. We mathematically guarantee a basic principle: the predicted complex is always identical regardless of the initial locations and orientations of the two structures. Our model, named EquiDock, approximates the binding pockets and predicts the docking poses using keypoint matching and alignment, achieved through optimal transport and a differentiable Kabsch algorithm. Empirically, we achieve significant running time improvements and often outperform existing docking software despite not relying on heavy candidate sampling, structure refinement, or templates.
△ Less
Submitted 15 March, 2022; v1 submitted 15 November, 2021;
originally announced November 2021.
-
NeuralPDE: Modelling Dynamical Systems from Data
Authors:
Andrzej Dulny,
Andreas Hotho,
Anna Krause
Abstract:
Many physical processes such as weather phenomena or fluid mechanics are governed by partial differential equations (PDEs). Modelling such dynamical systems using Neural Networks is an active research field. However, current methods are still very limited, as they do not exploit the knowledge about the dynamical nature of the system, require extensive prior knowledge about the governing equations…
▽ More
Many physical processes such as weather phenomena or fluid mechanics are governed by partial differential equations (PDEs). Modelling such dynamical systems using Neural Networks is an active research field. However, current methods are still very limited, as they do not exploit the knowledge about the dynamical nature of the system, require extensive prior knowledge about the governing equations or are limited to linear or first-order equations. In this work we make the observation that the Method of Lines used to solve PDEs can be represented using convolutions which makes convolutional neural networks (CNNs) the natural choice to parametrize arbitrary PDE dynamics. We combine this parametrization with differentiable ODE solvers to form the NeuralPDE Model, which explicitly takes into account the fact that the data is governed by differential equations. We show in several experiments on toy and real-world data that our model consistently outperforms state-of-the-art models used to learn dynamical systems.
△ Less
Submitted 11 October, 2022; v1 submitted 15 November, 2021;
originally announced November 2021.
-
Misspecified Gaussian Process Bandit Optimization
Authors:
Ilija Bogunovic,
Andreas Krause
Abstract:
We consider the problem of optimizing a black-box function based on noisy bandit feedback. Kernelized bandit algorithms have shown strong empirical and theoretical performance for this problem. They heavily rely on the assumption that the model is well-specified, however, and can fail without it. Instead, we introduce a \emph{misspecified} kernelized bandit setting where the unknown function can b…
▽ More
We consider the problem of optimizing a black-box function based on noisy bandit feedback. Kernelized bandit algorithms have shown strong empirical and theoretical performance for this problem. They heavily rely on the assumption that the model is well-specified, however, and can fail without it. Instead, we introduce a \emph{misspecified} kernelized bandit setting where the unknown function can be $ε$--uniformly approximated by a function with a bounded norm in some Reproducing Kernel Hilbert Space (RKHS). We design efficient and practical algorithms whose performance degrades minimally in the presence of model misspecification. Specifically, we present two algorithms based on Gaussian process (GP) methods: an optimistic EC-GP-UCB algorithm that requires knowing the misspecification error, and Phased GP Uncertainty Sampling, an elimination-type algorithm that can adapt to unknown model misspecification. We provide upper bounds on their cumulative regret in terms of $ε$, the time horizon, and the underlying kernel, and we show that our algorithm achieves optimal dependence on $ε$ with no prior knowledge of misspecification. In addition, in a stochastic contextual setting, we show that EC-GP-UCB can be effectively combined with the regret bound balancing strategy and attain similar regret bounds despite not knowing $ε$.
△ Less
Submitted 9 November, 2021;
originally announced November 2021.
-
Risk-averse Heteroscedastic Bayesian Optimization
Authors:
Anastasiia Makarova,
Ilnura Usmanova,
Ilija Bogunovic,
Andreas Krause
Abstract:
Many black-box optimization tasks arising in high-stakes applications require risk-averse decisions. The standard Bayesian optimization (BO) paradigm, however, optimizes the expected value only. We generalize BO to trade mean and input-dependent variance of the objective, both of which we assume to be unknown a priori. In particular, we propose a novel risk-averse heteroscedastic Bayesian optimiza…
▽ More
Many black-box optimization tasks arising in high-stakes applications require risk-averse decisions. The standard Bayesian optimization (BO) paradigm, however, optimizes the expected value only. We generalize BO to trade mean and input-dependent variance of the objective, both of which we assume to be unknown a priori. In particular, we propose a novel risk-averse heteroscedastic Bayesian optimization algorithm (RAHBO) that aims to identify a solution with high return and low noise variance, while learning the noise distribution on the fly. To this end, we model both expectation and variance as (unknown) RKHS functions, and propose a novel risk-aware acquisition function. We bound the regret for our approach and provide a robust rule to report the final decision point for applications where only a single solution must be identified. We demonstrate the effectiveness of RAHBO on synthetic benchmark functions and hyperparameter tuning tasks.
△ Less
Submitted 5 November, 2021;
originally announced November 2021.
-
Learning Stable Deep Dynamics Models for Partially Observed or Delayed Dynamical Systems
Authors:
Andreas Schlaginhaufen,
Philippe Wenk,
Andreas Krause,
Florian Dörfler
Abstract:
Learning how complex dynamical systems evolve over time is a key challenge in system identification. For safety critical systems, it is often crucial that the learned model is guaranteed to converge to some equilibrium point. To this end, neural ODEs regularized with neural Lyapunov functions are a promising approach when states are fully observed. For practical applications however, partial obser…
▽ More
Learning how complex dynamical systems evolve over time is a key challenge in system identification. For safety critical systems, it is often crucial that the learned model is guaranteed to converge to some equilibrium point. To this end, neural ODEs regularized with neural Lyapunov functions are a promising approach when states are fully observed. For practical applications however, partial observations are the norm. As we will demonstrate, initialization of unobserved augmented states can become a key problem for neural ODEs. To alleviate this issue, we propose to augment the system's state with its history. Inspired by state augmentation in discrete-time systems, we thus obtain neural delay differential equations. Based on classical time delay stability analysis, we then show how to ensure stability of the learned models, and theoretically analyze our approach. Our experiments demonstrate its applicability to stable system identification of partially observed systems and learning a stabilizing feedback policy in delayed feedback control.
△ Less
Submitted 10 December, 2021; v1 submitted 27 October, 2021;
originally announced October 2021.
-
Diversified Sampling for Batched Bayesian Optimization with Determinantal Point Processes
Authors:
Elvis Nava,
Mojmír Mutný,
Andreas Krause
Abstract:
In Bayesian Optimization (BO) we study black-box function optimization with noisy point evaluations and Bayesian priors. Convergence of BO can be greatly sped up by batching, where multiple evaluations of the black-box function are performed in a single round. The main difficulty in this setting is to propose at the same time diverse and informative batches of evaluation points. In this work, we i…
▽ More
In Bayesian Optimization (BO) we study black-box function optimization with noisy point evaluations and Bayesian priors. Convergence of BO can be greatly sped up by batching, where multiple evaluations of the black-box function are performed in a single round. The main difficulty in this setting is to propose at the same time diverse and informative batches of evaluation points. In this work, we introduce DPP-Batch Bayesian Optimization (DPP-BBO), a universal framework for inducing batch diversity in sampling based BO by leveraging the repulsive properties of Determinantal Point Processes (DPP) to naturally diversify the batch sampling procedure. We illustrate this framework by formulating DPP-Thompson Sampling (DPP-TS) as a variant of the popular Thompson Sampling (TS) algorithm and introducing a Markov Chain Monte Carlo procedure to sample from it. We then prove novel Bayesian simple regret bounds for both classical batched TS as well as our counterpart DPP-TS, with the latter bound being tighter. Our real-world, as well as synthetic, experiments demonstrate improved performance of DPP-BBO over classical batching methods with Gaussian process and Cox process models.
△ Less
Submitted 8 February, 2022; v1 submitted 22 October, 2021;
originally announced October 2021.
-
Sensing Cox Processes via Posterior Sampling and Positive Bases
Authors:
Mojmír Mutný,
Andreas Krause
Abstract:
We study adaptive sensing of Cox point processes, a widely used model from spatial statistics. We introduce three tasks: maximization of captured events, search for the maximum of the intensity function and learning level sets of the intensity function. We model the intensity function as a sample from a truncated Gaussian process, represented in a specially constructed positive basis. In this basi…
▽ More
We study adaptive sensing of Cox point processes, a widely used model from spatial statistics. We introduce three tasks: maximization of captured events, search for the maximum of the intensity function and learning level sets of the intensity function. We model the intensity function as a sample from a truncated Gaussian process, represented in a specially constructed positive basis. In this basis, the positivity constraint on the intensity function has a simple form. We show how an minimal description positive basis can be adapted to the covariance kernel, non-stationarity and make connections to common positive bases from prior works. Our adaptive sensing algorithms use Langevin dynamics and are based on posterior sampling (\textsc{Cox-Thompson}) and top-two posterior sampling (\textsc{Top2}) principles. With latter, the difference between samples serves as a surrogate to the uncertainty. We demonstrate the approach using examples from environmental monitoring and crime rate modeling, and compare it to the classical Bayesian experimental design approach.
△ Less
Submitted 29 March, 2022; v1 submitted 21 October, 2021;
originally announced October 2021.
-
Hierarchical Skills for Efficient Exploration
Authors:
Jonas Gehring,
Gabriel Synnaeve,
Andreas Krause,
Nicolas Usunier
Abstract:
In reinforcement learning, pre-trained low-level skills have the potential to greatly facilitate exploration. However, prior knowledge of the downstream task is required to strike the right balance between generality (fine-grained control) and specificity (faster learning) in skill design. In previous work on continuous control, the sensitivity of methods to this trade-off has not been addressed e…
▽ More
In reinforcement learning, pre-trained low-level skills have the potential to greatly facilitate exploration. However, prior knowledge of the downstream task is required to strike the right balance between generality (fine-grained control) and specificity (faster learning) in skill design. In previous work on continuous control, the sensitivity of methods to this trade-off has not been addressed explicitly, as locomotion provides a suitable prior for navigation tasks, which have been of foremost interest. In this work, we analyze this trade-off for low-level policy pre-training with a new benchmark suite of diverse, sparse-reward tasks for bipedal robots. We alleviate the need for prior knowledge by proposing a hierarchical skill learning framework that acquires skills of varying complexity in an unsupervised manner. For utilization on downstream tasks, we present a three-layered hierarchical learning algorithm to automatically trade off between general and specific skills as required by the respective task. In our experiments, we show that our approach performs this trade-off effectively and achieves better results than current state-of-the-art methods for end- to-end hierarchical reinforcement learning and unsupervised skill discovery. Code and videos are available at https://facebookresearch.github.io/hsd3 .
△ Less
Submitted 20 October, 2021;
originally announced October 2021.
-
Anomaly Detection in Beehives: An Algorithm Comparison
Authors:
Padraig Davidson,
Michael Steininger,
Florian Lautenschlager,
Anna Krause,
Andreas Hotho
Abstract:
Sensor-equipped beehives allow monitoring the living conditions of bees. Machine learning models can use the data of such hives to learn behavioral patterns and find anomalous events. One type of event that is of particular interest to apiarists for economical reasons is bee swarming. Other events of interest are behavioral anomalies from illness and technical anomalies, e.g. sensor failure. Beeke…
▽ More
Sensor-equipped beehives allow monitoring the living conditions of bees. Machine learning models can use the data of such hives to learn behavioral patterns and find anomalous events. One type of event that is of particular interest to apiarists for economical reasons is bee swarming. Other events of interest are behavioral anomalies from illness and technical anomalies, e.g. sensor failure. Beekeepers can be supported by suitable machine learning models which can detect these events. In this paper we compare multiple machine learning models for anomaly detection and evaluate them for their applicability in the context of beehives. Namely we employed Deep Recurrent Autoencoder, Elliptic Envelope, Isolation Forest, Local Outlier Factor and One-Class SVM. Through evaluation with real world datasets of different hives and with different sensor setups we find that the autoencoder is the best multi-purpose anomaly detector in comparison.
△ Less
Submitted 8 October, 2021;
originally announced October 2021.
-
Live Visualization of Dynamic Software Cities with Heat Map Overlays
Authors:
Alexander Krause,
Malte Hansen,
Wilhelm Hasselbring
Abstract:
The 3D city metaphor in software visualization is a well-explored rendering method. Numerous tools use their custom variation to visualize offline-analyzed data. Heat map overlays are one of these variants. They introduce a separate information layer in addition to the software city's own semantics. Results show that their usage facilitates program comprehension.
In this paper, we present our he…
▽ More
The 3D city metaphor in software visualization is a well-explored rendering method. Numerous tools use their custom variation to visualize offline-analyzed data. Heat map overlays are one of these variants. They introduce a separate information layer in addition to the software city's own semantics. Results show that their usage facilitates program comprehension.
In this paper, we present our heat map approach for the city metaphor visualization based on live trace analysis. In comparison to previous approaches, our implementation uses live dynamic analysis of a software system's runtime behavior. At any time, users can toggle the heat map feature and choose which runtime-dependent metric the heat map should visualize. Our approach continuously and automatically renders both software cities and heat maps. It does not require a manual or semi-automatic generation of heat maps and seamlessly blends into the overall software visualization. We implemented this approach in our web-based tool ExplorViz, such that the heat map overlay is also available in our augmented reality environment. ExplorViz is developed as open source software and is continuously published via Docker images. A live demo of ExplorViz is publicly available.
△ Less
Submitted 29 September, 2021;
originally announced September 2021.
-
Data Summarization via Bilevel Optimization
Authors:
Zalán Borsos,
Mojmír Mutný,
Marco Tagliasacchi,
Andreas Krause
Abstract:
The increasing availability of massive data sets poses a series of challenges for machine learning. Prominent among these is the need to learn models under hardware or human resource constraints. In such resource-constrained settings, a simple yet powerful approach is to operate on small subsets of the data. Coresets are weighted subsets of the data that provide approximation guarantees for the op…
▽ More
The increasing availability of massive data sets poses a series of challenges for machine learning. Prominent among these is the need to learn models under hardware or human resource constraints. In such resource-constrained settings, a simple yet powerful approach is to operate on small subsets of the data. Coresets are weighted subsets of the data that provide approximation guarantees for the optimization objective. However, existing coreset constructions are highly model-specific and are limited to simple models such as linear regression, logistic regression, and $k$-means. In this work, we propose a generic coreset construction framework that formulates the coreset selection as a cardinality-constrained bilevel optimization problem. In contrast to existing approaches, our framework does not require model-specific adaptations and applies to any twice differentiable model, including neural networks. We show the effectiveness of our framework for a wide range of models in various settings, including training non-convex models online and batch active learning.
△ Less
Submitted 26 September, 2021;
originally announced September 2021.
-
Fast Projection Onto Convex Smooth Constraints
Authors:
Ilnura Usmanova,
Maryam Kamgarpour,
Andreas Krause,
Kfir Yehuda Levy
Abstract:
The Euclidean projection onto a convex set is an important problem that arises in numerous constrained optimization tasks. Unfortunately, in many cases, computing projections is computationally demanding. In this work, we focus on projection problems where the constraints are smooth and the number of constraints is significantly smaller than the dimension. The runtime of existing approaches to sol…
▽ More
The Euclidean projection onto a convex set is an important problem that arises in numerous constrained optimization tasks. Unfortunately, in many cases, computing projections is computationally demanding. In this work, we focus on projection problems where the constraints are smooth and the number of constraints is significantly smaller than the dimension. The runtime of existing approaches to solving such problems is either cubic in the dimension or polynomial in the inverse of the target accuracy. Conversely, we propose a simple and efficient primal-dual approach, with a runtime that scales only linearly with the dimension, and only logarithmically in the inverse of the target accuracy. We empirically demonstrate its performance, and compare it with standard baselines.
△ Less
Submitted 20 September, 2021;
originally announced September 2021.
-
Joint Direction and Proximity Classification of Overlap** Sound Events from Binaural Audio
Authors:
Daniel Aleksander Krause,
Archontis Politis,
Annamaria Mesaros
Abstract:
Sound source proximity and distance estimation are of great interest in many practical applications, since they provide significant information for acoustic scene analysis. As both tasks share complementary qualities, ensuring efficient interaction between these two is crucial for a complete picture of an aural environment. In this paper, we aim to investigate several ways of performing joint prox…
▽ More
Sound source proximity and distance estimation are of great interest in many practical applications, since they provide significant information for acoustic scene analysis. As both tasks share complementary qualities, ensuring efficient interaction between these two is crucial for a complete picture of an aural environment. In this paper, we aim to investigate several ways of performing joint proximity and direction estimation from binaural recordings, both defined as coarse classification problems based on Deep Neural Networks (DNNs). Considering the limitations of binaural audio, we propose two methods of splitting the sphere into angular areas in order to obtain a set of directional classes. For each method we study different model types to acquire information about the direction-of-arrival (DoA). Finally, we propose various ways of combining the proximity and direction estimation problems into a joint task providing temporal information about the onsets and offsets of the appearing sources. Experiments are performed for a synthetic reverberant binaural dataset consisting of up to two overlap** sound events.
△ Less
Submitted 26 July, 2021;
originally announced July 2021.
-
Contextual Games: Multi-Agent Learning with Side Information
Authors:
Pier Giuseppe Sessa,
Ilija Bogunovic,
Andreas Krause,
Maryam Kamgarpour
Abstract:
We formulate the novel class of contextual games, a type of repeated games driven by contextual information at each round. By means of kernel-based regularity assumptions, we model the correlation between different contexts and game outcomes and propose a novel online (meta) algorithm that exploits such correlations to minimize the contextual regret of individual players. We define game-theoretic…
▽ More
We formulate the novel class of contextual games, a type of repeated games driven by contextual information at each round. By means of kernel-based regularity assumptions, we model the correlation between different contexts and game outcomes and propose a novel online (meta) algorithm that exploits such correlations to minimize the contextual regret of individual players. We define game-theoretic notions of contextual Coarse Correlated Equilibria (c-CCE) and optimal contextual welfare for this new class of games and show that c-CCEs and optimal welfare can be approached whenever players' contextual regrets vanish. Finally, we empirically validate our results in a traffic routing experiment, where our algorithm leads to better performance and higher welfare compared to baselines that do not exploit the available contextual information or the correlations present in the game.
△ Less
Submitted 13 July, 2021;
originally announced July 2021.
-
Analog Computing for Molecular Dynamics
Authors:
Sven Köppel,
Alexandra Krause,
Bernd Ulmann
Abstract:
Modern analog computers are ideally suited to solving large systems of ordinary differential equations at high speed with low energy consumtion and limited accuracy. In this article, we survey N-body physics, applied to a simple water model inspired by force fields which are popular in molecular dynamics. We demonstrate a setup which simulate a single water molecule in time. To the best of our kno…
▽ More
Modern analog computers are ideally suited to solving large systems of ordinary differential equations at high speed with low energy consumtion and limited accuracy. In this article, we survey N-body physics, applied to a simple water model inspired by force fields which are popular in molecular dynamics. We demonstrate a setup which simulate a single water molecule in time. To the best of our knowledge such a simulation has never been done on analog computers before. Important implementation aspects of the model, such as scaling, data range and circuit design, are highlighted. We also analyze the performance and compare the solution with a numerical approach.
△ Less
Submitted 13 July, 2021;
originally announced July 2021.
-
Efficient Model-Based Multi-Agent Mean-Field Reinforcement Learning
Authors:
Barna Pásztor,
Ilija Bogunovic,
Andreas Krause
Abstract:
Learning in multi-agent systems is highly challenging due to several factors including the non-stationarity introduced by agents' interactions and the combinatorial nature of their state and action spaces. In particular, we consider the Mean-Field Control (MFC) problem which assumes an asymptotically infinite population of identical agents that aim to collaboratively maximize the collective reward…
▽ More
Learning in multi-agent systems is highly challenging due to several factors including the non-stationarity introduced by agents' interactions and the combinatorial nature of their state and action spaces. In particular, we consider the Mean-Field Control (MFC) problem which assumes an asymptotically infinite population of identical agents that aim to collaboratively maximize the collective reward. In many cases, solutions of an MFC problem are good approximations for large systems, hence, efficient learning for MFC is valuable for the analogous discrete agent setting with many agents. Specifically, we focus on the case of unknown system dynamics where the goal is to simultaneously optimize for the rewards and learn from experience. We propose an efficient model-based reinforcement learning algorithm, $M^3-UCRL$, that runs in episodes, balances between exploration and exploitation during policy learning, and provably solves this problem. Our main theoretical contributions are the first general regret bounds for model-based reinforcement learning for MFC, obtained via a novel mean-field type analysis. To learn the system's dynamics, $M^3-UCRL$ can be instantiated with various statistical models, e.g., neural networks or Gaussian Processes. Moreover, we provide a practical parametrization of the core optimization problem that facilitates gradient-based optimization techniques when combined with differentiable dynamics approximation methods such as neural networks.
△ Less
Submitted 9 May, 2023; v1 submitted 8 July, 2021;
originally announced July 2021.
-
Neural Contextual Bandits without Regret
Authors:
Parnian Kassraie,
Andreas Krause
Abstract:
Contextual bandits are a rich model for sequential decision making given side information, with important applications, e.g., in recommender systems. We propose novel algorithms for contextual bandits harnessing neural networks to approximate the unknown reward function. We resolve the open problem of proving sublinear regret bounds in this setting for general context sequences, considering both f…
▽ More
Contextual bandits are a rich model for sequential decision making given side information, with important applications, e.g., in recommender systems. We propose novel algorithms for contextual bandits harnessing neural networks to approximate the unknown reward function. We resolve the open problem of proving sublinear regret bounds in this setting for general context sequences, considering both fully-connected and convolutional networks. To this end, we first analyze NTK-UCB, a kernelized bandit optimization algorithm employing the Neural Tangent Kernel (NTK), and bound its regret in terms of the NTK maximum information gain $γ_T$, a complexity parameter capturing the difficulty of learning. Our bounds on $γ_T$ for the NTK may be of independent interest. We then introduce our neural network based algorithm NN-UCB, and show that its regret closely tracks that of NTK-UCB. Under broad non-parametric assumptions about the reward function, our approach converges to the optimal policy at a $\tilde{\mathcal{O}}(T^{-1/2d})$ rate, where $d$ is the dimension of the context.
△ Less
Submitted 28 February, 2022; v1 submitted 7 July, 2021;
originally announced July 2021.
-
Distributional Gradient Matching for Learning Uncertain Neural Dynamics Models
Authors:
Lenart Treven,
Philippe Wenk,
Florian Dörfler,
Andreas Krause
Abstract:
Differential equations in general and neural ODEs in particular are an essential technique in continuous-time system identification. While many deterministic learning algorithms have been designed based on numerical integration via the adjoint method, many downstream tasks such as active learning, exploration in reinforcement learning, robust control, or filtering require accurate estimates of pre…
▽ More
Differential equations in general and neural ODEs in particular are an essential technique in continuous-time system identification. While many deterministic learning algorithms have been designed based on numerical integration via the adjoint method, many downstream tasks such as active learning, exploration in reinforcement learning, robust control, or filtering require accurate estimates of predictive uncertainties. In this work, we propose a novel approach towards estimating epistemically uncertain neural ODEs, avoiding the numerical integration bottleneck. Instead of modeling uncertainty in the ODE parameters, we directly model uncertainties in the state space. Our algorithm - distributional gradient matching (DGM) - jointly trains a smoother and a dynamics model and matches their gradients via minimizing a Wasserstein loss. Our experiments show that, compared to traditional approximate inference methods based on numerical integration, our approach is faster to train, faster at predicting previously unseen trajectories, and in the context of neural ODEs, significantly more accurate.
△ Less
Submitted 15 October, 2021; v1 submitted 22 June, 2021;
originally announced June 2021.
-
Modern Perspectives on Near-Equilibrium Analysis of Turing Systems
Authors:
Andrew L. Krause,
Eamonn A. Gaffney,
Philip K. Maini,
Václav Klika
Abstract:
In the nearly seven decades since the publication of Alan Turing's work on morphogenesis, enormous progress has been made in understanding both the mathematical and biological aspects of his proposed reaction-diffusion theory. Some of these developments were nascent in Turing's paper, and others have been due to new insights from modern mathematical techniques, advances in numerical simulations, a…
▽ More
In the nearly seven decades since the publication of Alan Turing's work on morphogenesis, enormous progress has been made in understanding both the mathematical and biological aspects of his proposed reaction-diffusion theory. Some of these developments were nascent in Turing's paper, and others have been due to new insights from modern mathematical techniques, advances in numerical simulations, and extensive biological experiments. Despite such progress, there are still important gaps between theory and experiment, with many examples of biological patterning where the underlying mechanisms are still unclear. Here we review modern developments in the mathematical theory pioneered by Turing, showing how his approach has been generalized to a range of settings beyond the classical two-species reaction-diffusion framework, including evolving and complex manifolds, systems heterogeneous in space and time, and more general reaction-transport equations. While substantial progress has been made in understanding these more complicated models, there are many remaining challenges that we highlight throughout. We focus on the mathematical theory, and in particular linear stability analysis of `trivial' base states. We emphasise important open questions in develo** this theory further, and discuss obstacles in using these techniques to understand biological reality.
△ Less
Submitted 13 September, 2021; v1 submitted 15 June, 2021;
originally announced June 2021.
-
PopSkipJump: Decision-Based Attack for Probabilistic Classifiers
Authors:
Carl-Johann Simon-Gabriel,
Noman Ahmed Sheikh,
Andreas Krause
Abstract:
Most current classifiers are vulnerable to adversarial examples, small input perturbations that change the classification output. Many existing attack algorithms cover various settings, from white-box to black-box classifiers, but typically assume that the answers are deterministic and often fail when they are not. We therefore propose a new adversarial decision-based attack specifically designed…
▽ More
Most current classifiers are vulnerable to adversarial examples, small input perturbations that change the classification output. Many existing attack algorithms cover various settings, from white-box to black-box classifiers, but typically assume that the answers are deterministic and often fail when they are not. We therefore propose a new adversarial decision-based attack specifically designed for classifiers with probabilistic outputs. It is based on the HopSkipJump attack by Chen et al. (2019, arXiv:1904.02144v5 ), a strong and query efficient decision-based attack originally designed for deterministic classifiers. Our P(robabilisticH)opSkipJump attack adapts its amount of queries to maintain HopSkipJump's original output quality across various noise levels, while converging to its query efficiency as the noise level decreases. We test our attack on various noise models, including state-of-the-art off-the-shelf randomized defenses, and show that they offer almost no extra robustness to decision-based attacks. Code is available at https://github.com/cjsg/PopSkipJump .
△ Less
Submitted 14 June, 2021;
originally announced June 2021.
-
Proximal Optimal Transport Modeling of Population Dynamics
Authors:
Charlotte Bunne,
Laetitia Meng-Papaxanthos,
Andreas Krause,
Marco Cuturi
Abstract:
We propose a new approach to model the collective dynamics of a population of particles evolving with time. As is often the case in challenging scientific applications, notably single-cell genomics, measuring features for these particles requires destroying them. As a result, the population can only be monitored with periodic snapshots, obtained by sampling a few particles that are sacrificed in e…
▽ More
We propose a new approach to model the collective dynamics of a population of particles evolving with time. As is often the case in challenging scientific applications, notably single-cell genomics, measuring features for these particles requires destroying them. As a result, the population can only be monitored with periodic snapshots, obtained by sampling a few particles that are sacrificed in exchange for measurements. Given only access to these snapshots, can we reconstruct likely individual trajectories for all other particles? We propose to model these trajectories as collective realizations of a causal Jordan-Kinderlehrer-Otto (JKO) flow of measures: The JKO scheme posits that the new configuration taken by a population at time $t+1$ is one that trades off an improvement, in the sense that it decreases an energy, while remaining close (in Wasserstein distance) to the previous configuration observed at $t$. In order to learn such an energy using only snapshots, we propose JKOnet, a neural architecture that computes (in end-to-end differentiable fashion) the JKO flow given a parametric energy and initial configuration of points. We demonstrate the good performance and robustness of the JKOnet fitting procedure, compared to a more direct forward method.
△ Less
Submitted 18 February, 2022; v1 submitted 11 June, 2021;
originally announced June 2021.
-
Robust Generalization despite Distribution Shift via Minimum Discriminating Information
Authors:
Tobias Sutter,
Andreas Krause,
Daniel Kuhn
Abstract:
Training models that perform well under distribution shifts is a central challenge in machine learning. In this paper, we introduce a modeling framework where, in addition to training data, we have partial structural knowledge of the shifted test distribution. We employ the principle of minimum discriminating information to embed the available prior knowledge, and use distributionally robust optim…
▽ More
Training models that perform well under distribution shifts is a central challenge in machine learning. In this paper, we introduce a modeling framework where, in addition to training data, we have partial structural knowledge of the shifted test distribution. We employ the principle of minimum discriminating information to embed the available prior knowledge, and use distributionally robust optimization to account for uncertainty due to the limited samples. By leveraging large deviation results, we obtain explicit generalization bounds with respect to the unknown shifted distribution. Lastly, we demonstrate the versatility of our framework by demonstrating it on two rather distinct applications: (1) training classifiers on systematically biased data and (2) off-policy evaluation in Markov Decision Processes.
△ Less
Submitted 26 October, 2021; v1 submitted 8 June, 2021;
originally announced June 2021.
-
Meta-Learning Reliable Priors in the Function Space
Authors:
Jonas Rothfuss,
Dominique Heyn,
**fan Chen,
Andreas Krause
Abstract:
When data are scarce meta-learning can improve a learner's accuracy by harnessing previous experience from related learning tasks. However, existing methods have unreliable uncertainty estimates which are often overconfident. Addressing these shortcomings, we introduce a novel meta-learning framework, called F-PACOH, that treats meta-learned priors as stochastic processes and performs meta-level r…
▽ More
When data are scarce meta-learning can improve a learner's accuracy by harnessing previous experience from related learning tasks. However, existing methods have unreliable uncertainty estimates which are often overconfident. Addressing these shortcomings, we introduce a novel meta-learning framework, called F-PACOH, that treats meta-learned priors as stochastic processes and performs meta-level regularization directly in the function space. This allows us to directly steer the probabilistic predictions of the meta-learner towards high epistemic uncertainty in regions of insufficient meta-training data and, thus, obtain well-calibrated uncertainty estimates. Finally, we showcase how our approach can be integrated with sequential decision making, where reliable uncertainty quantification is imperative. In our benchmark study on meta-learning for Bayesian Optimization (BO), F-PACOH significantly outperforms all other meta-learners and standard baselines.
△ Less
Submitted 11 January, 2022; v1 submitted 6 June, 2021;
originally announced June 2021.
-
Energy-Based Learning for Cooperative Games, with Applications to Valuation Problems in Machine Learning
Authors:
Yatao Bian,
Yu Rong,
Tingyang Xu,
Jiaxiang Wu,
Andreas Krause,
Junzhou Huang
Abstract:
Valuation problems, such as feature interpretation, data valuation and model valuation for ensembles, become increasingly more important in many machine learning applications. Such problems are commonly solved by well-known game-theoretic criteria, such as Shapley value or Banzhaf value. In this work, we present a novel energy-based treatment for cooperative games, with a theoretical justification…
▽ More
Valuation problems, such as feature interpretation, data valuation and model valuation for ensembles, become increasingly more important in many machine learning applications. Such problems are commonly solved by well-known game-theoretic criteria, such as Shapley value or Banzhaf value. In this work, we present a novel energy-based treatment for cooperative games, with a theoretical justification by the maximum entropy framework. Surprisingly, by conducting variational inference of the energy-based model, we recover various game-theoretic valuation criteria through conducting one-step fixed point iteration for maximizing the mean-field ELBO objective. This observation also verifies the rationality of existing criteria, as they are all attempting to decouple the correlations among the players through the mean-field approach. By running fixed point iteration for multiple steps, we achieve a trajectory of the valuations, among which we define the valuation with the best conceivable decoupling error as the Variational Index. We prove that under uniform initializations, these variational valuations all satisfy a set of game-theoretic axioms. We experimentally demonstrate that the proposed Variational Index enjoys lower decoupling error and better valuation performance on certain synthetic and real-world valuation problems.
△ Less
Submitted 12 May, 2022; v1 submitted 5 June, 2021;
originally announced June 2021.
-
Addressing the Long-term Impact of ML Decisions via Policy Regret
Authors:
David Lindner,
Hoda Heidari,
Andreas Krause
Abstract:
Machine Learning (ML) increasingly informs the allocation of opportunities to individuals and communities in areas such as lending, education, employment, and beyond. Such decisions often impact their subjects' future characteristics and capabilities in an a priori unknown fashion. The decision-maker, therefore, faces exploration-exploitation dilemmas akin to those in multi-armed bandits. Followin…
▽ More
Machine Learning (ML) increasingly informs the allocation of opportunities to individuals and communities in areas such as lending, education, employment, and beyond. Such decisions often impact their subjects' future characteristics and capabilities in an a priori unknown fashion. The decision-maker, therefore, faces exploration-exploitation dilemmas akin to those in multi-armed bandits. Following prior work, we model communities as arms. To capture the long-term effects of ML-based allocation decisions, we study a setting in which the reward from each arm evolves every time the decision-maker pulls that arm. We focus on reward functions that are initially increasing in the number of pulls but may become (and remain) decreasing after a certain point. We argue that an acceptable sequential allocation of opportunities must take an arm's potential for growth into account. We capture these considerations through the notion of policy regret, a much stronger notion than the often-studied external regret, and present an algorithm with provably sub-linear policy regret for sufficiently long time horizons. We empirically compare our algorithm with several baselines and find that it consistently outperforms them, in particular for long time horizons.
△ Less
Submitted 2 June, 2021;
originally announced June 2021.
-
Cherry-Picking Gradients: Learning Low-Rank Embeddings of Visual Data via Differentiable Cross-Approximation
Authors:
Mikhail Usvyatsov,
Anastasia Makarova,
Rafael Ballester-Ripoll,
Maxim Rakhuba,
Andreas Krause,
Konrad Schindler
Abstract:
We propose an end-to-end trainable framework that processes large-scale visual data tensors by looking at a fraction of their entries only. Our method combines a neural network encoder with a tensor train decomposition to learn a low-rank latent encoding, coupled with cross-approximation (CA) to learn the representation through a subset of the original samples. CA is an adaptive sampling algorithm…
▽ More
We propose an end-to-end trainable framework that processes large-scale visual data tensors by looking at a fraction of their entries only. Our method combines a neural network encoder with a tensor train decomposition to learn a low-rank latent encoding, coupled with cross-approximation (CA) to learn the representation through a subset of the original samples. CA is an adaptive sampling algorithm that is native to tensor decompositions and avoids working with the full high-resolution data explicitly. Instead, it actively selects local representative samples that we fetch out-of-core and on-demand. The required number of samples grows only logarithmically with the size of the input. Our implicit representation of the tensor in the network enables processing large grids that could not be otherwise tractable in their uncompressed form. The proposed approach is particularly useful for large-scale multidimensional grid data (e.g., 3D tomography), and for tasks that require context over a large receptive field (e.g., predicting the medical condition of entire organs). The code is available at https://github.com/aelphy/c-pic.
△ Less
Submitted 12 November, 2021; v1 submitted 29 May, 2021;
originally announced May 2021.
-
Near-Optimal Multi-Perturbation Experimental Design for Causal Structure Learning
Authors:
Scott Sussex,
Andreas Krause,
Caroline Uhler
Abstract:
Causal structure learning is a key problem in many domains. Causal structures can be learnt by performing experiments on the system of interest. We address the largely unexplored problem of designing a batch of experiments that each simultaneously intervene on multiple variables. While potentially more informative than the commonly considered single-variable interventions, selecting such intervent…
▽ More
Causal structure learning is a key problem in many domains. Causal structures can be learnt by performing experiments on the system of interest. We address the largely unexplored problem of designing a batch of experiments that each simultaneously intervene on multiple variables. While potentially more informative than the commonly considered single-variable interventions, selecting such interventions is algorithmically much more challenging, due to the doubly-exponential combinatorial search space over sets of composite interventions. In this paper, we develop efficient algorithms for optimizing different objective functions quantifying the informativeness of a budget-constrained batch of experiments. By establishing novel submodularity properties of these objectives, we provide approximation guarantees for our algorithms. Our algorithms empirically perform superior to both random interventions and algorithms that only select single-variable interventions.
△ Less
Submitted 24 November, 2021; v1 submitted 28 May, 2021;
originally announced May 2021.
-
DiBS: Differentiable Bayesian Structure Learning
Authors:
Lars Lorch,
Jonas Rothfuss,
Bernhard Schölkopf,
Andreas Krause
Abstract:
Bayesian structure learning allows inferring Bayesian network structure from data while reasoning about the epistemic uncertainty -- a key element towards enabling active causal discovery and designing interventions in real world systems. In this work, we propose a general, fully differentiable framework for Bayesian structure learning (DiBS) that operates in the continuous space of a latent proba…
▽ More
Bayesian structure learning allows inferring Bayesian network structure from data while reasoning about the epistemic uncertainty -- a key element towards enabling active causal discovery and designing interventions in real world systems. In this work, we propose a general, fully differentiable framework for Bayesian structure learning (DiBS) that operates in the continuous space of a latent probabilistic graph representation. Contrary to existing work, DiBS is agnostic to the form of the local conditional distributions and allows for joint posterior inference of both the graph structure and the conditional distribution parameters. This makes our formulation directly applicable to posterior inference of complex Bayesian network models, e.g., with nonlinear dependencies encoded by neural networks. Using DiBS, we devise an efficient, general purpose variational inference method for approximating distributions over structural models. In evaluations on simulated and real-world data, our method significantly outperforms related approaches to joint posterior inference.
△ Less
Submitted 16 December, 2021; v1 submitted 25 May, 2021;
originally announced May 2021.
-
Bias-Robust Bayesian Optimization via Dueling Bandits
Authors:
Johannes Kirschner,
Andreas Krause
Abstract:
We consider Bayesian optimization in settings where observations can be adversarially biased, for example by an uncontrolled hidden confounder. Our first contribution is a reduction of the confounded setting to the dueling bandit model. Then we propose a novel approach for dueling bandits based on information-directed sampling (IDS). Thereby, we obtain the first efficient kernelized algorithm for…
▽ More
We consider Bayesian optimization in settings where observations can be adversarially biased, for example by an uncontrolled hidden confounder. Our first contribution is a reduction of the confounded setting to the dueling bandit model. Then we propose a novel approach for dueling bandits based on information-directed sampling (IDS). Thereby, we obtain the first efficient kernelized algorithm for dueling bandits that comes with cumulative regret guarantees. Our analysis further generalizes a previously proposed semi-parametric linear bandit model to non-linear reward functions, and uncovers interesting links to doubly-robust estimation.
△ Less
Submitted 9 June, 2021; v1 submitted 25 May, 2021;
originally announced May 2021.
-
A note on the CAPM with endogenously consistent market returns
Authors:
Andreas Krause
Abstract:
I demonstrate that with the market return determined by the equilibrium returns of the CAPM, expected returns of an asset are affected by the risks of all assets jointly. Another implication is that the range of feasible market returns will be limited and dependent on the distribution of weights in the market portfolio. A large and well diversified market with no dominating asset will only return…
▽ More
I demonstrate that with the market return determined by the equilibrium returns of the CAPM, expected returns of an asset are affected by the risks of all assets jointly. Another implication is that the range of feasible market returns will be limited and dependent on the distribution of weights in the market portfolio. A large and well diversified market with no dominating asset will only return zero while a market dominated by a small number of assets will only return the risk-free rate. In the limiting case of atomistic assets, we recover the properties of the standard CAPM.
△ Less
Submitted 21 May, 2021;
originally announced May 2021.
-
Regret Bounds for Gaussian-Process Optimization in Large Domains
Authors:
Manuel Wüthrich,
Bernhard Schölkopf,
Andreas Krause
Abstract:
The goal of this paper is to characterize Gaussian-Process optimization in the setting where the function domain is large relative to the number of admissible function evaluations, i.e., where it is impossible to find the global optimum. We provide upper bounds on the suboptimality (Bayesian simple regret) of the solution found by optimization strategies that are closely related to the widely used…
▽ More
The goal of this paper is to characterize Gaussian-Process optimization in the setting where the function domain is large relative to the number of admissible function evaluations, i.e., where it is impossible to find the global optimum. We provide upper bounds on the suboptimality (Bayesian simple regret) of the solution found by optimization strategies that are closely related to the widely used expected improvement (EI) and upper confidence bound (UCB) algorithms. These regret bounds illuminate the relationship between the number of evaluations, the domain size (i.e. cardinality of finite domains / Lipschitz constant of the covariance function in continuous domains), and the optimality of the retrieved function value. In particular, we show that even when the number of evaluations is far too small to find the global optimum, we can find nontrivial function values (e.g. values that achieve a certain ratio with the optimal value).
△ Less
Submitted 24 January, 2022; v1 submitted 29 April, 2021;
originally announced April 2021.
-
Automatic Termination for Hyperparameter Optimization
Authors:
Anastasia Makarova,
Huibin Shen,
Valerio Perrone,
Aaron Klein,
Jean Baptiste Faddoul,
Andreas Krause,
Matthias Seeger,
Cedric Archambeau
Abstract:
Bayesian optimization (BO) is a widely popular approach for the hyperparameter optimization (HPO) in machine learning. At its core, BO iteratively evaluates promising configurations until a user-defined budget, such as wall-clock time or number of iterations, is exhausted. While the final performance after tuning heavily depends on the provided budget, it is hard to pre-specify an optimal value in…
▽ More
Bayesian optimization (BO) is a widely popular approach for the hyperparameter optimization (HPO) in machine learning. At its core, BO iteratively evaluates promising configurations until a user-defined budget, such as wall-clock time or number of iterations, is exhausted. While the final performance after tuning heavily depends on the provided budget, it is hard to pre-specify an optimal value in advance. In this work, we propose an effective and intuitive termination criterion for BO that automatically stops the procedure if it is sufficiently close to the global optimum. Our key insight is that the discrepancy between the true objective (predictive performance on test data) and the computable target (validation performance) suggests stop** once the suboptimality in optimizing the target is dominated by the statistical estimation error. Across an extensive range of real-world HPO problems and baselines, we show that our termination criterion achieves a better trade-off between the test performance and optimization time. Additionally, we find that overfitting may occur in the context of HPO, which is arguably an overlooked problem in the literature, and show how our termination criterion helps to mitigate this phenomenon on both small and large datasets.
△ Less
Submitted 22 July, 2022; v1 submitted 16 April, 2021;
originally announced April 2021.
-
Combining Pessimism with Optimism for Robust and Efficient Model-Based Deep Reinforcement Learning
Authors:
Sebastian Curi,
Ilija Bogunovic,
Andreas Krause
Abstract:
In real-world tasks, reinforcement learning (RL) agents frequently encounter situations that are not present during training time. To ensure reliable performance, the RL agents need to exhibit robustness against worst-case situations. The robust RL framework addresses this challenge via a worst-case optimization between an agent and an adversary. Previous robust RL algorithms are either sample ine…
▽ More
In real-world tasks, reinforcement learning (RL) agents frequently encounter situations that are not present during training time. To ensure reliable performance, the RL agents need to exhibit robustness against worst-case situations. The robust RL framework addresses this challenge via a worst-case optimization between an agent and an adversary. Previous robust RL algorithms are either sample inefficient, lack robustness guarantees, or do not scale to large problems. We propose the Robust Hallucinated Upper-Confidence RL (RH-UCRL) algorithm to provably solve this problem while attaining near-optimal sample complexity guarantees. RH-UCRL is a model-based reinforcement learning (MBRL) algorithm that effectively distinguishes between epistemic and aleatoric uncertainty and efficiently explores both the agent and adversary decision spaces during policy learning. We scale RH-UCRL to complex tasks via neural networks ensemble models as well as neural network policies. Experimentally, we demonstrate that RH-UCRL outperforms other robust deep RL algorithms in a variety of adversarial environments.
△ Less
Submitted 18 March, 2021;
originally announced March 2021.
-
Information Directed Reward Learning for Reinforcement Learning
Authors:
David Lindner,
Matteo Turchetta,
Sebastian Tschiatschek,
Kamil Ciosek,
Andreas Krause
Abstract:
For many reinforcement learning (RL) applications, specifying a reward is difficult. This paper considers an RL setting where the agent obtains information about the reward only by querying an expert that can, for example, evaluate individual states or provide binary preferences over trajectories. From such expensive feedback, we aim to learn a model of the reward that allows standard RL algorithm…
▽ More
For many reinforcement learning (RL) applications, specifying a reward is difficult. This paper considers an RL setting where the agent obtains information about the reward only by querying an expert that can, for example, evaluate individual states or provide binary preferences over trajectories. From such expensive feedback, we aim to learn a model of the reward that allows standard RL algorithms to achieve high expected returns with as few expert queries as possible. To this end, we propose Information Directed Reward Learning (IDRL), which uses a Bayesian model of the reward and selects queries that maximize the information gain about the difference in return between plausibly optimal policies. In contrast to prior active reward learning methods designed for specific types of queries, IDRL naturally accommodates different query types. Moreover, it achieves similar or better performance with significantly fewer queries by shifting the focus from reducing the reward approximation error to improving the policy induced by the reward model. We support our findings with extensive evaluations in multiple environments and with different query types.
△ Less
Submitted 31 January, 2022; v1 submitted 24 February, 2021;
originally announced February 2021.
-
Risk-Averse Offline Reinforcement Learning
Authors:
Núria Armengol Urpí,
Sebastian Curi,
Andreas Krause
Abstract:
Training Reinforcement Learning (RL) agents in high-stakes applications might be too prohibitive due to the risk associated to exploration. Thus, the agent can only use data previously collected by safe policies. While previous work considers optimizing the average performance using offline data, we focus on optimizing a risk-averse criteria, namely the CVaR. In particular, we present the Offline…
▽ More
Training Reinforcement Learning (RL) agents in high-stakes applications might be too prohibitive due to the risk associated to exploration. Thus, the agent can only use data previously collected by safe policies. While previous work considers optimizing the average performance using offline data, we focus on optimizing a risk-averse criteria, namely the CVaR. In particular, we present the Offline Risk-Averse Actor-Critic (O-RAAC), a model-free RL algorithm that is able to learn risk-averse policies in a fully offline setting. We show that O-RAAC learns policies with higher CVaR than risk-neutral approaches in different robot control tasks. Furthermore, considering risk-averse criteria guarantees distributional robustness of the average performance with respect to particular distribution shifts. We demonstrate empirically that in the presence of natural distribution-shifts, O-RAAC learns policies with good average performance.
△ Less
Submitted 10 February, 2021;
originally announced February 2021.
-
Efficient Pure Exploration for Combinatorial Bandits with Semi-Bandit Feedback
Authors:
Marc Jourdan,
Mojmír Mutný,
Johannes Kirschner,
Andreas Krause
Abstract:
Combinatorial bandits with semi-bandit feedback generalize multi-armed bandits, where the agent chooses sets of arms and observes a noisy reward for each arm contained in the chosen set. The action set satisfies a given structure such as forming a base of a matroid or a path in a graph. We focus on the pure-exploration problem of identifying the best arm with fixed confidence, as well as a more ge…
▽ More
Combinatorial bandits with semi-bandit feedback generalize multi-armed bandits, where the agent chooses sets of arms and observes a noisy reward for each arm contained in the chosen set. The action set satisfies a given structure such as forming a base of a matroid or a path in a graph. We focus on the pure-exploration problem of identifying the best arm with fixed confidence, as well as a more general setting, where the structure of the answer set differs from the one of the action set. Using the recently popularized game framework, we interpret this problem as a sequential zero-sum game and develop a CombGame meta-algorithm whose instances are asymptotically optimal algorithms with finite time guarantees. In addition to comparing two families of learners to instantiate our meta-algorithm, the main contribution of our work is a specific oracle efficient instance for best-arm identification with combinatorial actions. Based on a projection-free online learning algorithm for convex polytopes, it is the first computationally efficient algorithm which is asymptotically optimal and has competitive empirical performance.
△ Less
Submitted 21 January, 2021;
originally announced January 2021.
-
Safe and Efficient Model-free Adaptive Control via Bayesian Optimization
Authors:
Christopher König,
Matteo Turchetta,
John Lygeros,
Alisa Rupenyan,
Andreas Krause
Abstract:
Adaptive control approaches yield high-performance controllers when a precise system model or suitable parametrizations of the controller are available. Existing data-driven approaches for adaptive control mostly augment standard model-based methods with additional information about uncertainties in the dynamics or about disturbances. In this work, we propose a purely data-driven, model-free appro…
▽ More
Adaptive control approaches yield high-performance controllers when a precise system model or suitable parametrizations of the controller are available. Existing data-driven approaches for adaptive control mostly augment standard model-based methods with additional information about uncertainties in the dynamics or about disturbances. In this work, we propose a purely data-driven, model-free approach for adaptive control. Tuning low-level controllers based solely on system data raises concerns on the underlying algorithm safety and computational performance. Thus, our approach builds on GoOSE, an algorithm for safe and sample-efficient Bayesian optimization. We introduce several computational and algorithmic modifications in GoOSE that enable its practical use on a rotational motion system. We numerically demonstrate for several types of disturbances that our approach is sample efficient, outperforms constrained Bayesian optimization in terms of safety, and achieves the performance optima computed by grid evaluation. We further demonstrate the proposed adaptive control approach experimentally on a rotational motion system.
△ Less
Submitted 2 March, 2021; v1 submitted 19 January, 2021;
originally announced January 2021.
-
Incentive-Compatible Forecasting Competitions
Authors:
Jens Witkowski,
Rupert Freeman,
Jennifer Wortman Vaughan,
David M. Pennock,
Andreas Krause
Abstract:
We initiate the study of incentive-compatible forecasting competitions in which multiple forecasters make predictions about one or more events and compete for a single prize. We have two objectives: (1) to incentivize forecasters to report truthfully and (2) to award the prize to the most accurate forecaster. Proper scoring rules incentivize truthful reporting if all forecasters are paid according…
▽ More
We initiate the study of incentive-compatible forecasting competitions in which multiple forecasters make predictions about one or more events and compete for a single prize. We have two objectives: (1) to incentivize forecasters to report truthfully and (2) to award the prize to the most accurate forecaster. Proper scoring rules incentivize truthful reporting if all forecasters are paid according to their scores. However, incentives become distorted if only the best-scoring forecaster wins a prize, since forecasters can often increase their probability of having the highest score by reporting more extreme beliefs. In this paper, we introduce two novel forecasting competition mechanisms. Our first mechanism is incentive compatible and guaranteed to select the most accurate forecaster with probability higher than any other forecaster. Moreover, we show that in the standard single-event, two-forecaster setting and under mild technical conditions, no other incentive-compatible mechanism selects the most accurate forecaster with higher probability. Our second mechanism is incentive compatible when forecasters' beliefs are such that information about one event does not lead to belief updates on other events, and it selects the best forecaster with probability approaching 1 as the number of events grows. Our notion of incentive compatibility is more general than previous definitions of dominant strategy incentive compatibility in that it allows for reports to be correlated with the event outcomes. Moreover, our mechanisms are easy to implement and can be generalized to the related problems of outputting a ranking over forecasters and hiring a forecaster with high accuracy on future events.
△ Less
Submitted 7 September, 2021; v1 submitted 5 January, 2021;
originally announced January 2021.