-
Simulating, Fast and Slow: Learning Policies for Black-Box Optimization
Authors:
Fabio Valerio Massoli,
Tim Bakker,
Thomas Hehn,
Tribhuvanesh Orekondy,
Arash Behboodi
Abstract:
In recent years, solving optimization problems involving black-box simulators has become a point of focus for the machine learning community due to their ubiquity in science and engineering. The simulators describe a forward process $f_{\mathrm{sim}}: (ψ, x) \rightarrow y$ from simulation parameters $ψ$ and input data $x$ to observations $y$, and the goal of the optimization problem is to find par…
▽ More
In recent years, solving optimization problems involving black-box simulators has become a point of focus for the machine learning community due to their ubiquity in science and engineering. The simulators describe a forward process $f_{\mathrm{sim}}: (ψ, x) \rightarrow y$ from simulation parameters $ψ$ and input data $x$ to observations $y$, and the goal of the optimization problem is to find parameters $ψ$ that minimize a desired loss function. Sophisticated optimization algorithms typically require gradient information regarding the forward process, $f_{\mathrm{sim}}$, with respect to the parameters $ψ$. However, obtaining gradients from black-box simulators can often be prohibitively expensive or, in some cases, impossible. Furthermore, in many applications, practitioners aim to solve a set of related problems. Thus, starting the optimization ``ab initio", i.e. from scratch, each time might be inefficient if the forward model is expensive to evaluate. To address those challenges, this paper introduces a novel method for solving classes of similar black-box optimization problems by learning an active learning policy that guides a differentiable surrogate's training and uses the surrogate's gradients to optimize the simulation parameters with gradient descent. After training the policy, downstream optimization of problems involving black-box simulators requires up to $\sim$90\% fewer expensive simulator calls compared to baselines such as local surrogate-based approaches, numerical optimization, and Bayesian methods.
△ Less
Submitted 6 June, 2024;
originally announced June 2024.
-
Learning Objective-Specific Active Learning Strategies with Attentive Neural Processes
Authors:
Tim Bakker,
Herke van Hoof,
Max Welling
Abstract:
Pool-based active learning (AL) is a promising technology for increasing data-efficiency of machine learning models. However, surveys show that performance of recent AL methods is very sensitive to the choice of dataset and training setting, making them unsuitable for general application. In order to tackle this problem, the field Learning Active Learning (LAL) suggests to learn the active learnin…
▽ More
Pool-based active learning (AL) is a promising technology for increasing data-efficiency of machine learning models. However, surveys show that performance of recent AL methods is very sensitive to the choice of dataset and training setting, making them unsuitable for general application. In order to tackle this problem, the field Learning Active Learning (LAL) suggests to learn the active learning strategy itself, allowing it to adapt to the given setting. In this work, we propose a novel LAL method for classification that exploits symmetry and independence properties of the active learning problem with an Attentive Conditional Neural Process model. Our approach is based on learning from a myopic oracle, which gives our model the ability to adapt to non-standard objectives, such as those that do not equally weight the error on all data points. We experimentally verify that our Neural Process model outperforms a variety of baselines in these settings. Finally, our experiments show that our model exhibits a tendency towards improved stability to changing datasets. However, performance is sensitive to choice of classifier and more work is necessary to reduce the performance the gap with the myopic oracle and to improve scalability. We present our work as a proof-of-concept for LAL on nonstandard objectives and hope our analysis and modelling considerations inspire future LAL work.
△ Less
Submitted 11 September, 2023;
originally announced September 2023.
-
E-Valuating Classifier Two-Sample Tests
Authors:
Teodora Pandeva,
Tim Bakker,
Christian A. Naesseth,
Patrick Forré
Abstract:
We introduce a powerful deep classifier two-sample test for high-dimensional data based on E-values, called E-value Classifier Two-Sample Test (E-C2ST). Our test combines ideas from existing work on split likelihood ratio tests and predictive independence tests. The resulting E-values are suitable for anytime-valid sequential two-sample tests. This feature allows for more effective use of data in…
▽ More
We introduce a powerful deep classifier two-sample test for high-dimensional data based on E-values, called E-value Classifier Two-Sample Test (E-C2ST). Our test combines ideas from existing work on split likelihood ratio tests and predictive independence tests. The resulting E-values are suitable for anytime-valid sequential two-sample tests. This feature allows for more effective use of data in constructing test statistics. Through simulations and real data applications, we empirically demonstrate that E-C2ST achieves enhanced statistical power by partitioning datasets into multiple batches beyond the conventional two-split (training and testing) approach of standard classifier two-sample tests. This strategy increases the power of the test while kee** the type I error well below the desired significance level.
△ Less
Submitted 30 April, 2024; v1 submitted 24 October, 2022;
originally announced October 2022.
-
Three-Electrode Cell Calorimeter for Electrical Double Layer Capacitors
Authors:
Joren E. Vos,
Hendrik P. Rodenburg,
Danny Inder Maur,
Ties J. W. Bakker,
Henkjan Siekman,
Ben H. Erné
Abstract:
A calorimeter was built to measure the heat from a porous capacitive working electrode connected in a three-electrode configuration. This makes it possible to detect differences between cathodic and anodic heat production. The electrochemical cell contains a large electrolyte solution reservoir, ensuring a constant concentration of the salt solution probed by the reference electrode via a Luggin t…
▽ More
A calorimeter was built to measure the heat from a porous capacitive working electrode connected in a three-electrode configuration. This makes it possible to detect differences between cathodic and anodic heat production. The electrochemical cell contains a large electrolyte solution reservoir, ensuring a constant concentration of the salt solution probed by the reference electrode via a Luggin tube. A heat flux sensor is used to detect the heat, and its calibration as a gauge of the total amount of heat produced by the electrode is done on the basis of the net electrical work performed on the working electrode during a full charging-discharging cycle. In principle, from the measured heat and the electrical work, the change in internal energy of the working electrode can be determined as a function of applied potential. Such measurements inform about the potential energy and average electric potential of ions inside the pores, giving insight into the electrical double layer inside electrode micropores. Example measurements of the heat are shown for porous carbon electrodes in aqueous salt solution.
△ Less
Submitted 3 October, 2022;
originally announced October 2022.
-
On learning adaptive acquisition policies for undersampled multi-coil MRI reconstruction
Authors:
Tim Bakker,
Matthew Muckley,
Adriana Romero-Soriano,
Michal Drozdzal,
Luis Pineda
Abstract:
Most current approaches to undersampled multi-coil MRI reconstruction focus on learning the reconstruction model for a fixed, equidistant acquisition trajectory. In this paper, we study the problem of joint learning of the reconstruction model together with acquisition policies. To this end, we extend the End-to-End Variational Network with learnable acquisition policies that can adapt to differen…
▽ More
Most current approaches to undersampled multi-coil MRI reconstruction focus on learning the reconstruction model for a fixed, equidistant acquisition trajectory. In this paper, we study the problem of joint learning of the reconstruction model together with acquisition policies. To this end, we extend the End-to-End Variational Network with learnable acquisition policies that can adapt to different data points. We validate our model on a coil-compressed version of the large scale undersampled multi-coil fastMRI dataset using two undersampling factors: $4\times$ and $8\times$. Our experiments show on-par performance with the learnable non-adaptive and handcrafted equidistant strategies at $4\times$, and an observed improvement of more than $2\%$ in SSIM at $8\times$ acceleration, suggesting that potentially-adaptive $k$-space acquisition trajectories can improve reconstructed image quality for larger acceleration factors. However, and perhaps surprisingly, our best performing policies learn to be explicitly non-adaptive.
△ Less
Submitted 30 March, 2022;
originally announced March 2022.
-
Effects of microplastics and surfactants on surface roughness of water waves
Authors:
Yukun Sun,
Christopher Ruf,
Thomas Bakker,
Yulin Pan
Abstract:
In this paper, we study the flow physics underlying the recently developed remote sensing capability of detecting oceanic microplastics, which is based on the measurable surface roughness reduction induced by the presence of microplastics on the ocean surface. In particular, we are interested in whether this roughness reduction is caused by the microplastics as floating particles, or by the surfac…
▽ More
In this paper, we study the flow physics underlying the recently developed remote sensing capability of detecting oceanic microplastics, which is based on the measurable surface roughness reduction induced by the presence of microplastics on the ocean surface. In particular, we are interested in whether this roughness reduction is caused by the microplastics as floating particles, or by the surfactants which follow similar transport paths as microplastics. For this purpose, we experimentally test the effects of floating particles and surfactants on surface roughness, quantified by the mean square slope (MSS), with waves generated by a mechanical wave maker or by wind. For microplastics, we find that their effect on wave energy and MSS critically depends on the surface area fraction of coverage, irrespective of the particle sizes in the test range. The dam** by particles is observed only for fractions above $O(5-10\%)$, which is much higher than the realistic ocean condition. For surfactants, their dam** effect on mechanically generated irregular waves generally increases with the concentration of surfactants, but no optimal concentration corresponding to maximum dam** is observed, in contrast to previous studies based on monochromatic waves. In wind-wave experiments, the presence of surfactants suppresses the wave generation, due to the combined effects of reduced wind shear stress and increased wave dam**. For the same wind speed, the wind stress is identified to depend on the concentration of surfactants with a power-law relation. The implications of these findings to remote sensing are discussed.
△ Less
Submitted 12 November, 2021;
originally announced November 2021.
-
Back to Basics: Deep Reinforcement Learning in Traffic Signal Control
Authors:
Sierk Kanis,
Laurens Samson,
Daan Bloembergen,
Tim Bakker
Abstract:
In this paper we revisit some of the fundamental premises for a reinforcement learning (RL) approach to self-learning traffic lights. We propose RLight, a combination of choices that offers robust performance and good generalization to unseen traffic flows. In particular, our main contributions are threefold: our lightweight and cluster-aware state representation leads to improved performance; we…
▽ More
In this paper we revisit some of the fundamental premises for a reinforcement learning (RL) approach to self-learning traffic lights. We propose RLight, a combination of choices that offers robust performance and good generalization to unseen traffic flows. In particular, our main contributions are threefold: our lightweight and cluster-aware state representation leads to improved performance; we reformulate the Markov Decision Process (MDP) such that it skips redundant timesteps of yellow light, speeding up learning by 30%; and we investigate the action space and provide insight into the difference in performance between acyclic and cyclic phase transitions. Additionally, we provide insights into the generalisation of the methods to unseen traffic. Evaluations using the real-world Hangzhou traffic dataset show that RLight outperforms state-of-the-art rule-based and deep reinforcement learning algorithms, demonstrating the potential of RL-based methods to improve urban traffic flows.
△ Less
Submitted 21 November, 2021; v1 submitted 15 September, 2021;
originally announced September 2021.
-
Experimental design for MRI by greedy policy search
Authors:
Tim Bakker,
Herke van Hoof,
Max Welling
Abstract:
In today's clinical practice, magnetic resonance imaging (MRI) is routinely accelerated through subsampling of the associated Fourier domain. Currently, the construction of these subsampling strategies - known as experimental design - relies primarily on heuristics. We propose to learn experimental design strategies for accelerated MRI with policy gradient methods. Unexpectedly, our experiments sh…
▽ More
In today's clinical practice, magnetic resonance imaging (MRI) is routinely accelerated through subsampling of the associated Fourier domain. Currently, the construction of these subsampling strategies - known as experimental design - relies primarily on heuristics. We propose to learn experimental design strategies for accelerated MRI with policy gradient methods. Unexpectedly, our experiments show that a simple greedy approximation of the objective leads to solutions nearly on-par with the more general non-greedy approach. We offer a partial explanation for this phenomenon rooted in greater variance in the non-greedy objective's gradient estimates, and experimentally verify that this variance hampers non-greedy models in adapting their policies to individual MR images. We empirically show that this adaptivity is key to improving subsampling designs.
△ Less
Submitted 15 December, 2020; v1 submitted 30 October, 2020;
originally announced October 2020.
