-
Non-Sequential Ensemble Kalman Filtering using Distributed Arrays
Authors:
Cédric Travelletti,
Jörg Franke,
David Ginsbourger,
Stefan Brönnimann
Abstract:
This work introduces a new, distributed implementation of the Ensemble Kalman Filter (EnKF) that allows for non-sequential assimilation of large datasets in high-dimensional problems. The traditional EnKF algorithm is computationally intensive and exhibits difficulties in applications requiring interaction with the background covariance matrix, prompting the use of methods like sequential assimila…
▽ More
This work introduces a new, distributed implementation of the Ensemble Kalman Filter (EnKF) that allows for non-sequential assimilation of large datasets in high-dimensional problems. The traditional EnKF algorithm is computationally intensive and exhibits difficulties in applications requiring interaction with the background covariance matrix, prompting the use of methods like sequential assimilation which can introduce unwanted consequences, such as dependency on observation ordering. Our implementation leverages recent advancements in distributed computing to enable the construction and use of the full model error covariance matrix in distributed memory, allowing for single-batch assimilation of all observations and eliminating order dependencies. Comparative performance assessments, involving both synthetic and real-world paleoclimatic reconstruction applications, indicate that the new, non-sequential implementation outperforms the traditional, sequential one.
△ Less
Submitted 21 November, 2023;
originally announced November 2023.
-
Adaptive Quantile Computation for Brownian Bridge in Change-Point Analysis
Authors:
Jürgen Franke,
Mario Hefter,
André Herzwurm,
Klaus Ritter,
Stefanie Schwaar
Abstract:
As an example for the fast calculation of distributional parameters of Gaussian processes, we propose a new Monte Carlo algorithm for the computation of quantiles of the supremum norm of weighted Brownian bridges. As it is known, the corresponding distributions arise asymptotically for weighted CUSUM statistics for change-point detection. The new algorithm employs an adaptive (sequential) time dis…
▽ More
As an example for the fast calculation of distributional parameters of Gaussian processes, we propose a new Monte Carlo algorithm for the computation of quantiles of the supremum norm of weighted Brownian bridges. As it is known, the corresponding distributions arise asymptotically for weighted CUSUM statistics for change-point detection. The new algorithm employs an adaptive (sequential) time discretization for the trajectories of the Brownian bridge. A simulation study shows that the new algorithm by far outperforms the standard approach, which employs a uniform time discretization.
△ Less
Submitted 31 December, 2020;
originally announced January 2021.
-
Sample-Efficient Automated Deep Reinforcement Learning
Authors:
Jörg K. H. Franke,
Gregor Köhler,
André Biedenkapp,
Frank Hutter
Abstract:
Despite significant progress in challenging problems across various domains, applying state-of-the-art deep reinforcement learning (RL) algorithms remains challenging due to their sensitivity to the choice of hyperparameters. This sensitivity can partly be attributed to the non-stationarity of the RL problem, potentially requiring different hyperparameter settings at various stages of the learning…
▽ More
Despite significant progress in challenging problems across various domains, applying state-of-the-art deep reinforcement learning (RL) algorithms remains challenging due to their sensitivity to the choice of hyperparameters. This sensitivity can partly be attributed to the non-stationarity of the RL problem, potentially requiring different hyperparameter settings at various stages of the learning process. Additionally, in the RL setting, hyperparameter optimization (HPO) requires a large number of environment interactions, hindering the transfer of the successes in RL to real-world applications. In this work, we tackle the issues of sample-efficient and dynamic HPO in RL. We propose a population-based automated RL (AutoRL) framework to meta-optimize arbitrary off-policy RL algorithms. In this framework, we optimize the hyperparameters and also the neural architecture while simultaneously training the agent. By sharing the collected experience across the population, we substantially increase the sample efficiency of the meta-optimization. We demonstrate the capabilities of our sample-efficient AutoRL approach in a case study with the popular TD3 algorithm in the MuJoCo benchmark suite, where we reduce the number of environment interactions needed for meta-optimization by up to an order of magnitude compared to population-based training.
△ Less
Submitted 17 March, 2021; v1 submitted 3 September, 2020;
originally announced September 2020.
-
Neural Architecture Evolution in Deep Reinforcement Learning for Continuous Control
Authors:
Jörg K. H. Franke,
Gregor Köhler,
Noor Awad,
Frank Hutter
Abstract:
Current Deep Reinforcement Learning algorithms still heavily rely on handcrafted neural network architectures. We propose a novel approach to automatically find strong topologies for continuous control tasks while only adding a minor overhead in terms of interactions in the environment. To achieve this, we combine Neuroevolution techniques with off-policy training and propose a novel architecture…
▽ More
Current Deep Reinforcement Learning algorithms still heavily rely on handcrafted neural network architectures. We propose a novel approach to automatically find strong topologies for continuous control tasks while only adding a minor overhead in terms of interactions in the environment. To achieve this, we combine Neuroevolution techniques with off-policy training and propose a novel architecture mutation operator. Experiments on five continuous control benchmarks show that the proposed Actor-Critic Neuroevolution algorithm often outperforms the strong Actor-Critic baseline and is capable of automatically finding topologies in a sample-efficient manner which would otherwise have to be found by expensive architecture search.
△ Less
Submitted 27 February, 2020; v1 submitted 28 October, 2019;
originally announced October 2019.
-
Correlating Paleoclimate Time Series: Sources of Uncertainty and Potential Pitfalls
Authors:
Jasper G. Franke,
Reik V. Donner
Abstract:
Comparing paleoclimate time series is complicated by a variety of typical features, including irregular sampling, age model uncertainty (e.g., errors due to interpolation between radiocarbon sampling points) and time uncertainty (uncertainty in calibration), which, taken together, result in unequal and uncertain observation times of the individual time series to be correlated. Several methods have…
▽ More
Comparing paleoclimate time series is complicated by a variety of typical features, including irregular sampling, age model uncertainty (e.g., errors due to interpolation between radiocarbon sampling points) and time uncertainty (uncertainty in calibration), which, taken together, result in unequal and uncertain observation times of the individual time series to be correlated. Several methods have been proposed to approximate the joint probability distribution needed to estimate correlations, most of which rely either on interpolation or temporal downsampling.
Here, we compare the performance of some popular approximation methods using synthetic data resembling common properties of real world marine sediment records. Correlations are determined by estimating the parameters of a bivariate Gaussian model from the data using Markov Chain Monte Carlo sampling. We complement our pseudoproxy experiments by applying the same methodology to a pair of marine benthic oxygen records from the Atlantic Ocean.
We find that methods based upon interpolation yield better results in terms of precision and accuracy than those which reduce the number of observations. In all cases, the specific characteristics of the studied time series are, however, more important than the choice of a particular interpolation method. Relevant features include the number of observations, the persistence of each record, and the imposed coupling strength between the paired series. In most of our pseudoproxy experiments, uncertainty in observation times introduces less additional uncertainty than unequal sampling and errors in observation times do. Thus, it can be reasonable to rely on published time scales as long as calibration uncertainties are not known.
△ Less
Submitted 28 March, 2019;
originally announced March 2019.
