-
A Gauss-Newton Method for ODE Optimal Tracking Control
Authors:
Vicky Holfeld,
Michael Burger,
Claudia Schillings
Abstract:
This paper introduces and analyses a continuous optimization approach to solve optimal control problems involving ordinary differential equations (ODEs) and tracking type objectives. Our aim is to determine control or input functions, and potentially uncertain model parameters, for a dynamical system described by an ODE. We establish the mathematical framework and define the optimal control proble…
▽ More
This paper introduces and analyses a continuous optimization approach to solve optimal control problems involving ordinary differential equations (ODEs) and tracking type objectives. Our aim is to determine control or input functions, and potentially uncertain model parameters, for a dynamical system described by an ODE. We establish the mathematical framework and define the optimal control problem with a tracking functional, incorporating regularization terms and box-constraints for model parameters and input functions. Treating the problem as an infinite-dimensional optimization problem, we employ a Gauss-Newton method within a suitable function space framework. This leads to an iterative process where, at each step, we solve a linearization of the problem by considering a linear surrogate model around the current solution estimate. The resulting linear auxiliary problem resembles a linear-quadratic ODE optimal tracking control problem, which we tackle using either a gradient descent method in function spaces or a Riccati-based approach. Finally, we present and analyze the efficacy of our method through numerical experiments.
△ Less
Submitted 8 May, 2024;
originally announced May 2024.
-
Quasi-Monte Carlo for Bayesian design of experiment problems governed by parametric PDEs
Authors:
Vesa Kaarnioja,
Claudia Schillings
Abstract:
This paper contributes to the study of optimal experimental design for Bayesian inverse problems governed by partial differential equations (PDEs). We derive estimates for the parametric regularity of multivariate double integration problems over high-dimensional parameter and data domains arising in Bayesian optimal design problems. We provide a detailed analysis for these double integration prob…
▽ More
This paper contributes to the study of optimal experimental design for Bayesian inverse problems governed by partial differential equations (PDEs). We derive estimates for the parametric regularity of multivariate double integration problems over high-dimensional parameter and data domains arising in Bayesian optimal design problems. We provide a detailed analysis for these double integration problems using two approaches: a full tensor product and a sparse tensor product combination of quasi-Monte Carlo (QMC) cubature rules over the parameter and data domains. Specifically, we show that the latter approach significantly improves the convergence rate, exhibiting performance comparable to that of QMC integration of a single high-dimensional integral. Furthermore, we numerically verify the predicted convergence rates for an elliptic PDE problem with an unknown diffusion coefficient in two spatial dimensions, offering empirical evidence supporting the theoretical results and highlighting practical applicability.
△ Less
Submitted 9 May, 2024; v1 submitted 6 May, 2024;
originally announced May 2024.
-
Generative Modelling with Tensor Train approximations of Hamilton--Jacobi--Bellman equations
Authors:
David Sommer,
Robert Gruhlke,
Max Kirstein,
Martin Eigel,
Claudia Schillings
Abstract:
Sampling from probability densities is a common challenge in fields such as Uncertainty Quantification (UQ) and Generative Modelling (GM). In GM in particular, the use of reverse-time diffusion processes depending on the log-densities of Ornstein-Uhlenbeck forward processes are a popular sampling tool. In Berner et al. [2022] the authors point out that these log-densities can be obtained by soluti…
▽ More
Sampling from probability densities is a common challenge in fields such as Uncertainty Quantification (UQ) and Generative Modelling (GM). In GM in particular, the use of reverse-time diffusion processes depending on the log-densities of Ornstein-Uhlenbeck forward processes are a popular sampling tool. In Berner et al. [2022] the authors point out that these log-densities can be obtained by solution of a \textit{Hamilton-Jacobi-Bellman} (HJB) equation known from stochastic optimal control. While this HJB equation is usually treated with indirect methods such as policy iteration and unsupervised training of black-box architectures like Neural Networks, we propose instead to solve the HJB equation by direct time integration, using compressed polynomials represented in the Tensor Train (TT) format for spatial discretization. Crucially, this method is sample-free, agnostic to normalization constants and can avoid the curse of dimensionality due to the TT compression. We provide a complete derivation of the HJB equation's action on Tensor Train polynomials and demonstrate the performance of the proposed time-step-, rank- and degree-adaptive integration method on a nonlinear sampling task in 20 dimensions.
△ Less
Submitted 23 February, 2024;
originally announced February 2024.
-
Ensemble Kalman Inversion for Image Guided Guide Wire Navigation in Vascular Systems
Authors:
Matei Hanu,
Jürgen Hesser,
Guido Kanschat,
Javier Moviglia,
Claudia Schillings,
Jan Stallkamp
Abstract:
This paper addresses the challenging task of guide wire navigation in cardiovascular interventions, focusing on the parameter estimation of a guide wire system using Ensemble Kalman Inversion (EKI) with a subsampling technique. The EKI uses an ensemble of particles to estimate the unknown quantities. However since the data misfit has to be computed for each particle in each iteration, the EKI may…
▽ More
This paper addresses the challenging task of guide wire navigation in cardiovascular interventions, focusing on the parameter estimation of a guide wire system using Ensemble Kalman Inversion (EKI) with a subsampling technique. The EKI uses an ensemble of particles to estimate the unknown quantities. However since the data misfit has to be computed for each particle in each iteration, the EKI may become computationally infeasible in the case of high-dimensional data, e.g. high-resolution images. This issue can been addressed by randomised algorithms that utilize only a random subset of the data in each iteration. We introduce and analyse a subsampling technique for the EKI, which is based on a continuous-time representation of stochastic gradient methods and apply it to on the parameter estimation of our guide wire system. Numerical experiments with real data from a simplified test setting demonstrate the potential of the method.
△ Less
Submitted 11 December, 2023;
originally announced December 2023.
-
On the concentration of subgaussian vectors and positive quadratic forms in Hilbert spaces
Authors:
Mattes Mollenhauer,
Claudia Schillings
Abstract:
In these notes, we investigate the tail behaviour of the norm of subgaussian vectors in a Hilbert space. The subgaussian variance proxy is given as a trace class operator, allowing for a precise control of the moments along each dimension of the space. This leads to useful extensions and analogues of known Hoeffding-type inequalities and deviation bounds for positive random quadratic forms. We giv…
▽ More
In these notes, we investigate the tail behaviour of the norm of subgaussian vectors in a Hilbert space. The subgaussian variance proxy is given as a trace class operator, allowing for a precise control of the moments along each dimension of the space. This leads to useful extensions and analogues of known Hoeffding-type inequalities and deviation bounds for positive random quadratic forms. We give a straightforward application in terms of a variance bound for the regularisation of statistical inverse problems.
△ Less
Submitted 3 October, 2023; v1 submitted 20 June, 2023;
originally announced June 2023.
-
Subsampling in ensemble Kalman inversion
Authors:
Matei Hanu,
Jonas Latz,
Claudia Schillings
Abstract:
We consider the Ensemble Kalman Inversion which has been recently introduced as an efficient, gradient-free optimisation method to estimate unknown parameters in an inverse setting. In the case of large data sets, the Ensemble Kalman Inversion becomes computationally infeasible as the data misfit needs to be evaluated for each particle in each iteration. Here, randomised algorithms like stochastic…
▽ More
We consider the Ensemble Kalman Inversion which has been recently introduced as an efficient, gradient-free optimisation method to estimate unknown parameters in an inverse setting. In the case of large data sets, the Ensemble Kalman Inversion becomes computationally infeasible as the data misfit needs to be evaluated for each particle in each iteration. Here, randomised algorithms like stochastic gradient descent have been demonstrated to successfully overcome this issue by using only a random subset of the data in each iteration, so-called subsampling techniques. Based on a recent analysis of a continuous-time representation of stochastic gradient methods, we propose, analyse, and apply subsampling-techniques within Ensemble Kalman Inversion. Indeed, we propose two different subsampling techniques: either every particle observes the same data subset (single subsampling) or every particle observes a different data subset (batch subsampling).
△ Less
Submitted 4 December, 2023; v1 submitted 22 February, 2023;
originally announced February 2023.
-
Ensemble-based gradient inference for particle methods in optimization and sampling
Authors:
Claudia Schillings,
Claudia Totzeck,
Philipp Wacker
Abstract:
We propose an approach based on function evaluations and Bayesian inference to extract higher-order differential information of objective functions {from a given ensemble of particles}. Pointwise evaluation $\{V(x^i)\}_i$ of some potential $V$ in an ensemble $\{x^i\}_i$ contains implicit information about first or higher order derivatives, which can be made explicit with little computational effor…
▽ More
We propose an approach based on function evaluations and Bayesian inference to extract higher-order differential information of objective functions {from a given ensemble of particles}. Pointwise evaluation $\{V(x^i)\}_i$ of some potential $V$ in an ensemble $\{x^i\}_i$ contains implicit information about first or higher order derivatives, which can be made explicit with little computational effort (ensemble-based gradient inference -- EGI). We suggest to use this information for the improvement of established ensemble-based numerical methods for optimization and sampling such as Consensus-based optimization and Langevin-based samplers. Numerical studies indicate that the augmented algorithms are often superior to their gradient-free variants, in particular the augmented methods help the ensembles to escape their initial domain, to explore multimodal, non-Gaussian settings and to speed up the collapse at the end of optimization dynamics.}
The code for the numerical examples in this manuscript can be found in the paper's Github repository (https://github.com/MercuryBench/ensemble-based-gradient.git).
△ Less
Submitted 1 March, 2023; v1 submitted 23 September, 2022;
originally announced September 2022.
-
Parabolic PDE-constrained optimal control under uncertainty with entropic risk measure using quasi-Monte Carlo integration
Authors:
Philipp A. Guth,
Vesa Kaarnioja,
Frances Y. Kuo,
Claudia Schillings,
Ian H. Sloan
Abstract:
We study the application of a tailored quasi-Monte Carlo (QMC) method to a class of optimal control problems subject to parabolic partial differential equation (PDE) constraints under uncertainty: the state in our setting is the solution of a parabolic PDE with a random thermal diffusion coefficient, steered by a control function. To account for the presence of uncertainty in the optimal control p…
▽ More
We study the application of a tailored quasi-Monte Carlo (QMC) method to a class of optimal control problems subject to parabolic partial differential equation (PDE) constraints under uncertainty: the state in our setting is the solution of a parabolic PDE with a random thermal diffusion coefficient, steered by a control function. To account for the presence of uncertainty in the optimal control problem, the objective function is composed with a risk measure. We focus on two risk measures, both involving high-dimensional integrals over the stochastic variables: the expected value and the (nonlinear) entropic risk measure. The high-dimensional integrals are computed numerically using specially designed QMC methods and, under moderate assumptions on the input random field, the error rate is shown to be essentially linear, independently of the stochastic dimension of the problem -- and thereby superior to ordinary Monte Carlo methods. Numerical results demonstrate the effectiveness of our method.
△ Less
Submitted 27 March, 2024; v1 submitted 4 August, 2022;
originally announced August 2022.
-
One-shot Learning of Surrogates in PDE-constrained Optimization Under Uncertainty
Authors:
Philipp A. Guth,
Claudia Schillings,
Simon Weissmann
Abstract:
We propose a general framework for machine learning based optimization under uncertainty. Our approach replaces the complex forward model by a surrogate, which is learned simultaneously in a one-shot sense when solving the optimal control problem. Our approach relies on a reformulation of the problem as a penalized empirical risk minimization problem for which we provide a consistency analysis in…
▽ More
We propose a general framework for machine learning based optimization under uncertainty. Our approach replaces the complex forward model by a surrogate, which is learned simultaneously in a one-shot sense when solving the optimal control problem. Our approach relies on a reformulation of the problem as a penalized empirical risk minimization problem for which we provide a consistency analysis in terms of large data and increasing penalty parameter. To solve the resulting problem, we suggest a stochastic gradient method with adaptive control of the penalty parameter and prove convergence under suitable assumptions on the surrogate model. Numerical experiments illustrate the results for linear and nonlinear surrogate models.
△ Less
Submitted 22 December, 2023; v1 submitted 21 December, 2021;
originally announced December 2021.
-
Adaptive Tikhonov strategies for stochastic ensemble Kalman inversion
Authors:
Simon Weissmann,
Neil K. Chada,
Claudia Schillings,
Xin T. Tong
Abstract:
Ensemble Kalman inversion (EKI) is a derivative-free optimizer aimed at solving inverse problems, taking motivation from the celebrated ensemble Kalman filter. The purpose of this article is to consider the introduction of adaptive Tikhonov strategies for EKI. This work builds upon Tikhonov EKI (TEKI) which was proposed for a fixed regularization constant. By adaptively learning the regularization…
▽ More
Ensemble Kalman inversion (EKI) is a derivative-free optimizer aimed at solving inverse problems, taking motivation from the celebrated ensemble Kalman filter. The purpose of this article is to consider the introduction of adaptive Tikhonov strategies for EKI. This work builds upon Tikhonov EKI (TEKI) which was proposed for a fixed regularization constant. By adaptively learning the regularization parameter, this procedure is known to improve the recovery of the underlying unknown. For the analysis, we consider a continuous-time setting where we extend known results such as well-posdeness and convergence of various loss functions, but with the addition of noisy observations. Furthermore, we allow a time-varying noise and regularization covariance in our presented convergence result which mimic adaptive regularization schemes. In turn we present three adaptive regularization schemes, which are highlighted from both the deterministic and Bayesian approaches for inverse problems, which include bilevel optimization, the MAP formulation and covariance learning. We numerically test these schemes and the theory on linear and nonlinear partial differential equations, where they outperform the non-adaptive TEKI and EKI.
△ Less
Submitted 18 October, 2021;
originally announced October 2021.
-
Continuous time limit of the stochastic ensemble Kalman inversion: Strong convergence analysis
Authors:
Dirk Blömker,
Claudia Schillings,
Philipp Wacker,
Simon Weissmann
Abstract:
The Ensemble Kalman inversion (EKI) method is a method for the estimation of unknown parameters in the context of (Bayesian) inverse problems. The method approximates the underlying measure by an ensemble of particles and iteratively applies the ensemble Kalman update to evolve (the approximation of the) prior into the posterior measure.
For the convergence analysis of the EKI it is common pract…
▽ More
The Ensemble Kalman inversion (EKI) method is a method for the estimation of unknown parameters in the context of (Bayesian) inverse problems. The method approximates the underlying measure by an ensemble of particles and iteratively applies the ensemble Kalman update to evolve (the approximation of the) prior into the posterior measure.
For the convergence analysis of the EKI it is common practice to derive a continuous version, replacing the iteration with a stochastic differential equation. In this paper we validate this approach by showing that the stochastic EKI iteration converges to paths of the continuous-time stochastic differential equation by considering both the nonlinear and linear setting, and we prove convergence in probability for the former, and convergence in moments for the latter. The methods employed can also be applied to the analysis of more general numerical schemes for stochastic differential equations in general.
△ Less
Submitted 30 July, 2021;
originally announced July 2021.
-
Hierarchical surrogate-based Approximate Bayesian Computation for an electric motor test bench
Authors:
David N. John,
Livia Stohrer,
Claudia Schillings,
Michael Schick,
Vincent Heuveline
Abstract:
Inferring parameter distributions of complex industrial systems from noisy time series data requires methods to deal with the uncertainty of the underlying data and the used simulation model. Bayesian inference is well suited for these uncertain inverse problems. Standard methods used to identify uncertain parameters are Markov Chain Monte Carlo (MCMC) methods with explicit evaluation of a likelih…
▽ More
Inferring parameter distributions of complex industrial systems from noisy time series data requires methods to deal with the uncertainty of the underlying data and the used simulation model. Bayesian inference is well suited for these uncertain inverse problems. Standard methods used to identify uncertain parameters are Markov Chain Monte Carlo (MCMC) methods with explicit evaluation of a likelihood function. However, if the likelihood is very complex, such that its evaluation is computationally expensive, or even unknown in its explicit form, Approximate Bayesian Computation (ABC) methods provide a promising alternative. In this work both methods are first applied to artificially generated data and second on a real world problem, by using data of an electric motor test bench. We show that both methods are able to infer the distribution of varying parameters with a Bayesian hierarchical approach. But the proposed ABC method is computationally much more efficient in order to achieve results with similar accuracy. We suggest to use summary statistics in order to reduce the dimension of the data which significantly increases the efficiency of the algorithm. Further the simulation model is replaced by a Polynomial Chaos Expansion (PCE) surrogate to speed up model evaluations. We proof consistency for the proposed surrogate-based ABC method with summary statistics under mild conditions on the (approximated) forward model.
△ Less
Submitted 17 June, 2021;
originally announced June 2021.
-
Consistency analysis of bilevel data-driven learning in inverse problems
Authors:
Neil K. Chada,
Claudia Schillings,
Xin T. Tong,
Simon Weissmann
Abstract:
One fundamental problem when solving inverse problems is how to find regularization parameters. This article considers solving this problem using data-driven bilevel optimization, i.e. we consider the adaptive learning of the regularization parameter from data by means of optimization. This approach can be interpreted as solving an empirical risk minimization problem, and we analyze its performanc…
▽ More
One fundamental problem when solving inverse problems is how to find regularization parameters. This article considers solving this problem using data-driven bilevel optimization, i.e. we consider the adaptive learning of the regularization parameter from data by means of optimization. This approach can be interpreted as solving an empirical risk minimization problem, and we analyze its performance in the large data sample size limit for general nonlinear problems. We demonstrate how to implement our framework on linear inverse problems, where we can further show the inverse accuracy does not depend on the ambient space dimension. To reduce the associated computational cost, online numerical schemes are derived using the stochastic gradient descent method. We prove convergence of these numerical schemes under suitable assumptions on the forward problem. Numerical experiments are presented illustrating the theoretical results and demonstrating the applicability and efficiency of the proposed approaches for various linear and nonlinear inverse problems, including Darcy flow, the eikonal equation, and an image denoising example.
△ Less
Submitted 7 January, 2021; v1 submitted 6 July, 2020;
originally announced July 2020.
-
Ensemble Kalman filter for neural network based one-shot inversion
Authors:
Philipp A. Guth,
Claudia Schillings,
Simon Weissmann
Abstract:
We study the use of novel techniques arising in machine learning for inverse problems. Our approach replaces the complex forward model by a neural network, which is trained simultaneously in a one-shot sense when estimating the unknown parameters from data, i.e. the neural network is trained only for the unknown parameter. By establishing a link to the Bayesian approach to inverse problems, an alg…
▽ More
We study the use of novel techniques arising in machine learning for inverse problems. Our approach replaces the complex forward model by a neural network, which is trained simultaneously in a one-shot sense when estimating the unknown parameters from data, i.e. the neural network is trained only for the unknown parameter. By establishing a link to the Bayesian approach to inverse problems, an algorithmic framework is developed which ensures the feasibility of the parameter estimate w.r. to the forward model. We propose an efficient, derivative-free optimization method based on variants of the ensemble Kalman inversion. Numerical experiments show that the ensemble Kalman filter for neural network based one-shot inversion is a promising direction combining optimization and machine learning techniques for inverse problems.
△ Less
Submitted 14 September, 2020; v1 submitted 5 May, 2020;
originally announced May 2020.
-
A quasi-Monte Carlo Method for an Optimal Control Problem Under Uncertainty
Authors:
Philipp A. Guth,
Vesa Kaarnioja,
Frances Y. Kuo,
Claudia Schillings,
Ian H. Sloan
Abstract:
We study an optimal control problem under uncertainty, where the target function is the solution of an elliptic partial differential equation with random coefficients, steered by a control function. The robust formulation of the optimization problem is stated as a high-dimensional integration problem over the stochastic variables. It is well known that carrying out a high-dimensional numerical int…
▽ More
We study an optimal control problem under uncertainty, where the target function is the solution of an elliptic partial differential equation with random coefficients, steered by a control function. The robust formulation of the optimization problem is stated as a high-dimensional integration problem over the stochastic variables. It is well known that carrying out a high-dimensional numerical integration of this kind using a Monte Carlo method has a notoriously slow convergence rate; meanwhile, a faster rate of convergence can potentially be obtained by using sparse grid quadratures, but these lead to discretized systems that are non-convex due to the involvement of negative quadrature weights. In this paper, we analyze instead the application of a quasi-Monte Carlo method, which retains the desirable convexity structure of the system and has a faster convergence rate compared to ordinary Monte Carlo methods. In particular, we show that under moderate assumptions on the decay of the input random field, the error rate obtained by using a specially designed, randomly shifted rank-1 lattice quadrature rule is essentially inversely proportional to the number of quadrature nodes. The overall discretization error of the problem, consisting of the dimension truncation error, finite element discretization error and quasi-Monte Carlo quadrature error, is derived in detail. We assess the theoretical findings in numerical experiments.
△ Less
Submitted 22 October, 2019;
originally announced October 2019.
-
On the Incorporation of Box-Constraints for Ensemble Kalman Inversion
Authors:
Neil K. Chada,
Claudia Schillings,
Simon Weissmann
Abstract:
The Bayesian approach to inverse problems is widely used in practice to infer unknown parameters from noisy observations. In this framework, the ensemble Kalman inversion has been successfully applied for the quantification of uncertainties in various areas of applications. In recent years, a complete analysis of the method has been developed for linear inverse problems adopting an optimization vi…
▽ More
The Bayesian approach to inverse problems is widely used in practice to infer unknown parameters from noisy observations. In this framework, the ensemble Kalman inversion has been successfully applied for the quantification of uncertainties in various areas of applications. In recent years, a complete analysis of the method has been developed for linear inverse problems adopting an optimization viewpoint. However, many applications require the incorporation of additional constraints on the parameters, e.g. arising due to physical constraints. We propose a new variant of the ensemble Kalman inversion to include box constraints on the unknown parameters motivated by the theory of projected preconditioned gradient flows. Based on the continuous time limit of the constrained ensemble Kalman inversion, we discuss a complete convergence analysis for linear forward problems. We adopt techniques from filtering which are crucial in order to improve the performance and establish a correct descent, such as variance inflation. These benefits are highlighted through a number of numerical examples on various inverse problems based on partial differential equations.
△ Less
Submitted 14 October, 2019; v1 submitted 2 August, 2019;
originally announced August 2019.
-
Sampling Sup-Normalized Spectral Functions for Brown-Resnick Processes
Authors:
Marco Oesting,
Martin Schlather,
Claudia Schillings
Abstract:
Sup-normalized spectral functions form building blocks of max-stable and Pareto processes and therefore play an important role in modeling spatial extremes. For one of the most popular examples, the Brown-Resnick process, simulation is not straightforward. In this paper, we generalize two approaches for simulation via Markov Chain Monte Carlo methods and rejection sampling by introducing new class…
▽ More
Sup-normalized spectral functions form building blocks of max-stable and Pareto processes and therefore play an important role in modeling spatial extremes. For one of the most popular examples, the Brown-Resnick process, simulation is not straightforward. In this paper, we generalize two approaches for simulation via Markov Chain Monte Carlo methods and rejection sampling by introducing new classes of proposal densities. In both cases, we provide an optimal choice of the proposal density with respect to sampling efficiency. The performance of the procedures is demonstrated in an example.
△ Less
Submitted 25 February, 2019;
originally announced February 2019.
-
On the Convergence of the Laplace Approximation and Noise-Level-Robustness of Laplace-based Monte Carlo Methods for Bayesian Inverse Problems
Authors:
Claudia Schillings,
Björn Sprungk,
Philipp Wacker
Abstract:
The Bayesian approach to inverse problems provides a rigorous framework for the incorporation and quantification of uncertainties in measurements, parameters and models. We are interested in designing numerical methods which are robust w.r.t. the size of the observational noise, i.e., methods which behave well in case of concentrated posterior measures. The concentration of the posterior is a high…
▽ More
The Bayesian approach to inverse problems provides a rigorous framework for the incorporation and quantification of uncertainties in measurements, parameters and models. We are interested in designing numerical methods which are robust w.r.t. the size of the observational noise, i.e., methods which behave well in case of concentrated posterior measures. The concentration of the posterior is a highly desirable situation in practice, since it relates to informative or large data. However, it can pose a computational challenge for numerical methods based on the prior or reference measure. We propose to employ the Laplace approximation of the posterior as the base measure for numerical integration in this context. The Laplace approximation is a Gaussian measure centered at the maximum a-posteriori estimate and with covariance matrix depending on the logposterior density. We discuss convergence results of the Laplace approximation in terms of the Hellinger distance and analyze the efficiency of Monte Carlo methods based on it. In particular, we show that Laplace-based importance sampling and Laplace-based quasi-Monte-Carlo methods are robust w.r.t. the concentration of the posterior for large classes of posterior distributions and integrands whereas prior-based importance sampling and plain quasi-Monte Carlo are not. Numerical experiments are presented to illustrate the theoretical findings.
△ Less
Submitted 26 June, 2020; v1 submitted 13 January, 2019;
originally announced January 2019.
-
Well Posedness and Convergence Analysis of the Ensemble Kalman Inversion
Authors:
Dirk Blömker,
Claudia Schillings,
Philipp Wacker,
Simon Weissmann
Abstract:
The ensemble Kalman inversion is widely used in practice to estimate unknown parameters from noisy measurement data. Its low computational costs, straightforward implementation, and non-intrusive nature makes the method appealing in various areas of application. We present a complete analysis of the ensemble Kalman inversion with perturbed observations for a fixed ensemble size when applied to lin…
▽ More
The ensemble Kalman inversion is widely used in practice to estimate unknown parameters from noisy measurement data. Its low computational costs, straightforward implementation, and non-intrusive nature makes the method appealing in various areas of application. We present a complete analysis of the ensemble Kalman inversion with perturbed observations for a fixed ensemble size when applied to linear inverse problems. The well-posedness and convergence results are based on the continuous time scaling limits of the method. The resulting coupled system of stochastic differential equations allows to derive estimates on the long-time behaviour and provides insights into the convergence properties of the ensemble Kalman inversion. We view the method as a derivative free optimization method for the least-squares misfit functional, which opens up the perspective to use the method in various areas of applications such as imaging, groundwater flow problems, biological problems as well as in the context of the training of neural networks.
△ Less
Submitted 26 February, 2019; v1 submitted 19 October, 2018;
originally announced October 2018.
-
A strongly convergent numerical scheme from Ensemble Kalman inversion
Authors:
Dirk Blömker,
Claudia Schillings,
Philipp Wacker
Abstract:
The Ensemble Kalman methodology in an inverse problems setting can be viewed as an iterative scheme, which is a weakly tamed discretization scheme for a certain stochastic differential equation (SDE). Assuming a suitable approximation result, dynamical properties of the SDE can be rigorously pulled back via the discrete scheme to the original Ensemble Kalman inversion.
The results of this paper…
▽ More
The Ensemble Kalman methodology in an inverse problems setting can be viewed as an iterative scheme, which is a weakly tamed discretization scheme for a certain stochastic differential equation (SDE). Assuming a suitable approximation result, dynamical properties of the SDE can be rigorously pulled back via the discrete scheme to the original Ensemble Kalman inversion.
The results of this paper make a step towards closing the gap of the missing approximation result by proving a strong convergence result in a simplified model of a scalar stochastic differential equation. We focus here on a toy model with similar properties than the one arising in the context of Ensemble Kalman filter. The proposed model can be interpreted as a single particle filter for a linear map and thus forms the basis for further analysis. The difficulty in the analysis arises from the formally derived limiting SDE with non-globally Lipschitz continuous nonlinearities both in the drift and in the diffusion. Here the standard Euler-Maruyama scheme might fail to provide a strongly convergent numerical scheme and taming is necessary. In contrast to the strong taming usually used, the method presented here provides a weaker form of taming.
We present a strong convergence analysis by first proving convergence on a domain of high probability by using a cut-off or localisation, which then leads, combined with bounds on moments for both the SDE and the numerical scheme, by a bootstrap** argument to strong convergence.
△ Less
Submitted 18 June, 2018; v1 submitted 20 March, 2017;
originally announced March 2017.
-
Convergence Analysis of Ensemble Kalman Inversion: The Linear, Noisy Case
Authors:
Claudia Schillings,
Andrew Stuart
Abstract:
We present an analysis of ensemble Kalman inversion, based on the continuous time limit of the algorithm. The analysis of the dynamical behaviour of the ensemble allows us to establish well-posedness and convergence results for a fixed ensemble size. We will build on the results presented in [26] and generalise them to the case of noisy observational data, in particular the influence of the noise…
▽ More
We present an analysis of ensemble Kalman inversion, based on the continuous time limit of the algorithm. The analysis of the dynamical behaviour of the ensemble allows us to establish well-posedness and convergence results for a fixed ensemble size. We will build on the results presented in [26] and generalise them to the case of noisy observational data, in particular the influence of the noise on the convergence will be investigated, both theoretically and numerically. We focus on linear inverse problems where a very complete theoretical analysis is possible.
△ Less
Submitted 8 August, 2017; v1 submitted 25 February, 2017;
originally announced February 2017.
-
Analysis of the ensemble Kalman filter for inverse problems
Authors:
Claudia Schillings,
Andrew M. Stuart
Abstract:
The ensemble Kalman filter (EnKF) is a widely used methodology for state estimation in partial, noisily observed dynamical systems, and for parameter estimation in inverse problems. Despite its widespread use in the geophysical sciences, and its gradual adoption in many other areas of application, analysis of the method is in its infancy. Furthermore, much of the existing analysis deals with the l…
▽ More
The ensemble Kalman filter (EnKF) is a widely used methodology for state estimation in partial, noisily observed dynamical systems, and for parameter estimation in inverse problems. Despite its widespread use in the geophysical sciences, and its gradual adoption in many other areas of application, analysis of the method is in its infancy. Furthermore, much of the existing analysis deals with the large ensemble limit, far from the regime in which the method is typically used. The goal of this paper is to analyze the method when applied to inverse problems with fixed ensemble size. A continuous-time limit is derived and the long-time behavior of the resulting dynamical system is studied. Most of the rigorous analysis is confined to the linear forward problem, where we demonstrate that the continuous time limit of the EnKF corresponds to a set of gradient flows for the data misfit in each ensemble member, coupled through a common pre-conditioner which is the empirical covariance matrix of the ensemble. Numerical results demonstrate that the conclusions of the analysis extend beyond the linear inverse problem setting. Numerical experiments are also given which demonstrate the benefits of various extensions of the basic methodology.
△ Less
Submitted 20 September, 2016; v1 submitted 5 February, 2016;
originally announced February 2016.
-
Quantification of airfoil geometry-induced aerodynamic uncertainties - comparison of approaches
Authors:
Dishi Liu,
Alexander Litvinenko,
Claudia Schillings,
Volker Schulz
Abstract:
Uncertainty quantification in aerodynamic simulations calls for efficient numerical methods since it is computationally expensive, especially for the uncertainties caused by random geometry variations which involve a large number of variables. This paper compares five methods, including quasi-Monte Carlo quadrature, polynomial chaos with coefficients determined by sparse quadrature and gradient-en…
▽ More
Uncertainty quantification in aerodynamic simulations calls for efficient numerical methods since it is computationally expensive, especially for the uncertainties caused by random geometry variations which involve a large number of variables. This paper compares five methods, including quasi-Monte Carlo quadrature, polynomial chaos with coefficients determined by sparse quadrature and gradient-enhanced version of Kriging, radial basis functions and point collocation polynomial chaos, in their efficiency in estimating statistics of aerodynamic performance upon random perturbation to the airfoil geometry which is parameterized by 9 independent Gaussian variables. The results show that gradient-enhanced surrogate methods achieve better accuracy than direct integration methods with the same computational cost.
△ Less
Submitted 28 December, 2016; v1 submitted 21 May, 2015;
originally announced May 2015.
