Showing 1–14 of 14 results for author: Fiedler, C

On Safety in Safe Bayesian Optimization

Authors: Christian Fiedler, Johanna Menn, Lukas Kreisköther, Sebastian Trimpe

Abstract: Optimizing an unknown function under safety constraints is a central task in robotics, biomedical engineering, and many other disciplines, and increasingly safe Bayesian Optimization (BO) is used for this. Due to the safety critical nature of these applications, it is of utmost importance that theoretical safety guarantees for these algorithms translate into the real world. In this work, we invest… ▽ More Optimizing an unknown function under safety constraints is a central task in robotics, biomedical engineering, and many other disciplines, and increasingly safe Bayesian Optimization (BO) is used for this. Due to the safety critical nature of these applications, it is of utmost importance that theoretical safety guarantees for these algorithms translate into the real world. In this work, we investigate three safety-related issues of the popular class of SafeOpt-type algorithms. First, these algorithms critically rely on frequentist uncertainty bounds for Gaussian Process (GP) regression, but concrete implementations typically utilize heuristics that invalidate all safety guarantees. We provide a detailed analysis of this problem and introduce Real-\b{eta}-SafeOpt, a variant of the SafeOpt algorithm that leverages recent GP bounds and thus retains all theoretical guarantees. Second, we identify assuming an upper bound on the reproducing kernel Hilbert space (RKHS) norm of the target function, a key technical assumption in SafeOpt-like algorithms, as a central obstacle to real-world usage. To overcome this challenge, we introduce the Lipschitz-only Safe Bayesian Optimization (LoSBO) algorithm, which guarantees safety without an assumption on the RKHS bound, and empirically show that this algorithm is not only safe, but also exhibits superior performance compared to the state-of-the-art on several function classes. Third, SafeOpt and derived algorithms rely on a discrete search space, making them difficult to apply to higher-dimensional problems. To widen the applicability of these algorithms, we introduce Lipschitz-only GP-UCB (LoS-GP-UCB), a variant of LoSBO applicable to moderately high-dimensional problems, while retaining safety. △ Less

Submitted 19 March, 2024; originally announced March 2024.

Automatic nonlinear MPC approximation with closed-loop guarantees

Authors: Abdullah Tokmak, Christian Fiedler, Melanie N. Zeilinger, Sebastian Trimpe, Johannes Köhler

Abstract: Safety guarantees are vital in many control applications, such as robotics. Model predictive control (MPC) provides a constructive framework for controlling safety-critical systems, but is limited by its computational complexity. We address this problem by presenting a novel algorithm that automatically computes an explicit approximation to nonlinear MPC schemes while retaining closed-loop guarant… ▽ More Safety guarantees are vital in many control applications, such as robotics. Model predictive control (MPC) provides a constructive framework for controlling safety-critical systems, but is limited by its computational complexity. We address this problem by presenting a novel algorithm that automatically computes an explicit approximation to nonlinear MPC schemes while retaining closed-loop guarantees. Specifically, the problem can be reduced to a function approximation problem, which we then tackle by proposing ALKIA-X, the Adaptive and Localized Kernel Interpolation Algorithm with eXtrapolated reproducing kernel Hilbert space norm. ALKIA-X is a non-iterative algorithm that ensures numerically well-conditioned computations, a fast-to-evaluate approximating function, and the guaranteed satisfaction of any desired bound on the approximation error. Hence, ALKIA-X automatically computes an explicit function that approximates the MPC, yielding a controller suitable for safety-critical systems and high sampling rates. We apply ALKIA-X to approximate two nonlinear MPC schemes, demonstrating reduced computational demand and applicability to realistic problems. △ Less

Submitted 11 April, 2024; v1 submitted 15 December, 2023; originally announced December 2023.

Comments: Submitted to IEEE Transactions on Automatic Control. Compared to the previously uploaded version, this version contains an additional numerical example

Mean field limits for discrete-time dynamical systems via kernel mean embeddings

Authors: Christian Fiedler, Michael Herty, Sebastian Trimpe

Abstract: Mean field limits are an important tool in the context of large-scale dynamical systems, in particular, when studying multiagent and interacting particle systems. While the continuous-time theory is well-developed, few works have considered mean field limits for deterministic discrete-time systems, which are relevant for the analysis and control of large-scale discrete-time multiagent system. We p… ▽ More Mean field limits are an important tool in the context of large-scale dynamical systems, in particular, when studying multiagent and interacting particle systems. While the continuous-time theory is well-developed, few works have considered mean field limits for deterministic discrete-time systems, which are relevant for the analysis and control of large-scale discrete-time multiagent system. We prove existence results for the mean field limit of very general discrete-time control systems, for which we utilize kernel mean embeddings. These results are then applied in a typical optimal control setup, where we establish the mean field limit of the relaxed dynamic programming principle. Our results can serve as a rigorous foundation for many applications of mean field approaches for discrete-time dynamical systems. △ Less

Submitted 11 December, 2023; originally announced December 2023.

Comments: Accepted at IEEE Control Systems Letters

MSC Class: 82C22 (Primary); 82B40; 93C55 (Secondary)

Lipschitz and Hölder Continuity in Reproducing Kernel Hilbert Spaces

Authors: Christian Fiedler

Abstract: Reproducing kernel Hilbert spaces (RKHSs) are very important function spaces, playing an important role in machine learning, statistics, numerical analysis and pure mathematics. Since Lipschitz and Hölder continuity are important regularity properties, with many applications in interpolation, approximation and optimization problems, in this work we investigate these continuity notion in RKHSs. We… ▽ More Reproducing kernel Hilbert spaces (RKHSs) are very important function spaces, playing an important role in machine learning, statistics, numerical analysis and pure mathematics. Since Lipschitz and Hölder continuity are important regularity properties, with many applications in interpolation, approximation and optimization problems, in this work we investigate these continuity notion in RKHSs. We provide several sufficient conditions as well as an in depth investigation of reproducing kernels inducing prescribed Lipschitz or Hölder continuity. Apart from new results, we also collect related known results from the literature, making the present work also a convenient reference on this topic. △ Less

Submitted 27 October, 2023; originally announced October 2023.

Comments: Preprint, under review

MSC Class: 46E22 (Primary); 51F30; 47B34; 47G10 (Secondary)

On kernel-based statistical learning in the mean field limit

Authors: Christian Fiedler, Michael Herty, Sebastian Trimpe

Abstract: In many applications of machine learning, a large number of variables are considered. Motivated by machine learning of interacting particle systems, we consider the situation when the number of input variables goes to infinity. First, we continue the recent investigation of the mean field limit of kernels and their reproducing kernel Hilbert spaces, completing the existing theory. Next, we provide… ▽ More In many applications of machine learning, a large number of variables are considered. Motivated by machine learning of interacting particle systems, we consider the situation when the number of input variables goes to infinity. First, we continue the recent investigation of the mean field limit of kernels and their reproducing kernel Hilbert spaces, completing the existing theory. Next, we provide results relevant for approximation with such kernels in the mean field limit, including a representer theorem. Finally, we use these kernels in the context of statistical learning in the mean field limit, focusing on Support Vector Machines. In particular, we show mean field convergence of empirical and infinite-sample solutions as well as the convergence of the corresponding risks. On the one hand, our results establish rigorous mean field limits in the context of kernel methods, providing new theoretical tools and insights for large-scale problems. On the other hand, our setting corresponds to a new form of limit of learning problems, which seems to have not been investigated yet in the statistical learning theory literature. △ Less

Submitted 27 October, 2023; originally announced October 2023.

Comments: NeurIPS 2023

Um banco de dados de empregos formais georreferenciados em cidades brasileiras

Authors: Andre Borgato Morelli, André de Carvalho Fiedler, André Luiz Cunha

Abstract: Currently, transport planning has changed its paradigm from projects oriented to guarantee service levels to projects oriented to guarantee accessibility to opportunities. In this context, a number of studies and tools aimed at calculating accessibility are being made available, however these tools depend on job location data that are not always easily accessible. Thus, this work proposes the crea… ▽ More Currently, transport planning has changed its paradigm from projects oriented to guarantee service levels to projects oriented to guarantee accessibility to opportunities. In this context, a number of studies and tools aimed at calculating accessibility are being made available, however these tools depend on job location data that are not always easily accessible. Thus, this work proposes the creation of a database with the locations of formal jobs in Brazilian cities. The method uses the RAIS jobs database and the CNEFE street faces database to infer the location of jobs in urban regions from the zip code and the number of non-residential addresses on street faces. As a result, jobs can be located more accurately in large and medium-sized cities and approximately in single zip code cities. Finally, the databases are made available openly so that researchers and planning professionals can easily apply accessibility analyzes throughout the national territory. △ Less

Submitted 16 March, 2023; originally announced March 2023.

Comments: 9 pages, in Portuguese language, 3 figures, 1 table, Data presentation paper

MSC Class: 68-00

Reproducing kernel Hilbert spaces in the mean field limit

Authors: Christian Fiedler, Michael Herty, Michael Rom, Chiara Segala, Sebastian Trimpe

Abstract: Kernel methods, being supported by a well-developed theory and coming with efficient algorithms, are among the most popular and successful machine learning techniques. From a mathematical point of view, these methods rest on the concept of kernels and function spaces generated by kernels, so called reproducing kernel Hilbert spaces. Motivated by recent developments of learning approaches in the co… ▽ More Kernel methods, being supported by a well-developed theory and coming with efficient algorithms, are among the most popular and successful machine learning techniques. From a mathematical point of view, these methods rest on the concept of kernels and function spaces generated by kernels, so called reproducing kernel Hilbert spaces. Motivated by recent developments of learning approaches in the context of interacting particle systems, we investigate kernel methods acting on data with many measurement variables. We show the rigorous mean field limit of kernels and provide a detailed analysis of the limiting reproducing kernel Hilbert space. Furthermore, several examples of kernels, that allow a rigorous mean field limit, are presented. △ Less

Submitted 17 March, 2023; v1 submitted 28 February, 2023; originally announced February 2023.

Comments: Updated author email addresses

MSC Class: 46E22; 82B40; 74A25; 82C40

Revisiting the derivation of stage costs in infinite horizon discrete-time optimal control

Authors: Christian Fiedler, Sebastian Trimpe

Abstract: In many applications of optimal control, the stage cost is not fixed, but rather a design choice with considerable impact on the control performance. In infinite horizon optimal control, the choice of stage cost is often restricted by the requirement of uniform cost controllability, which is nontrivial to satisfy. Here we revisit a previously proposed constructive technique for stage cost design.… ▽ More In many applications of optimal control, the stage cost is not fixed, but rather a design choice with considerable impact on the control performance. In infinite horizon optimal control, the choice of stage cost is often restricted by the requirement of uniform cost controllability, which is nontrivial to satisfy. Here we revisit a previously proposed constructive technique for stage cost design. We generalize its setting, weaken the required assumptions and add additional flexibility. Furthermore, we show that the required assumptions essentially cannot be weakened anymore. By providing improved design options for stage costs, this work contributes to expanding the applicability of optimization-based control methodologies, in particular, model predictive control. △ Less

Submitted 12 July, 2022; originally announced July 2022.

Learning-enhanced robust controller synthesis with rigorous statistical and control-theoretic guarantees

Authors: Christian Fiedler, Carsten W. Scherer, Sebastian Trimpe

Abstract: The combination of machine learning with control offers many opportunities, in particular for robust control. However, due to strong safety and reliability requirements in many real-world applications, providing rigorous statistical and control-theoretic guarantees is of utmost importance, yet difficult to achieve for learning-based control schemes. We present a general framework for learning-enha… ▽ More The combination of machine learning with control offers many opportunities, in particular for robust control. However, due to strong safety and reliability requirements in many real-world applications, providing rigorous statistical and control-theoretic guarantees is of utmost importance, yet difficult to achieve for learning-based control schemes. We present a general framework for learning-enhanced robust control that allows for systematic integration of prior engineering knowledge, is fully compatible with modern robust control and still comes with rigorous and practically meaningful guarantees. Building on the established Linear Fractional Representation and Integral Quadratic Constraints framework, we integrate Gaussian Process Regression as a learning component and state-of-the-art robust controller synthesis. In a concrete robust control example, our approach is demonstrated to yield improved performance with more data, while guarantees are maintained throughout. △ Less

Submitted 7 May, 2021; originally announced May 2021.

Practical and Rigorous Uncertainty Bounds for Gaussian Process Regression

Authors: Christian Fiedler, Carsten W. Scherer, Sebastian Trimpe

Abstract: Gaussian Process Regression is a popular nonparametric regression method based on Bayesian principles that provides uncertainty estimates for its predictions. However, these estimates are of a Bayesian nature, whereas for some important applications, like learning-based control with safety guarantees, frequentist uncertainty bounds are required. Although such rigorous bounds are available for Gaus… ▽ More Gaussian Process Regression is a popular nonparametric regression method based on Bayesian principles that provides uncertainty estimates for its predictions. However, these estimates are of a Bayesian nature, whereas for some important applications, like learning-based control with safety guarantees, frequentist uncertainty bounds are required. Although such rigorous bounds are available for Gaussian Processes, they are too conservative to be useful in applications. This often leads practitioners to replacing these bounds by heuristics, thus breaking all theoretical guarantees. To address this problem, we introduce new uncertainty bounds that are rigorous, yet practically useful at the same time. In particular, the bounds can be explicitly evaluated and are much less conservative than state of the art results. Furthermore, we show that certain model misspecifications lead to only graceful degradation. We demonstrate these advantages and the usefulness of our results for learning-based control with numerical examples. △ Less

Submitted 8 August, 2023; v1 submitted 6 May, 2021; originally announced May 2021.

Comments: Contains supplementary material and corrections to the original version

Analogue Hawking temperature of a laser-driven plasma

Authors: C. Fiedler, D. A. Burton

Abstract: We present a method for exploring analogue Hawking radiation using a laser pulse propagating through an underdense plasma. The propagating fields in the Hawking effect are local perturbations of the plasma density and laser amplitude. We derive the dependence of the resulting Hawking temperature on the dimensionless amplitude of the laser and the behaviour of the spot area of the laser at the anal… ▽ More We present a method for exploring analogue Hawking radiation using a laser pulse propagating through an underdense plasma. The propagating fields in the Hawking effect are local perturbations of the plasma density and laser amplitude. We derive the dependence of the resulting Hawking temperature on the dimensionless amplitude of the laser and the behaviour of the spot area of the laser at the analogue event horizon. We demonstrate one possible way of obtaining the analogue Hawking temperature in terms of the plasma wavelength, and our analysis shows that for a high intensity near-IR laser the analogue Hawking temperature is less than approximately 25K for a reasonable choice of parameters. △ Less

Submitted 22 April, 2021; v1 submitted 4 February, 2021; originally announced February 2021.

Comments: 12 pages. Green OA version of an article accepted for publication by Physics Letters A

Journal ref: Phys. Lett. A 403 (2021) 127380

Stable Recovery of Entangled Weights: Towards Robust Identification of Deep Neural Networks from Minimal Samples

Authors: Christian Fiedler, Massimo Fornasier, Timo Klock, Michael Rauchensteiner

Abstract: In this paper we approach the problem of unique and stable identifiability of generic deep artificial neural networks with pyramidal shape and smooth activation functions from a finite number of input-output samples. More specifically we introduce the so-called entangled weights, which compose weights of successive layers intertwined with suitable diagonal and invertible matrices depending on the… ▽ More In this paper we approach the problem of unique and stable identifiability of generic deep artificial neural networks with pyramidal shape and smooth activation functions from a finite number of input-output samples. More specifically we introduce the so-called entangled weights, which compose weights of successive layers intertwined with suitable diagonal and invertible matrices depending on the activation functions and their shifts. We prove that entangled weights are completely and stably approximated by an efficient and robust algorithm as soon as $\mathcal O(D^2 \times m)$ nonadaptive input-output samples of the network are collected, where $D$ is the input dimension and $m$ is the number of neurons of the network. Moreover, we empirically observe that the approach applies to networks with up to $\mathcal O(D \times m_L)$ neurons, where $m_L$ is the number of output neurons at layer $L$. Provided knowledge of layer assignments of entangled weights and of remaining scaling and shift parameters, which may be further heuristically obtained by least squares, the entangled weights identify the network completely and uniquely. To highlight the relevance of the theoretical result of stable recovery of entangled weights, we present numerical experiments, which demonstrate that multilayered networks with generic weights can be robustly identified and therefore uniformly approximated by the presented algorithmic pipeline. In contrast backpropagation cannot generalize stably very well in this setting, being always limited by relatively large uniform error. In terms of practical impact, our study shows that we can relate input-output information uniquely and stably to network parameters, providing a form of explainability. Moreover, our method paves the way for compression of overparametrized networks and for the training of minimal complexity networks. △ Less

Submitted 18 January, 2021; originally announced January 2021.

MSC Class: 65D15; 68T07; 90C26

A Kernel Two-sample Test for Dynamical Systems

Authors: Friedrich Solowjow, Dominik Baumann, Christian Fiedler, Andreas Jocham, Thomas Seel, Sebastian Trimpe

Abstract: Evaluating whether data streams are drawn from the same distribution is at the heart of various machine learning problems. This is particularly relevant for data generated by dynamical systems since such systems are essential for many real-world processes in biomedical, economic, or engineering systems. While kernel two-sample tests are powerful for comparing independent and identically distribute… ▽ More Evaluating whether data streams are drawn from the same distribution is at the heart of various machine learning problems. This is particularly relevant for data generated by dynamical systems since such systems are essential for many real-world processes in biomedical, economic, or engineering systems. While kernel two-sample tests are powerful for comparing independent and identically distributed random variables, no established method exists for comparing dynamical systems. The main problem is the inherently violated independence assumption. We propose a two-sample test for dynamical systems by addressing three core challenges: we (i) introduce a novel notion of mixing that captures autocorrelations in a relevant metric, (ii) propose an efficient way to estimate the speed of mixing relying purely on data, and (iii) integrate these into established kernel two-sample tests. The result is a data-driven method that is straightforward to use in practice and comes with sound theoretical guarantees. In an example application to anomaly detection from human walking data, we show that the test is readily applicable without any human expert knowledge and feature engineering. △ Less

Submitted 4 September, 2022; v1 submitted 23 April, 2020; originally announced April 2020.

Quantum backreaction in laser-driven plasma

Authors: A. Conroy, C. Fiedler, A. Noble, D. A. Burton

Abstract: We present a new approach for investigating quantum effects in laser-driven plasma. Unlike the modelling strategies underpinning particle-in-cell codes that include the effects of quantum electrodynamics, our new field theory incorporates multi-particle effects from the outset. Our approach is based on the path-integral quantisation of a classical bi-scalar field theory describing the behaviour of… ▽ More We present a new approach for investigating quantum effects in laser-driven plasma. Unlike the modelling strategies underpinning particle-in-cell codes that include the effects of quantum electrodynamics, our new field theory incorporates multi-particle effects from the outset. Our approach is based on the path-integral quantisation of a classical bi-scalar field theory describing the behaviour of a laser pulse propagating through an underdense plasma. Results established in the context of quantum field theory on curved spacetime are used to derive a non-linear, non-local, effective field theory that describes the evolution of the laser-driven plasma due to quantum fluctuations. As the first application of our new theory, we explore the behaviour of perturbations to fields describing a uniform, monochromatic, laser beam propagating through a uniform plasma. Our results suggest that quantum fluctuations could play a significant role in the evolution of an underdense plasma driven by an x-ray laser pulse. △ Less

Submitted 14 May, 2020; v1 submitted 23 June, 2019; originally announced June 2019.

Comments: 18 pages. Several corrections have been made. The first and last sections have been substantially rewritten. An appendix has been included