Showing 1–16 of 16 results for author: Muehlebach, M

Gray-Box Nonlinear Feedback Optimization

Authors: Zhiyu He, Saverio Bolognani, Michael Muehlebach, Florian Dörfler

Abstract: Feedback optimization enables autonomous optimality seeking of a dynamical system through its closed-loop interconnection with iterative optimization algorithms. Among various iteration structures, model-based approaches require the input-output sensitivity of the system to construct gradients, whereas model-free approaches bypass this need by estimating gradients from real-time evaluations of the… ▽ More Feedback optimization enables autonomous optimality seeking of a dynamical system through its closed-loop interconnection with iterative optimization algorithms. Among various iteration structures, model-based approaches require the input-output sensitivity of the system to construct gradients, whereas model-free approaches bypass this need by estimating gradients from real-time evaluations of the objective. These approaches own complementary benefits in sample efficiency and accuracy against model mismatch, i.e., errors of sensitivities. To achieve the best of both worlds, we propose gray-box feedback optimization controllers, featuring systematic incorporation of approximate sensitivities into model-free updates via adaptive convex combination. We quantify conditions on the accuracy of the sensitivities that render the gray-box approach preferable. We elucidate how the closed-loop performance is determined by the number of iterations, the problem dimension, and the cumulative effect of inaccurate sensitivities. The proposed controller contributes to a balanced closed-loop behavior, which retains provable sample efficiency and optimality guarantees for nonconvex problems. We further develop a running gray-box controller to handle constrained time-varying problems with changing objectives and steady-state maps. △ Less

Submitted 5 April, 2024; originally announced April 2024.

Balancing a 3D Inverted Pendulum using Remote Magnetic Manipulation

Authors: Jasan Zughaibi, Bradley J. Nelson, Michael Muehlebach

Abstract: Remote magnetic manipulation offers wireless control over magnetic objects, which has important medical applications, such as targeted drug delivery and minimally invasive surgeries. Magnetic manipulation systems are categorized into systems using permanent magnets and systems based on electromagnets. Electro-Magnetic Navigation Systems (eMNSs) are believed to have a superior actuation bandwidth,… ▽ More Remote magnetic manipulation offers wireless control over magnetic objects, which has important medical applications, such as targeted drug delivery and minimally invasive surgeries. Magnetic manipulation systems are categorized into systems using permanent magnets and systems based on electromagnets. Electro-Magnetic Navigation Systems (eMNSs) are believed to have a superior actuation bandwidth, facilitating trajectory tracking and disturbance rejection. This greatly expands the range of potential medical applications and includes even dynamic environments as encountered in cardiovascular interventions. In order to highlight the dynamic capabilities of eMNSs, we successfully stabilize a (non-magnetic) inverted pendulum on the tip of a magnetically driven arm. Our method employs a model-based design approach, where we capture the dynamics that describe the interaction of the pendulum system and the magnetic field through Lagrangian mechanics. Using system identification we estimate the system parameters, the actuation bandwidth, and characterize the system's nonlinearity. We design a state-feedback controller to stabilize the inherently unstable dynamics, and compensate for errors arising from the calibration of the magnetic field and the angle measurement system. Additionally, we integrate an iterative learning control scheme that allows us to accurately track non-equilibrium trajectories while concurrently maintaining stability of the inverted pendulum. To our knowledge, this is the first effort to stabilize a 3D inverted pendulum through remote magnetic manipulation. △ Less

Submitted 8 February, 2024; originally announced February 2024.

Towards a Systems Theory of Algorithms

Authors: Florian Dörfler, Zhiyu He, Giuseppe Belgioioso, Saverio Bolognani, John Lygeros, Michael Muehlebach

Abstract: Traditionally, numerical algorithms are seen as isolated pieces of code confined to an {\em in silico} existence. However, this perspective is not appropriate for many modern computational approaches in control, learning, or optimization, wherein {\em in vivo} algorithms interact with their environment. Examples of such {\em open algorithms} include various real-time optimization-based control str… ▽ More Traditionally, numerical algorithms are seen as isolated pieces of code confined to an {\em in silico} existence. However, this perspective is not appropriate for many modern computational approaches in control, learning, or optimization, wherein {\em in vivo} algorithms interact with their environment. Examples of such {\em open algorithms} include various real-time optimization-based control strategies, reinforcement learning, decision-making architectures, online optimization, and many more. Further, even {\em closed} algorithms in learning or optimization are increasingly abstracted in block diagrams with interacting dynamic modules and pipelines. In this opinion paper, we state our vision on a to-be-cultivated {\em systems theory of algorithms} and argue in favor of viewing algorithms as open dynamical systems interacting with other algorithms, physical systems, humans, or databases. Remarkably, the manifold tools developed under the umbrella of systems theory are well suited for addressing a range of challenges in the algorithmic domain. We survey various instances where the principles of algorithmic systems theory are being developed and outline pertinent modeling, analysis, and design challenges. △ Less

Submitted 30 April, 2024; v1 submitted 25 January, 2024; originally announced January 2024.

Data-Efficient Online Learning of Ball Placement in Robot Table Tennis

Authors: Philip Tobuschat, Hao Ma, Dieter Büchler, Bernhard Schölkopf, Michael Muehlebach

Abstract: We present an implementation of an online optimization algorithm for hitting a predefined target when returning **-pong balls with a table tennis robot. The online algorithm optimizes over so-called interception policies, which define the manner in which the robot arm intercepts the ball. In our case, these are composed of the state of the robot arm (position and velocity) at interception time.… ▽ More We present an implementation of an online optimization algorithm for hitting a predefined target when returning **-pong balls with a table tennis robot. The online algorithm optimizes over so-called interception policies, which define the manner in which the robot arm intercepts the ball. In our case, these are composed of the state of the robot arm (position and velocity) at interception time. Gradient information is provided to the optimization algorithm via the map** from the interception policy to the landing point of the ball on the table, which is approximated with a black-box and a grey-box approach. Our algorithm is applied to a robotic arm with four degrees of freedom that is driven by pneumatic artificial muscles. As a result, the robot arm is able to return the ball onto any predefined target on the table after about 2-5 iterations. We highlight the robustness of our approach by showing rapid convergence with both the black-box and the grey-box gradients. In addition, the small number of iterations required to reach close proximity to the target also underlines the sample efficiency. A demonstration video can be found here: https://youtu.be/VC3KJoCss0k. △ Less

Submitted 28 August, 2023; originally announced August 2023.

Comments: 7 pages, 6 figures, to be published in proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 2023

Black-Box vs. Gray-Box: A Case Study on Learning Table Tennis Ball Trajectory Prediction with Spin and Impacts

Authors: Jan Achterhold, Philip Tobuschat, Hao Ma, Dieter Buechler, Michael Muehlebach, Joerg Stueckler

Abstract: In this paper, we present a method for table tennis ball trajectory filtering and prediction. Our gray-box approach builds on a physical model. At the same time, we use data to learn parameters of the dynamics model, of an extended Kalman filter, and of a neural model that infers the ball's initial condition. We demonstrate superior prediction performance of our approach over two black-box approac… ▽ More In this paper, we present a method for table tennis ball trajectory filtering and prediction. Our gray-box approach builds on a physical model. At the same time, we use data to learn parameters of the dynamics model, of an extended Kalman filter, and of a neural model that infers the ball's initial condition. We demonstrate superior prediction performance of our approach over two black-box approaches, which are not supplied with physical prior knowledge. We demonstrate that initializing the spin from parameters of the ball launcher using a neural network drastically improves long-time prediction performance over estimating the spin purely from measured ball positions. An accurate prediction of the ball trajectory is crucial for successful returns. We therefore evaluate the return performance with a pneumatic artificial muscular robot and achieve a return rate of 29/30 (97.7%). △ Less

Submitted 12 June, 2023; v1 submitted 24 May, 2023; originally announced May 2023.

Comments: Accepted for publication at the 5th Annual Conference on Learning for Dynamics and Control (L4DC) 2023 (camera-ready). With supplementary material

Adaptive Decision-Making with Constraints and Dependent Losses: Performance Guarantees and Applications to Online and Nonlinear Identification

Authors: Michael Muehlebach

Abstract: We consider adaptive decision-making problems where an agent optimizes a cumulative performance objective by repeatedly choosing among a finite set of options. Compared to the classical prediction-with-expert-advice set-up, we consider situations where losses are constrained and derive algorithms that exploit the additional structure in optimal and computationally efficient ways. Our algorithm and… ▽ More We consider adaptive decision-making problems where an agent optimizes a cumulative performance objective by repeatedly choosing among a finite set of options. Compared to the classical prediction-with-expert-advice set-up, we consider situations where losses are constrained and derive algorithms that exploit the additional structure in optimal and computationally efficient ways. Our algorithm and our analysis is instance dependent, that is, suboptimal choices of the environment are exploited and reflected in our regret bounds. The constraints handle general dependencies between losses (even across time), and are flexible enough to also account for a loss budget, which the environment is not allowed to exceed. The performance of the resulting algorithms is highlighted in two numerical examples, which include a nonlinear and online system identification task. △ Less

Submitted 6 April, 2023; originally announced April 2023.

Comments: 8 pages

Accelerated First-Order Optimization under Nonlinear Constraints

Authors: Michael Muehlebach, Michael I. Jordan

Abstract: We exploit analogies between first-order algorithms for constrained optimization and non-smooth dynamical systems to design a new class of accelerated first-order algorithms for constrained optimization. Unlike Frank-Wolfe or projected gradients, these algorithms avoid optimization over the entire feasible set at each iteration. We prove convergence to stationary points even in a nonconvex setting… ▽ More We exploit analogies between first-order algorithms for constrained optimization and non-smooth dynamical systems to design a new class of accelerated first-order algorithms for constrained optimization. Unlike Frank-Wolfe or projected gradients, these algorithms avoid optimization over the entire feasible set at each iteration. We prove convergence to stationary points even in a nonconvex setting and we derive accelerated rates for the convex setting both in continuous time, as well as in discrete time. An important property of these algorithms is that constraints are expressed in terms of velocities instead of positions, which naturally leads to sparse, local and convex approximations of the feasible set (even if the feasible set is nonconvex). Thus, the complexity tends to grow mildly in the number of decision variables and in the number of constraints, which makes the algorithms suitable for machine learning applications. We apply our algorithms to a compressed sensing and a sparse regression problem, showing that we can treat nonconvex $\ell^p$ constraints ($p<1$) efficiently, while recovering state-of-the-art performance for $p=1$. △ Less

Submitted 2 January, 2024; v1 submitted 1 February, 2023; originally announced February 2023.

Comments: 44 pages, 6 figures

Robust Recurrent Neural Network to Identify Ship Motion in Open Water with Performance Guarantees -- Technical Report

Authors: Daniel Frank, Decky Aspandi Latif, Michael Muehlebach, Benjamin Unger, Steffen Staab

Abstract: Recurrent neural networks are capable of learning the dynamics of an unknown nonlinear system purely from input-output measurements. However, the resulting models do not provide any stability guarantees on the input-output map**. In this work, we represent a recurrent neural network as a linear time-invariant system with nonlinear disturbances. By introducing constraints on the parameters, we ca… ▽ More Recurrent neural networks are capable of learning the dynamics of an unknown nonlinear system purely from input-output measurements. However, the resulting models do not provide any stability guarantees on the input-output map**. In this work, we represent a recurrent neural network as a linear time-invariant system with nonlinear disturbances. By introducing constraints on the parameters, we can guarantee finite gain stability and incremental finite gain stability. We apply this identification method to learn the motion of a four-degrees-of-freedom ship that is moving in open water and compare it against other purely learning-based approaches with unconstrained parameters. Our analysis shows that the constrained recurrent neural network has a lower prediction accuracy on the test set, but it achieves comparable results on an out-of-distribution set and respects stability conditions. △ Less

Submitted 16 December, 2022; v1 submitted 12 December, 2022; originally announced December 2022.

On Constraints in First-Order Optimization: A View from Non-Smooth Dynamical Systems

Authors: Michael Muehlebach, Michael I. Jordan

Abstract: We introduce a class of first-order methods for smooth constrained optimization that are based on an analogy to non-smooth dynamical systems. Two distinctive features of our approach are that (i) projections or optimizations over the entire feasible set are avoided, in stark contrast to projected gradient methods or the Frank-Wolfe method, and (ii) iterates are allowed to become infeasible, which… ▽ More We introduce a class of first-order methods for smooth constrained optimization that are based on an analogy to non-smooth dynamical systems. Two distinctive features of our approach are that (i) projections or optimizations over the entire feasible set are avoided, in stark contrast to projected gradient methods or the Frank-Wolfe method, and (ii) iterates are allowed to become infeasible, which differs from active set or feasible direction methods, where the descent motion stops as soon as a new constraint is encountered. The resulting algorithmic procedure is simple to implement even when constraints are nonlinear, and is suitable for large-scale constrained optimization problems in which the feasible set fails to have a simple structure. The key underlying idea is that constraints are expressed in terms of velocities instead of positions, which has the algorithmic consequence that optimizations over feasible sets at each iteration are replaced with optimizations over local, sparse convex approximations. In particular, this means that at each iteration only constraints that are violated are taken into account. The result is a simplified suite of algorithms and an expanded range of possible applications in machine learning. △ Less

Submitted 5 November, 2022; v1 submitted 17 July, 2021; originally announced July 2021.

Comments: 47 pages, 11 figures

Continuous-time Lower Bounds for Gradient-based Algorithms

Authors: Michael Muehlebach, Michael I. Jordan

Abstract: This article derives lower bounds on the convergence rate of continuous-time gradient-based optimization algorithms. The algorithms are subjected to a time-normalization constraint that avoids a reparametrization of time in order to make the discussion of continuous-time convergence rates meaningful. We reduce the multi-dimensional problem to a single dimension, recover well-known lower bounds fro… ▽ More This article derives lower bounds on the convergence rate of continuous-time gradient-based optimization algorithms. The algorithms are subjected to a time-normalization constraint that avoids a reparametrization of time in order to make the discussion of continuous-time convergence rates meaningful. We reduce the multi-dimensional problem to a single dimension, recover well-known lower bounds from the discrete-time setting, and provide insight into why these lower bounds occur. We present algorithms that achieve the proposed lower bounds, even when the function class under consideration includes certain nonconvex functions. △ Less

Submitted 3 August, 2020; v1 submitted 10 February, 2020; originally announced February 2020.

Comments: 13 pages

The Silver Ratio and its Relation to Controllability

Authors: Michael Muehlebach

Abstract: This note investigates the controllability of two unstable second-order systems that are coupled through a common input. These dynamics occur for different types of inverted-pendulum systems. Controllability is quantified by the volume of the state-space that can be reached with unit energy, provided that the system starts and ends at the origin. It is shown that controllability is maximized when… ▽ More This note investigates the controllability of two unstable second-order systems that are coupled through a common input. These dynamics occur for different types of inverted-pendulum systems. Controllability is quantified by the volume of the state-space that can be reached with unit energy, provided that the system starts and ends at the origin. It is shown that controllability is maximized when the ratio between the time constants amounts to the silver ratio. △ Less

Submitted 19 August, 2019; originally announced August 2019.

A Dynamical Systems Perspective on Nesterov Acceleration

Authors: Michael Muehlebach, Michael I. Jordan

Abstract: We present a dynamical system framework for understanding Nesterov's accelerated gradient method. In contrast to earlier work, our derivation does not rely on a vanishing step size argument. We show that Nesterov acceleration arises from discretizing an ordinary differential equation with a semi-implicit Euler integration scheme. We analyze both the underlying differential equation as well as the… ▽ More We present a dynamical system framework for understanding Nesterov's accelerated gradient method. In contrast to earlier work, our derivation does not rely on a vanishing step size argument. We show that Nesterov acceleration arises from discretizing an ordinary differential equation with a semi-implicit Euler integration scheme. We analyze both the underlying differential equation as well as the discretization to obtain insights into the phenomenon of acceleration. The analysis suggests that a curvature-dependent dam** term lies at the heart of the phenomenon. We further establish connections between the discretized and the continuous-time dynamics. △ Less

Submitted 17 May, 2019; originally announced May 2019.

Comments: 11 pages, 4 figures, to appear in the Proceedings of the 36th International Conference on Machine Learning

A Method for Reducing the Complexity of Model Predictive Control in Robotics Applications

Authors: Michael Muehlebach, Raffaello D'Andrea

Abstract: This article describes an approach for parametrizing input and state trajectories in model predictive control. The parametrization is designed to be invariant to time shifts, which enables warm-starting the successive optimization problems and reduces the computational complexity of the online optimization. It is shown that in certain cases (e.g. for linear time-invariant dynamics with input and s… ▽ More This article describes an approach for parametrizing input and state trajectories in model predictive control. The parametrization is designed to be invariant to time shifts, which enables warm-starting the successive optimization problems and reduces the computational complexity of the online optimization. It is shown that in certain cases (e.g. for linear time-invariant dynamics with input and state constraints) the parametrization leads to inherent stability and recursive feasibility guarantees without additional terminal set constraints. Due to the fact that the number of decision variables are greatly reduced through the parametrization, while the warm-starting capabilities are preserved, the approach is suitable for applications where the available computational resources (memory and CPU-power) are limited. △ Less

Submitted 18 March, 2019; originally announced March 2019.

On the Approximation of Constrained Linear Quadratic Regulator Problems and their Application to Model Predictive Control - Supplementary Notes

Authors: Michael Muehlebach, Raffaello D'Andrea

Abstract: By parametrizing input and state trajectories with basis functions different approximations to the constrained linear quadratic regulator problem are obtained. These notes present and discuss technical results that are intended to supplement a corresponding journal article. The results can be applied in a model predictive control context. By parametrizing input and state trajectories with basis functions different approximations to the constrained linear quadratic regulator problem are obtained. These notes present and discuss technical results that are intended to supplement a corresponding journal article. The results can be applied in a model predictive control context. △ Less

Submitted 23 February, 2018; originally announced March 2018.

Comments: 19 pages, 1 figure

Distributed Event-Based State Estimation for Networked Systems: An LMI-Approach

Authors: Michael Muehlebach, Sebastian Trimpe

Abstract: In this work, a dynamic system is controlled by multiple sensor-actuator agents, each of them commanding and observing parts of the system's input and output. The different agents sporadically exchange data with each other via a common bus network according to local event-triggering protocols. From these data, each agent estimates the complete dynamic state of the system and uses its estimate for… ▽ More In this work, a dynamic system is controlled by multiple sensor-actuator agents, each of them commanding and observing parts of the system's input and output. The different agents sporadically exchange data with each other via a common bus network according to local event-triggering protocols. From these data, each agent estimates the complete dynamic state of the system and uses its estimate for feedback control. We propose a synthesis procedure for designing the agents' state estimators and the event triggering thresholds. The resulting distributed and event-based control system is guaranteed to be stable and to satisfy a predefined estimation performance criterion. The approach is applied to the control of a vehicle platoon, where the method's trade-off between performance and communication, and the scalability in the number of agents is demonstrated. △ Less

Submitted 6 July, 2017; originally announced July 2017.

Comments: This is an extended version of an article to appear in the IEEE Transactions on Automatic Control (additional parts in the Appendix)

Approximation of Continuous-Time Infinite-Horizon Optimal Control Problems Arising in Model Predictive Control - Supplementary Notes

Authors: Michael Muehlebach, Raffaello D'Andrea

Abstract: These notes present preliminary results regarding two different approximations of linear infinite-horizon optimal control problems arising in model predictive control. Input and state trajectories are parametrized with basis functions and a finite dimensional representation of the dynamics is obtained via a Galerkin approach. It is shown that the two approximations provide lower, respectively uppe… ▽ More These notes present preliminary results regarding two different approximations of linear infinite-horizon optimal control problems arising in model predictive control. Input and state trajectories are parametrized with basis functions and a finite dimensional representation of the dynamics is obtained via a Galerkin approach. It is shown that the two approximations provide lower, respectively upper bounds on the optimal cost of the underlying infinite dimensional optimal control problem. These bounds get tighter as the number of basis functions is increased. In addition, conditions guaranteeing convergence to the cost of the underlying problem are provided. △ Less

Submitted 31 August, 2016; originally announced August 2016.

Comments: Supplementary notes, 10 pages