Showing 51–100 of 211 results for author: Lygeros, J

BIG Hype: Best Intervention in Games via Distributed Hypergradient Descent

Authors: Panagiotis D. Grontas, Giuseppe Belgioioso, Carlo Cenedese, Marta Fochesato, John Lygeros, Florian Dörfler

Abstract: Hierarchical decision making problems, such as bilevel programs and Stackelberg games, are attracting increasing interest in both the engineering and machine learning communities. Yet, existing solution methods lack either convergence guarantees or computational efficiency, due to the absence of smoothness and convexity. In this work, we bridge this gap by designing a first-order hypergradient-bas… ▽ More Hierarchical decision making problems, such as bilevel programs and Stackelberg games, are attracting increasing interest in both the engineering and machine learning communities. Yet, existing solution methods lack either convergence guarantees or computational efficiency, due to the absence of smoothness and convexity. In this work, we bridge this gap by designing a first-order hypergradient-based algorithm for Stackelberg games and mathematically establishing its convergence using tools from nonsmooth analysis. To evaluate the \textit{hypergradient}, namely, the gradient of the upper-level objective, we develop an online scheme that simultaneously computes the lower-level equilibrium and its Jacobian. Crucially, this scheme exploits and preserves the original hierarchical and distributed structure of the problem, which renders it scalable and privacy-preserving. We numerically verify the computational efficiency and scalability of our algorithm on a large-scale hierarchical demand-response model. △ Less

Submitted 28 March, 2024; v1 submitted 2 March, 2023; originally announced March 2023.

Designing Fairness in Autonomous Peer-to-peer Energy Trading

Authors: Varsha Behrunani, Andrew Irvine, Giuseppe Belgioioso, Philipp Heer, John Lygeros, Florian Dörfler

Abstract: Several autonomous energy management and peer-to-peer trading mechanisms for future energy markets have been recently proposed based on optimization and game theory. In this paper, we study the impact of trading prices on the outcome of these market designs for energy-hub networks. We prove that, for a generic choice of trading prices, autonomous peer-to-peer trading is always network-wide benefic… ▽ More Several autonomous energy management and peer-to-peer trading mechanisms for future energy markets have been recently proposed based on optimization and game theory. In this paper, we study the impact of trading prices on the outcome of these market designs for energy-hub networks. We prove that, for a generic choice of trading prices, autonomous peer-to-peer trading is always network-wide beneficial but not necessarily individually beneficial for each hub. Therefore, we leverage hierarchical game theory to formalize the problem of designing locally-beneficial and network-wide fair peer-to-peer trading prices. Then, we propose a scalable and privacy-preserving price-mediation algorithm that provably converges to a profile of such prices. Numerical simulations on a 3-hub network show that the proposed algorithm can indeed incentivize active participation of energy hubs in autonomous peer-to-peer trading schemes. △ Less

Submitted 9 February, 2023; originally announced February 2023.

Comments: 9 pages, 6 figures. Submitted to IFAC World Congress 2023

Stress Flow Guided Non-Planar Print Trajectory Optimization for Additive Manufacturing of Anisotropic Polymers

Authors: Xavier Guidetti, Efe C. Balta, Yannick Nagel, Hang Yin, Alisa Rupenyan, John Lygeros

Abstract: When manufacturing parts using material extrusion additive manufacturing and anisotropic polymers, the mechanical properties of a manufactured component are strongly dependent on the print trajectory orientation. We conduct non-planar slicing and optimize the print trajectories to maximize the alignment between the material deposition direction and the stress flow induced by a predefined load case… ▽ More When manufacturing parts using material extrusion additive manufacturing and anisotropic polymers, the mechanical properties of a manufactured component are strongly dependent on the print trajectory orientation. We conduct non-planar slicing and optimize the print trajectories to maximize the alignment between the material deposition direction and the stress flow induced by a predefined load case. The trajectory optimization framework considers manufacturability constraints in the form of uniform layer height and line spacing. We demonstrate the method by manufacturing a load bearing mechanical bracket using a 5-axis 3D printer and a liquid crystal polymer material. The failure strength and stiffness of the optimized bracket are improved by a factor of 44 and 6 respectively when compared with conventional printing. △ Less

Submitted 12 June, 2023; v1 submitted 12 January, 2023; originally announced January 2023.

Comments: Accepted and published on Elsevier Additive Manufacturing

Journal ref: Additive Manufacturing, Volume 72, 2023, 103628

Efficient sample selection for safe learning

Authors: Marta Zagorowska, Efe C. Balta, Varsha Behrunani, Alisa Rupenyan, John Lygeros

Abstract: Ensuring safety in industrial control systems usually involves imposing constraints at the design stage of the control algorithm. Enforcing constraints is challenging if the underlying functional form is unknown. The challenge can be addressed by using surrogate models, such as Gaussian processes, which provide confidence intervals used to find solutions that can be considered safe. This in turn i… ▽ More Ensuring safety in industrial control systems usually involves imposing constraints at the design stage of the control algorithm. Enforcing constraints is challenging if the underlying functional form is unknown. The challenge can be addressed by using surrogate models, such as Gaussian processes, which provide confidence intervals used to find solutions that can be considered safe. This in turn involves an exhaustive search on the entire search space. That approach can quickly become computationally expensive. We reformulate the exhaustive search as a series of optimization problems to find the next recommended points. We show that the proposed reformulation allows using a wide range of available optimization solvers, such as derivative-free methods. We show that by exploiting the properties of the solver, we enable the introduction of new stop** criteria into safe learning methods and increase flexibility in trading off solver accuracy and computational time. The results from a non-convex optimization problem and an application for controller tuning confirm the flexibility and the performance of the proposed reformulation. △ Less

Submitted 8 June, 2023; v1 submitted 25 November, 2022; originally announced November 2022.

Optimal service station design for traffic mitigation via genetic algorithm and neural network

Authors: Carlo Cenedese, Michele Cucuzzella, Adriano Cotta Ramusino, Davide Spalenza, John Lygeros, Antonella Ferrara

Abstract: This paper analyzes how the presence of service stations on highways affects traffic congestion. We focus on the problem of optimally designing a service station to achieve beneficial effects in terms of total traffic congestion and peak traffic reduction. Microsimulators cannot be used for this task due to their computational inefficiency. We propose a genetic algorithm based on the recently prop… ▽ More This paper analyzes how the presence of service stations on highways affects traffic congestion. We focus on the problem of optimally designing a service station to achieve beneficial effects in terms of total traffic congestion and peak traffic reduction. Microsimulators cannot be used for this task due to their computational inefficiency. We propose a genetic algorithm based on the recently proposed CTMs, that efficiently describes the dynamics of a service station. Then, we leverage the algorithm to train a neural network capable of solving the same problem, avoiding implementing the CTMs. Finally, we examine two case studies to validate the capabilities and performance of our algorithms. In these simulations, we use real data extracted from Dutch highways. △ Less

Submitted 18 November, 2022; originally announced November 2022.

Comments: Submitted to IFAC Worlds conference 2023

Drone-based Volume Estimation in Indoor Environments

Authors: Samuel Balula, Dominic Liao-McPherson, Stefan Stevšić, Alisa Rupenyan, John Lygeros

Abstract: Volume estimation in large indoor spaces is an important challenge in robotic inspection of industrial warehouses. We propose an approach for volume estimation for autonomous systems using visual features for indoor localization and surface reconstruction from 2D-LiDAR measurements. A Gaussian Process-based model incorporates information collected from measurements given statistical prior informat… ▽ More Volume estimation in large indoor spaces is an important challenge in robotic inspection of industrial warehouses. We propose an approach for volume estimation for autonomous systems using visual features for indoor localization and surface reconstruction from 2D-LiDAR measurements. A Gaussian Process-based model incorporates information collected from measurements given statistical prior information about the terrain, from which the volume estimate is computed. Our algorithm finds feasible trajectories which minimize the uncertainty of the volume estimate. We show results in simulation for the surface reconstruction and volume estimate of topographic data. △ Less

Submitted 15 November, 2022; originally announced November 2022.

Comments: 6 pages, 10 figures, Conference paper

Probabilistic Reachability and Invariance Computation of Stochastic Systems using Linear Programming

Authors: Niklas Schmid, John Lygeros

Abstract: We consider the safety evaluation of discrete time, stochastic systems over a finite horizon. Therefore, we discuss and link probabilistic invariance with reachability as well as reach-avoid problems. We show how to efficiently compute these quantities using dynamic and linear programming. We consider the safety evaluation of discrete time, stochastic systems over a finite horizon. Therefore, we discuss and link probabilistic invariance with reachability as well as reach-avoid problems. We show how to efficiently compute these quantities using dynamic and linear programming. △ Less

Submitted 14 April, 2023; v1 submitted 14 November, 2022; originally announced November 2022.

Implications of Regret on Stability of Linear Dynamical Systems

Authors: Aren Karapetyan, Anastasios Tsiamis, Efe C. Balta, Andrea Iannelli, John Lygeros

Abstract: The setting of an agent making decisions under uncertainty and under dynamic constraints is common for the fields of optimal control, reinforcement learning, and recently also for online learning. In the online learning setting, the quality of an agent's decision is often quantified by the concept of regret, comparing the performance of the chosen decisions to the best possible ones in hindsight.… ▽ More The setting of an agent making decisions under uncertainty and under dynamic constraints is common for the fields of optimal control, reinforcement learning, and recently also for online learning. In the online learning setting, the quality of an agent's decision is often quantified by the concept of regret, comparing the performance of the chosen decisions to the best possible ones in hindsight. While regret is a useful performance measure, when dynamical systems are concerned, it is important to also assess the stability of the closed-loop system for a chosen policy. In this work, we show that for linear state feedback policies and linear systems subject to adversarial disturbances, linear regret implies asymptotic stability in both time-varying and time-invariant settings. Conversely, we also show that bounded input bounded state stability and summability of the state transition matrices imply linear regret. △ Less

Submitted 15 April, 2023; v1 submitted 14 November, 2022; originally announced November 2022.

Comments: Accepted for presentation at the IFAC World Congress 2023, Yokohama, Japan

Follow the Clairvoyant: an Imitation Learning Approach to Optimal Control

Authors: Andrea Martin, Luca Furieri, Florian Dörfler, John Lygeros, Giancarlo Ferrari-Trecate

Abstract: We consider control of dynamical systems through the lens of competitive analysis. Most prior work in this area focuses on minimizing regret, that is, the loss relative to an ideal clairvoyant policy that has noncausal access to past, present, and future disturbances. Motivated by the observation that the optimal cost only provides coarse information about the ideal closed-loop behavior, we instea… ▽ More We consider control of dynamical systems through the lens of competitive analysis. Most prior work in this area focuses on minimizing regret, that is, the loss relative to an ideal clairvoyant policy that has noncausal access to past, present, and future disturbances. Motivated by the observation that the optimal cost only provides coarse information about the ideal closed-loop behavior, we instead propose directly minimizing the tracking error relative to the optimal trajectories in hindsight, i.e., imitating the clairvoyant policy. By embracing a system level perspective, we present an efficient optimization-based approach for computing follow-the-clairvoyant (FTC) safe controllers. We prove that these attain minimal regret if no constraints are imposed on the noncausal benchmark. In addition, we present numerical experiments to show that our policy retains the hallmark of competitive algorithms of interpolating between classical $\mathcal{H}_2$ and $\mathcal{H}_\infty$ control laws - while consistently outperforming regret minimization methods in constrained scenarios thanks to the superior ability to chase the clairvoyant. △ Less

Submitted 14 November, 2022; originally announced November 2022.

Stability and Robustness of Distributed Suboptimal Model Predictive Control

Authors: Giuseppe Belgioioso, Dominic Liao-McPherson, Mathias Hudoba de Badyn, Nicolas Pelzmann, John Lygeros, Florian Dörfler

Abstract: In distributed model predictive control (MPC), the control input at each sampling time is computed by solving a large-scale optimal control problem (OCP) over a finite horizon using distributed algorithms. Typically, such algorithms require several (virtually, infinite) communication rounds between the subsystems to converge, which is a major drawback both computationally and from an energetic per… ▽ More In distributed model predictive control (MPC), the control input at each sampling time is computed by solving a large-scale optimal control problem (OCP) over a finite horizon using distributed algorithms. Typically, such algorithms require several (virtually, infinite) communication rounds between the subsystems to converge, which is a major drawback both computationally and from an energetic perspective (for wireless systems). Motivated by these challenges, we propose a suboptimal distributed MPC scheme in which the total communication burden is distributed also in time, by maintaining a running solution estimate for the large-scale OCP and updating it at each sampling time. We demonstrate that, under some regularity conditions, the resulting suboptimal MPC control law recovers the qualitative robust stability properties of optimal MPC, if the communication budget at each sampling time is large enough. △ Less

Submitted 27 March, 2023; v1 submitted 14 November, 2022; originally announced November 2022.

Performance Bounds of Model Predictive Control for Unconstrained and Constrained Linear Quadratic Problems and Beyond

Authors: Yuchao Li, Aren Karapetyan, John Lygeros, Karl H. Johansson, Jonas Mårtensson

Abstract: We study unconstrained and constrained linear quadratic problems and investigate the suboptimality of the model predictive control (MPC) method applied to such problems. Considering MPC as an approximate scheme for solving the related fixed point equations, we derive performance bounds for the closed-loop system under MPC. Our analysis, as well as numerical examples, suggests new ways of choosing… ▽ More We study unconstrained and constrained linear quadratic problems and investigate the suboptimality of the model predictive control (MPC) method applied to such problems. Considering MPC as an approximate scheme for solving the related fixed point equations, we derive performance bounds for the closed-loop system under MPC. Our analysis, as well as numerical examples, suggests new ways of choosing the terminal cost and terminal constraints, which are \emph{not} related to the solution of the Riccati equation of the original problem. The resulting method can have a larger feasible region, and cause hardly any loss of performance in terms of the closed-loop cost over an infinite horizon. △ Less

Submitted 3 June, 2023; v1 submitted 11 November, 2022; originally announced November 2022.

Inexact GMRES Policy Iteration for Large-Scale Markov Decision Processes

Authors: Matilde Gargiani, Dominic Liao-McPherson, Andrea Zanelli, John Lygeros

Abstract: Policy iteration enjoys a local quadratic rate of contraction, but its iterations are computationally expensive for Markov decision processes (MDPs) with a large number of states. In light of the connection between policy iteration and the semismooth Newton method and taking inspiration from the inexact variants of the latter, we propose \textit{inexact policy iteration}, a new class of methods fo… ▽ More Policy iteration enjoys a local quadratic rate of contraction, but its iterations are computationally expensive for Markov decision processes (MDPs) with a large number of states. In light of the connection between policy iteration and the semismooth Newton method and taking inspiration from the inexact variants of the latter, we propose \textit{inexact policy iteration}, a new class of methods for large-scale finite MDPs with local contraction guarantees. We then design an instance based on the deployment of GMRES for the approximate policy evaluation step, which we call inexact GMRES policy iteration. Finally, we demonstrate the superior practical performance of inexact GMRES policy iteration on an MDP with 10000 states, where it achieves a $\times 5.8$ and $\times 2.2$ speedup with respect to policy iteration and optimistic policy iteration, respectively. △ Less

Submitted 8 November, 2022; originally announced November 2022.

Data-Driven Process Optimization of Fused Filament Fabrication based on In Situ Measurements

Authors: Xavier Guidetti, Marino Kühne, Yannick Nagel, Efe C. Balta, Alisa Rupenyan, John Lygeros

Abstract: The tuning of fused filament fabrication parameters is notoriously challenging. We propose an autonomous data-driven method to select parameters based on in situ measurements. We use a laser sensor to evaluate the surface roughness of a printed part. We then correlate the roughness to the mechanical properties of the part, and show how print quality affects mechanical performance. Finally, we use… ▽ More The tuning of fused filament fabrication parameters is notoriously challenging. We propose an autonomous data-driven method to select parameters based on in situ measurements. We use a laser sensor to evaluate the surface roughness of a printed part. We then correlate the roughness to the mechanical properties of the part, and show how print quality affects mechanical performance. Finally, we use Bayesian optimization to search for optimal print parameters. We demonstrate our method by printing liquid crystal polymer samples, and successfully find parameters that produce high-performance prints and maximize the manufacturing process efficiency. △ Less

Submitted 27 October, 2022; originally announced October 2022.

Comments: 7 pages, 6 figures. Submitted to IFAC WC 2023

Online Feedback Equilibrium Seeking

Authors: Giuseppe Belgioioso, Dominic Liao-McPherson, Mathias Hudoba de Badyn, Saverio Bolognani, Roy S. Smith, John Lygeros, Florian Dörfler

Abstract: This paper proposes a unifying design framework for dynamic feedback controllers that track solution trajectories of time-varying generalized equations, such as local minimizers of nonlinear programs or competitive equilibria (e.g., Nash) of non-cooperative games. Inspired by the feedback optimization paradigm, the core idea of the proposed approach is to re-purpose classic iterative algorithms fo… ▽ More This paper proposes a unifying design framework for dynamic feedback controllers that track solution trajectories of time-varying generalized equations, such as local minimizers of nonlinear programs or competitive equilibria (e.g., Nash) of non-cooperative games. Inspired by the feedback optimization paradigm, the core idea of the proposed approach is to re-purpose classic iterative algorithms for solving generalized equations (e.g., Josephy--Newton, forward-backward splitting) as dynamic feedback controllers by integrating online measurements of the continuous-time nonlinear plant. Sufficient conditions for closed-loop stability and robustness of the algorithm-plant cyber-physical interconnection are derived in a sampled-data setting by combining and tailoring results from (monotone) operator, fixed-point, and nonlinear systems theory. Numerical simulations on smart building automation and competitive supply-chain management are presented to support the theoretical findings. △ Less

Submitted 14 February, 2024; v1 submitted 21 October, 2022; originally announced October 2022.

Data-driven distributionally robust MPC for systems with uncertain dynamics

Authors: Francesco Micheli, Tyler Summers, John Lygeros

Abstract: We present a novel data-driven distributionally robust Model Predictive Control formulation for unknown discrete-time linear time-invariant systems affected by unknown and possibly unbounded additive uncertainties. We use off-line collected data and an approximate model of the dynamics to formulate a finite-horizon optimization problem. To account for both the uncertainty related to the dynamics a… ▽ More We present a novel data-driven distributionally robust Model Predictive Control formulation for unknown discrete-time linear time-invariant systems affected by unknown and possibly unbounded additive uncertainties. We use off-line collected data and an approximate model of the dynamics to formulate a finite-horizon optimization problem. To account for both the uncertainty related to the dynamics and the disturbance acting on the system, we resort to a distributionally robust formulation that optimizes the cost expectation while satisfying Conditional Value-at-Risk constraints with respect to the worst-case probability distributions of the uncertainties within an ambiguity set defined using the Wasserstein metric. Using results from the distributionally robust optimization literature we derive a tractable finite-dimensional convex optimization problem with finite-sample guarantees for the class of convex piecewise affine cost and constraint functions. The performance of the proposed algorithm is demonstrated in closed-loop simulation on a simple numerical example. △ Less

Submitted 19 September, 2022; originally announced September 2022.

CARMA: Fair and efficient bottleneck congestion management via non-tradable karma credits

Authors: Ezzat Elokda, Carlo Cenedese, Kenan Zhang, Andrea Censi, John Lygeros, Emilio Frazzoli, Florian Dörfler

Abstract: This paper proposes a non-monetary traffic demand management scheme, named CARMA, as a fair solution to the morning commute congestion. We consider heterogeneous commuters traveling through a single bottleneck that differ in both the desired arrival time and Value of Time (VOT). We consider a generalized notion of VOT by allowing it to vary dynamically on each day (e.g., according to trip purpose… ▽ More This paper proposes a non-monetary traffic demand management scheme, named CARMA, as a fair solution to the morning commute congestion. We consider heterogeneous commuters traveling through a single bottleneck that differ in both the desired arrival time and Value of Time (VOT). We consider a generalized notion of VOT by allowing it to vary dynamically on each day (e.g., according to trip purpose and urgency), rather than being a static characteristic of each individual. In our CARMA scheme, the bottleneck is divided into a fast lane that is kept in free flow and a slow lane that is subject to congestion. We introduce a non-tradable mobility credit, named karma, that is used by commuters to bid for access to the fast lane. Commuters who get outbid or do not participate in the CARMA scheme instead use the slow lane. At the end of each day, karma collected from the bidders is redistributed, and the process repeats day by day. We model the collective commuter behaviors under CARMA as a Dynamic Population Game (DPG), in which a Stationary Nash Equilibrium (SNE) is guaranteed to exist. Unlike existing monetary schemes, CARMA is demonstrated, both analytically and numerically, to achieve a) an equitable traffic assignment with respect to heterogeneous income classes and b) a strong Pareto improvement in the long-term average travel disutility with respect to no policy intervention. With extensive numerical analysis, we show that CARMA is able to retain the same congestion reduction as an optimal monetary tolling scheme under uniform karma redistribution and even outperform tolling under a well-designed redistribution scheme. We also highlight the privacy-preserving feature of CARMA, i.e., its ability to tailor to the private preferences of commuters without centrally collecting the information. △ Less

Submitted 18 September, 2023; v1 submitted 15 August, 2022; originally announced August 2022.

Scenario-based Stochastic MPC for systems with uncertain dynamics

Authors: Francesco Micheli, John Lygeros

Abstract: Model Predictive Control is an extremely effective control method for systems with input and state constraints. Model Predictive Control performance heavily depends on the accuracy of the open-loop prediction. For systems with uncertainty this in turn depends on the information that is available about the properties of the model and disturbance uncertainties. Here we are interested in situations w… ▽ More Model Predictive Control is an extremely effective control method for systems with input and state constraints. Model Predictive Control performance heavily depends on the accuracy of the open-loop prediction. For systems with uncertainty this in turn depends on the information that is available about the properties of the model and disturbance uncertainties. Here we are interested in situations where such information is only available through realizations of the system trajectories. We propose a general scenario-based optimization framework for stochastic control of a linear system affected by additive disturbance, when the dynamics are only approximately known. The main contribution is in the derivation of an upper bound on the number of scenarios required to provide probabilistic guarantees on the quality of the solution to the deterministic scenario-based finite horizon optimal control problem. We provide a theoretical analysis of the sample complexity of the proposed method and demonstrate its performance on a simple simulation example. Since the proposed approach leverages sampling, it does not rely on the explicit knowledge of the model or disturbance distributions, making it applicable in a wide variety of contexts. △ Less

Submitted 25 July, 2022; originally announced July 2022.

Moving-Horizon State Estimation for Power Networks and Synchronous Generators

Authors: Milos Katanic, John Lygeros, Gabriela Hug

Abstract: Power network and generators state estimation are usually tackled as separate problems. We propose a dynamic scheme for the simultaneous estimation of the network and the generator states. The estimation is formulated as an optimization problem on a moving-horizon of past observations. The framework is a generalization of static state estimation; it can handle incomplete model knowledge and does n… ▽ More Power network and generators state estimation are usually tackled as separate problems. We propose a dynamic scheme for the simultaneous estimation of the network and the generator states. The estimation is formulated as an optimization problem on a moving-horizon of past observations. The framework is a generalization of static state estimation; it can handle incomplete model knowledge and does not require static network observability by PMUs. The numerical results show an improved estimation accuracy compared to static state estimation. Moreover, accurate estimation of the internal states of generators without PMUs on their terminals can be achieved. Finally, we highlight the capability of the proposed estimator to detect and identify bad data. △ Less

Submitted 25 July, 2022; originally announced July 2022.

Reconfigurable Plug-and-play Distributed Model Predictive Control for Reference Tracking

Authors: Ahmed Aboudonia, Andrea Martinelli, Nicolas Hoischen, John Lygeros

Abstract: A plug-and-play model predictive control (PnP MPC) scheme is proposed for varying-topology networks to track piecewise constant references. The proposed scheme allows subsystems to occasionally join and leave the network while preserving asymptotic stability and recursive feasibility and comprises two main phases. In the redesign phase, passivity-based control is used to ensure that asymptotic sta… ▽ More A plug-and-play model predictive control (PnP MPC) scheme is proposed for varying-topology networks to track piecewise constant references. The proposed scheme allows subsystems to occasionally join and leave the network while preserving asymptotic stability and recursive feasibility and comprises two main phases. In the redesign phase, passivity-based control is used to ensure that asymptotic stability of the network is preserved. In the transition phase, reconfigurable terminal ingredients are used to ensure that the distributed MPC problem is initially feasible after the PnP operation. The efficacy of the proposed scheme is evaluated by applying it to a network of mass-spring-damper systems and comparing it to a benchmark scheme. It is found that the novel redesign phase results in faster PnP operations, whereas the novel transition phase increases flexibility by accepting more requests. △ Less

Submitted 16 November, 2022; v1 submitted 19 July, 2022; originally announced July 2022.

Online Computation of Terminal Ingredients in Distributed Model Predictive Control for Reference Tracking

Authors: Ahmed Aboudonia, Goran Banjac, Annika Eichler, John Lygeros

Abstract: A distributed model predictive control scheme is developed for tracking piecewise constant references where the terminal set is reconfigured online, whereas the terminal controller is computed offline. Unlike many standard existing schemes, this scheme yields large feasible regions without performing offline centralized computations. Although the resulting optimal control problem (OCP) is a semide… ▽ More A distributed model predictive control scheme is developed for tracking piecewise constant references where the terminal set is reconfigured online, whereas the terminal controller is computed offline. Unlike many standard existing schemes, this scheme yields large feasible regions without performing offline centralized computations. Although the resulting optimal control problem (OCP) is a semidefinite program (SDP), an SDP scalability method based on diagonal dominance is used to approximate the derived SDP by a second-order cone program. The OCPs of the proposed scheme and its approximation are amenable to distributed optimization. Both schemes are evaluated using a power network example and compared to a scheme where the terminal controller is reconfigured online as well. It is found that fixing the terminal controller results in better performance, noticeable reduction in computational cost and similar feasible region compared to the case in which this controller is reconfigured online. △ Less

Submitted 19 July, 2022; originally announced July 2022.

Robust and Kernelized Data-Enabled Predictive Control for Nonlinear Systems

Authors: Linbin Huang, John Lygeros, Florian Dörfler

Abstract: This paper presents a robust and kernelized data-enabled predictive control (RoKDeePC) algorithm to perform model-free optimal control for nonlinear systems using only input and output data. The algorithm combines robust predictive control and a non-parametric representation of nonlinear systems enabled by regularized kernel methods. The latter is based on implicitly learning the nonlinear behavio… ▽ More This paper presents a robust and kernelized data-enabled predictive control (RoKDeePC) algorithm to perform model-free optimal control for nonlinear systems using only input and output data. The algorithm combines robust predictive control and a non-parametric representation of nonlinear systems enabled by regularized kernel methods. The latter is based on implicitly learning the nonlinear behavior of the system via the representer theorem. Instead of seeking a model and then performing control design, our method goes directly from data to control. This allows us to robustify the control inputs against the uncertainties in data by considering a min-max optimization problem to calculate the optimal control sequence. We show that by incorporating a proper uncertainty set, this min-max problem can be reformulated as a nonconvex but structured minimization problem. By exploiting its structure, we present a projected gradient descent algorithm to effectively solve this problem. Finally, we test the RoKDeePC on two nonlinear example systems - one academic case study and a grid-forming converter feeding a nonlinear load - and compare it with some existing nonlinear data-driven predictive control methods. △ Less

Submitted 3 June, 2022; originally announced June 2022.

Data-driven Reference Trajectory Optimization for Precision Motion Systems

Authors: Samuel Balula, Dominic Liao-McPherson, Alisa Rupenyan, John Lygeros

Abstract: We propose a data-driven optimization-based pre-compensation method to improve the contour tracking performance of precision motion stages by modifying the reference trajectory and without modifying any built-in low-level controllers. The position of the precision motion stage is predicted with data-driven models, a linear low-fidelity model is used to optimize traversal time, by changing the path… ▽ More We propose a data-driven optimization-based pre-compensation method to improve the contour tracking performance of precision motion stages by modifying the reference trajectory and without modifying any built-in low-level controllers. The position of the precision motion stage is predicted with data-driven models, a linear low-fidelity model is used to optimize traversal time, by changing the path velocity and acceleration profiles then a non-linear high-fidelity model is used to refine the previously found time-optimal solution. We experimentally demonstrate that the proposed method is capable of simultaneously improving the productivity and accuracy of a high precision motion stage. Given the data-based nature of the models, the proposed method can easily be adapted to a wide family of precision motion systems. △ Less

Submitted 5 September, 2022; v1 submitted 31 May, 2022; originally announced May 2022.

A Novel Control-Oriented Cell Transmission Model Including Service Stations on Highways

Authors: Carlo Cenedese, Michele Cucuzzella, Antonella Ferrara, John Lygeros

Abstract: In this paper, we propose a novel model that describes how the traffic evolution on a highway stretch is affected by the presence of a service station. The presented model enhances the classical CTM dynamics by adding the dynamics associated with the service stations, where the vehicles may stop before merging back into the mainstream. We name it CTMs. We discuss its flexibility in describing diff… ▽ More In this paper, we propose a novel model that describes how the traffic evolution on a highway stretch is affected by the presence of a service station. The presented model enhances the classical CTM dynamics by adding the dynamics associated with the service stations, where the vehicles may stop before merging back into the mainstream. We name it CTMs. We discuss its flexibility in describing different complex scenarios where multiple stations are characterized by different drivers' average stop** times corresponding to different services. The model has been developed to help design control strategies aimed at decreasing traffic congestion. Thus, we discuss how classical control schemes can interact with the proposed \gls{CTMs}. Finally, we validate the proposed model through numerical simulations and assess the effects of service stations on traffic evolution, which appear to be beneficial, especially for relatively short congested periods. △ Less

Submitted 13 September, 2022; v1 submitted 23 May, 2022; originally announced May 2022.

Comments: This work is submitted to the 61st IEEE CDC 2022

Controller-Aware Dynamic Network Management for Industry 4.0

Authors: Efe C. Balta, Mohammad H. Mamduhi, John Lygeros, Alisa Rupenyan

Abstract: In this paper, we consider a cyber-physical manufacturing system (CPMS) scenario containing physical components (robots, sensors, and actuators), operating in a digitally connected, constrained environment to perform industrial tasks. The CPMS has a centralized control plane with digital twins (DTs) of the physical resources, computational resources, and a network manager that allocates network re… ▽ More In this paper, we consider a cyber-physical manufacturing system (CPMS) scenario containing physical components (robots, sensors, and actuators), operating in a digitally connected, constrained environment to perform industrial tasks. The CPMS has a centralized control plane with digital twins (DTs) of the physical resources, computational resources, and a network manager that allocates network resources. Existing approaches for allocation of network resources are typically fixed with respect to controller-dependent run-time specifications, which may impact the performance of physical processes. We propose a dynamic network management framework, where the network resource allocation schemes are controller-aware. The information about the controllers of the physical resources is implemented at the DT level, and metrics, such as regret bounds, take the process performance measures into account. The proposed network management schemes optimize physical system performance by balancing the shared resources between the physical assets on the plant floor, and by considering their control requirements, providing a new perspective for dynamic resource allocation. A simulation study is provided to illustrate the performance of the proposed network management approaches and compare their efficiencies. △ Less

Submitted 28 May, 2022; originally announced May 2022.

Advanced Manufacturing Configuration by Sample-efficient Batch Bayesian Optimization

Authors: Xavier Guidetti, Alisa Rupenyan, Lutz Fassl, Majid Nabavi, John Lygeros

Abstract: We propose a framework for the configuration and operation of expensive-to-evaluate advanced manufacturing methods, based on Bayesian optimization. The framework unifies a tailored acquisition function, a parallel acquisition procedure, and the integration of process information providing context to the optimization procedure. \cmtb{The novel acquisition function is demonstrated, analyzed and comp… ▽ More We propose a framework for the configuration and operation of expensive-to-evaluate advanced manufacturing methods, based on Bayesian optimization. The framework unifies a tailored acquisition function, a parallel acquisition procedure, and the integration of process information providing context to the optimization procedure. \cmtb{The novel acquisition function is demonstrated, analyzed and compared on state-of-the-art benchmarking problems. We apply the optimization approach to atmospheric plasma spraying and fused deposition modeling.} Our results demonstrate that the proposed framework can efficiently find input parameters that produce the desired outcome and minimize the process cost. △ Less

Submitted 12 September, 2022; v1 submitted 24 May, 2022; originally announced May 2022.

Comments: Accepted for IEEE RA-L. 8 pages, 6 figures. arXiv admin note: text overlap with arXiv:2103.13881

Journal ref: IEEE Robotics and Automation Letters, 2022

A Stackelberg game for incentive-based demand response in energy markets

Authors: Marta Fochesato, Carlo Cenedese, John Lygeros

Abstract: In modern buildings renewable energy generators and storage devices are spreading, and consequently the role of the users in the power grid is shifting from passive to active. We design a demand response scheme that exploits the prosumers' flexibility to provide ancillary services to the main grid. We propose a hierarchical scheme to coordinate the interactions between the distribution system oper… ▽ More In modern buildings renewable energy generators and storage devices are spreading, and consequently the role of the users in the power grid is shifting from passive to active. We design a demand response scheme that exploits the prosumers' flexibility to provide ancillary services to the main grid. We propose a hierarchical scheme to coordinate the interactions between the distribution system operator and a community of smart prosumers. The framework inherits characteristics from price-based and incentive-based schemes and it retains the advantages of both. We cast the problem as a Stackelberg game with the prosumers as followers and the distribution system operator as leader. We solve the resulting bilevel optimization program via a KKT reformulation, proving the existence and the convergence to a local Stackelberg equilibrium. Finally, we provide numerical simulations to corroborate our claims on the benefits of the proposed framework. △ Less

Submitted 9 November, 2022; v1 submitted 19 April, 2022; originally announced April 2022.

Comments: Accepted to 61st IEEE Conference on Decision and Control (CDC), 2022

Regret Analysis of Online Gradient Descent-based Iterative Learning Control with Model Mismatch

Authors: Efe C. Balta, Andrea Iannelli, Roy S. Smith, John Lygeros

Abstract: In Iterative Learning Control (ILC), a sequence of feedforward control actions is generated at each iteration on the basis of partial model knowledge and past measurements with the goal of steering the system toward a desired reference trajectory. This is framed here as an online learning task, where the decision-maker takes sequential decisions by solving a sequence of optimization problems havin… ▽ More In Iterative Learning Control (ILC), a sequence of feedforward control actions is generated at each iteration on the basis of partial model knowledge and past measurements with the goal of steering the system toward a desired reference trajectory. This is framed here as an online learning task, where the decision-maker takes sequential decisions by solving a sequence of optimization problems having only partial knowledge of the cost functions. Having established this connection, the performance of an online gradient-descent based scheme using inexact gradient information is analyzed in the setting of dynamic and static regret, standard measures in online learning. Fundamental limitations of the scheme and its integration with adaptation mechanisms are further investigated, followed by numerical simulations on a benchmark ILC problem. △ Less

Submitted 10 April, 2022; originally announced April 2022.

Behavioral uncertainty quantification for data-driven control

Authors: Alberto Padoan, Jeremy Coulson, Henk J. van Waarde, John Lygeros, Florian Dörfler

Abstract: This paper explores the problem of uncertainty quantification in the behavioral setting for data-driven control. Building on classical ideas from robust control, the problem is regarded as that of selecting a metric which is best suited to a data-based description of uncertainties. Leveraging on Willems' fundamental lemma, restricted behaviors are viewed as subspaces of fixed dimension, which may… ▽ More This paper explores the problem of uncertainty quantification in the behavioral setting for data-driven control. Building on classical ideas from robust control, the problem is regarded as that of selecting a metric which is best suited to a data-based description of uncertainties. Leveraging on Willems' fundamental lemma, restricted behaviors are viewed as subspaces of fixed dimension, which may be represented by data matrices. Consequently, metrics between restricted behaviors are defined as distances between points on the Grassmannian, i.e., the set of all subspaces of equal dimension in a given vector space. A new metric is defined on the set of restricted behaviors as a direct finite-time counterpart of the classical gap metric. The metric is shown to capture parametric uncertainty for the class of autoregressive (AR) models. Numerical simulations illustrate the value of the new metric with a data-driven mode recognition and control case study. △ Less

Submitted 6 April, 2022; originally announced April 2022.

Comments: Submitted to the 61st IEEE Conference on Decision and Control

Policy Iteration for Multiplicative Noise Output Feedback Control

Authors: Benjamin Gravell, Matilde Gargiani, John Lygeros, Tyler H. Summers

Abstract: We propose a policy iteration algorithm for solving the multiplicative noise linear quadratic output feedback design problem. The algorithm solves a set of coupled Riccati equations for estimation and control arising from a partially observable Markov decision process (POMDP) under a class of linear dynamic control policies. We show in numerical experiments far faster convergence than a value iter… ▽ More We propose a policy iteration algorithm for solving the multiplicative noise linear quadratic output feedback design problem. The algorithm solves a set of coupled Riccati equations for estimation and control arising from a partially observable Markov decision process (POMDP) under a class of linear dynamic control policies. We show in numerical experiments far faster convergence than a value iteration algorithm, formerly the only known algorithm for solving this class of problem. The results suggest promising future research directions for policy optimization algorithms in more general POMDPs, including the potential to develop novel approximate data-driven approaches when model parameters are not available. △ Less

Submitted 31 March, 2022; originally announced March 2022.

On the Regret of $\mathcal{H}_{\infty}$ Control

Authors: Aren Karapetyan, Andrea Iannelli, John Lygeros

Abstract: The $\mathcal{H}_{\infty}$ synthesis approach is a cornerstone robust control design technique, but is known to be conservative in some cases. The objective of this paper is to quantify the additional cost the controller incurs planning for the worst-case scenario, by adopting an approach inspired by regret from online learning. We define the \textit{disturbance-reality gap} as the difference betw… ▽ More The $\mathcal{H}_{\infty}$ synthesis approach is a cornerstone robust control design technique, but is known to be conservative in some cases. The objective of this paper is to quantify the additional cost the controller incurs planning for the worst-case scenario, by adopting an approach inspired by regret from online learning. We define the \textit{disturbance-reality gap} as the difference between the predicted worst-case disturbance signal and the actual realization. The regret is shown to scale with the norm of this \textit{gap}, which turns out to have a similar structure to that of the certainty equivalent controller with inaccurate predictions, obtained here in terms of the \textit{prediction error} norm. △ Less

Submitted 9 August, 2022; v1 submitted 30 March, 2022; originally announced March 2022.

Comments: Accepted to the 2022 IEEE Conference on Decision and Control (CDC)

Data-Driven Optimal Control of Affine Systems: A Linear Programming Perspective

Authors: Andrea Martinelli, Matilde Gargiani, Marina Draskovic, John Lygeros

Abstract: In this letter, we discuss the problem of optimal control for affine systems in the context of data-driven linear programming. First, we introduce a unified framework for the fixed point characterization of the value function, Q-function and relaxed Bellman operators. Then, in a model-free setting, we show how to synthesize and estimate Bellman inequalities from a small but sufficiently rich datas… ▽ More In this letter, we discuss the problem of optimal control for affine systems in the context of data-driven linear programming. First, we introduce a unified framework for the fixed point characterization of the value function, Q-function and relaxed Bellman operators. Then, in a model-free setting, we show how to synthesize and estimate Bellman inequalities from a small but sufficiently rich dataset. To guarantee exploration richness, we complete the extension of Willem's fundamental lemma to affine systems. △ Less

Submitted 11 July, 2022; v1 submitted 22 March, 2022; originally announced March 2022.

Journal ref: IEEE Control Systems Letters, vol. 6, pp. 3092-3097, 2022

Dynamic Programming Through the Lens of Semismooth Newton-Type Methods (Extended Version)

Authors: Matilde Gargiani, Andrea Zanelli, Dominic Liao-McPherson, Tyler Summers, John Lygeros

Abstract: Policy iteration and value iteration are at the core of many (approximate) dynamic programming methods. For Markov Decision Processes with finite state and action spaces, we show that they are instances of semismooth Newton-type methods to solve the Bellman equation. In particular, we prove that policy iteration is equivalent to the exact semismooth Newton method and enjoys local quadratic converg… ▽ More Policy iteration and value iteration are at the core of many (approximate) dynamic programming methods. For Markov Decision Processes with finite state and action spaces, we show that they are instances of semismooth Newton-type methods to solve the Bellman equation. In particular, we prove that policy iteration is equivalent to the exact semismooth Newton method and enjoys local quadratic convergence rate. This finding is corroborated by extensive numerical evidence in the fields of control and operations research, which confirms that policy iteration generally requires few iterations to achieve convergence even when the number of policies is vast. We then show that value iteration is an instance of the fixed-point iteration method. In this spirit, we develop a novel locally accelerated version of value iteration with global convergence guarantees and negligible extra computational costs. △ Less

Submitted 24 June, 2022; v1 submitted 16 March, 2022; originally announced March 2022.

On Robustness in Optimization-Based Constrained Iterative Learning Control

Authors: Dominic Liao-McPherson, Efe C. Balta, Alisa Rupenyan, John Lygeros

Abstract: Iterative learning control (ILC) is a control strategy for repetitive tasks wherein information from previous runs is leveraged to improve future performance. Optimization-based ILC (OB-ILC) is a powerful design framework for constrained ILC where measurements from the process are integrated into an optimization algorithm to provide robustness against noise and modelling error. This paper proposes… ▽ More Iterative learning control (ILC) is a control strategy for repetitive tasks wherein information from previous runs is leveraged to improve future performance. Optimization-based ILC (OB-ILC) is a powerful design framework for constrained ILC where measurements from the process are integrated into an optimization algorithm to provide robustness against noise and modelling error. This paper proposes a robust ILC controller for constrained linear processes based on the forward-backward splitting algorithm. It demonstrates how structured uncertainty information can be leveraged to ensure constraint satisfaction and provides a rigorous stability analysis in the iteration domain by combining concepts from monotone operator theory and robust control. Numerical simulations of a precision motion stage support the theoretical results. △ Less

Submitted 26 May, 2022; v1 submitted 10 March, 2022; originally announced March 2022.

Safe Control with Minimal Regret

Authors: Andrea Martin, Luca Furieri, Florian Dörfler, John Lygeros, Giancarlo Ferrari-Trecate

Abstract: As we move towards safety-critical cyber-physical systems that operate in non-stationary and uncertain environments, it becomes crucial to close the gap between classical optimal control algorithms and adaptive learning-based methods. In this paper, we present an efficient optimization-based approach for computing a finite-horizon robustly safe control policy that minimizes dynamic regret, in the… ▽ More As we move towards safety-critical cyber-physical systems that operate in non-stationary and uncertain environments, it becomes crucial to close the gap between classical optimal control algorithms and adaptive learning-based methods. In this paper, we present an efficient optimization-based approach for computing a finite-horizon robustly safe control policy that minimizes dynamic regret, in the sense of the loss relative to the optimal sequence of control actions selected in hindsight by a clairvoyant controller. By leveraging the system level synthesis framework (SLS), our method extends recent results on regret minimization for the linear quadratic regulator to optimal control subject to hard safety constraints, and allows competing against a safety-aware clairvoyant policy with minor modifications. Numerical experiments confirm superior performance with respect to finite-horizon constrained $\mathcal{H}_2$ and $\mathcal{H}_\infty$ control laws when the disturbance realizations poorly fit classical assumptions. △ Less

Submitted 14 November, 2022; v1 submitted 1 March, 2022; originally announced March 2022.

PAGE-PG: A Simple and Loopless Variance-Reduced Policy Gradient Method with Probabilistic Gradient Estimation

Authors: Matilde Gargiani, Andrea Zanelli, Andrea Martinelli, Tyler Summers, John Lygeros

Abstract: Despite their success, policy gradient methods suffer from high variance of the gradient estimate, which can result in unsatisfactory sample complexity. Recently, numerous variance-reduced extensions of policy gradient methods with provably better sample complexity and competitive numerical performance have been proposed. After a compact survey on some of the main variance-reduced REINFORCE-type m… ▽ More Despite their success, policy gradient methods suffer from high variance of the gradient estimate, which can result in unsatisfactory sample complexity. Recently, numerous variance-reduced extensions of policy gradient methods with provably better sample complexity and competitive numerical performance have been proposed. After a compact survey on some of the main variance-reduced REINFORCE-type methods, we propose ProbAbilistic Gradient Estimation for Policy Gradient (PAGE-PG), a novel loopless variance-reduced policy gradient method based on a probabilistic switch between two types of updates. Our method is inspired by the PAGE estimator for supervised learning and leverages importance sampling to obtain an unbiased gradient estimator. We show that PAGE-PG enjoys a $\mathcal{O}\left( ε^{-3} \right)$ average sample complexity to reach an $ε$-stationary solution, which matches the sample complexity of its most competitive counterparts under the same setting. A numerical evaluation confirms the competitive performance of our method on classical control tasks. △ Less

Submitted 1 February, 2022; originally announced February 2022.

Stochastic convex optimization for provably efficient apprenticeship learning

Authors: Angeliki Kamoutsi, Goran Banjac, John Lygeros

Abstract: We consider large-scale Markov decision processes (MDPs) with an unknown cost function and employ stochastic convex optimization tools to address the problem of imitation learning, which consists of learning a policy from a finite set of expert demonstrations. We adopt the apprenticeship learning formalism, which carries the assumption that the true cost function can be represented as a linear c… ▽ More We consider large-scale Markov decision processes (MDPs) with an unknown cost function and employ stochastic convex optimization tools to address the problem of imitation learning, which consists of learning a policy from a finite set of expert demonstrations. We adopt the apprenticeship learning formalism, which carries the assumption that the true cost function can be represented as a linear combination of some known features. Existing inverse reinforcement learning algorithms come with strong theoretical guarantees, but are computationally expensive because they use reinforcement learning or planning algorithms as a subroutine. On the other hand, state-of-the-art policy gradient based algorithms (like IM-REINFORCE, IM-TRPO, and GAIL), achieve significant empirical success in challenging benchmark tasks, but are not well understood in terms of theory. With an emphasis on non-asymptotic guarantees of performance, we propose a method that directly learns a policy from expert demonstrations, bypassing the intermediate step of learning the cost function, by formulating the problem as a single convex optimization problem over occupancy measures. We develop a computationally efficient algorithm and derive high confidence regret bounds on the quality of the extracted policy, utilizing results from stochastic convex optimization and recent works in approximate linear programming for solving forward MDPs. △ Less

Submitted 31 December, 2021; originally announced January 2022.

Comments: arXiv admin note: text overlap with arXiv:2112.14004

Journal ref: Optimization Foundations for Reinforcement Learning Workshop at NeurIPS 2019, Vancouver, Canada

Efficient Performance Bounds for Primal-Dual Reinforcement Learning from Demonstrations

Authors: Angeliki Kamoutsi, Goran Banjac, John Lygeros

Abstract: We consider large-scale Markov decision processes with an unknown cost function and address the problem of learning a policy from a finite set of expert demonstrations. We assume that the learner is not allowed to interact with the expert and has no access to reinforcement signal of any kind. Existing inverse reinforcement learning methods come with strong theoretical guarantees, but are computati… ▽ More We consider large-scale Markov decision processes with an unknown cost function and address the problem of learning a policy from a finite set of expert demonstrations. We assume that the learner is not allowed to interact with the expert and has no access to reinforcement signal of any kind. Existing inverse reinforcement learning methods come with strong theoretical guarantees, but are computationally expensive, while state-of-the-art policy optimization algorithms achieve significant empirical success, but are hampered by limited theoretical understanding. To bridge the gap between theory and practice, we introduce a novel bilinear saddle-point framework using Lagrangian duality. The proposed primal-dual viewpoint allows us to develop a model-free provably efficient algorithm through the lens of stochastic convex optimization. The method enjoys the advantages of simplicity of implementation, low memory requirements, and computational and sample complexities independent of the number of states. We further present an equivalent no-regret online-learning interpretation. △ Less

Submitted 28 December, 2021; originally announced December 2021.

Journal ref: International Conference of Machine Learning (ICML) 2021

Learning-Based Repetitive Precision Motion Control with Mismatch Compensation

Authors: Efe C. Balta, Kira Barton, Dawn M. Tilbury, Alisa Rupenyan, John Lygeros

Abstract: Learning-based control methods utilize run-time data from the underlying process to improve the controller performance under model mismatch and unmodeled disturbances. This is beneficial for optimizing industrial processes, where the dynamics are difficult to model, and the repetitive nature of the process can be exploited. In this work, we develop an iterative approach for repetitive precision mo… ▽ More Learning-based control methods utilize run-time data from the underlying process to improve the controller performance under model mismatch and unmodeled disturbances. This is beneficial for optimizing industrial processes, where the dynamics are difficult to model, and the repetitive nature of the process can be exploited. In this work, we develop an iterative approach for repetitive precision motion control problems where the objective is to follow a reference geometry with minimal tracking error. Our method utilizes a nominal model of the process and learns the mismatch using Gaussian Process Regression (GPR). The control input and the GPR data are updated after each iteration to improve the performance in a run-to-run fashion. We provide a preliminary convergence analysis, implementation details of the proposed controller for minimizing different error types, and a case study where we demonstrate improved tracking performance with simulation and experimental results. △ Less

Submitted 19 November, 2021; originally announced November 2021.

Batch Model Predictive Control for Selective Laser Melting

Authors: Riccardo Zuliani, Efe C. Balta, Alisa Rupenyan, John Lygeros

Abstract: Selective laser melting is a promising additive manufacturing technology enabling the fabrication of highly customizable products. A major challenge in selective laser melting is ensuring the quality of produced parts, which is influenced greatly by the thermal history of printed layers. We propose a Batch-Model Predictive Control technique based on the combination of model predictive control and… ▽ More Selective laser melting is a promising additive manufacturing technology enabling the fabrication of highly customizable products. A major challenge in selective laser melting is ensuring the quality of produced parts, which is influenced greatly by the thermal history of printed layers. We propose a Batch-Model Predictive Control technique based on the combination of model predictive control and iterative learning control. This approach succeeds in rejecting both repetitive and non-repetitive disturbances and thus achieves improved tracking performance and process quality. In a simulation study, the selective laser melting dynamics is approximated with a reduced-order control-oriented linear model to ensure reasonable computational complexity. The proposed approach provides convergence to the desired temperature field profile despite model uncertainty and disturbances. △ Less

Submitted 8 April, 2022; v1 submitted 16 November, 2021; originally announced November 2021.

Incentive-Based Electric Vehicle Charging for Managing Bottleneck Congestion

Authors: Carlo Cenedese, Patrick Stokkink, Nikolas Gerolimins, John Lygeros

Abstract: We propose an incentive-based traffic demand management policy to alleviate traffic congestion on a road stretch that creates a bottleneck for the commuters. The incentive targets electric vehicles owners by proposing a discount on the energy price they use to charge their vehicles if they are flexible in their departure time. We show that, with a sufficient monetary budget, it is possible to comp… ▽ More We propose an incentive-based traffic demand management policy to alleviate traffic congestion on a road stretch that creates a bottleneck for the commuters. The incentive targets electric vehicles owners by proposing a discount on the energy price they use to charge their vehicles if they are flexible in their departure time. We show that, with a sufficient monetary budget, it is possible to completely eliminate the traffic congestion and we compute the optimal discount. We analyse also the case of limited budget, when the congestion cannot be completely eliminated. We compute analytically the policy minimising the congestion and estimate the level of inefficiency for different budgets. We corroborate our theoretical findings with numerical simulations that allow us to highlight the power of the proposed method in providing practical advice for the design of policies. △ Less

Submitted 10 November, 2021; originally announced November 2021.

In-layer Thermal Control of a Multi-layer Selective Laser Melting Process

Authors: Dominic Liao-McPherson, Efe C. Balta, Ryan Wüest, Alisa Rupenyan, John Lygeros

Abstract: Selective Laser Melting (SLM) is an additive manufacturing technology that builds three dimensional parts by melting layers of metal powder together with a laser that traces out a desired geometry. SLM is popular in industry, however the inherent melting and re-solidification of the metal during the process can, if left uncontrolled, cause excessive residual stress, porosity, and other defects in… ▽ More Selective Laser Melting (SLM) is an additive manufacturing technology that builds three dimensional parts by melting layers of metal powder together with a laser that traces out a desired geometry. SLM is popular in industry, however the inherent melting and re-solidification of the metal during the process can, if left uncontrolled, cause excessive residual stress, porosity, and other defects in the final printed parts. This paper presents a control-oriented thermal model of a multi-layer SLM process and proposes a structured model reduction methodology with an associated reduced order model based in-layer controller to track temperature references. Simulation studies demonstrate that the controller is able to prevent layer-to-layer heat buildup and that good closed-loop performance is possible using relatively low-order models. △ Less

Submitted 1 April, 2022; v1 submitted 1 November, 2021; originally announced November 2021.

Physics-informed linear regression is competitive with two Machine Learning methods in residential building MPC

Authors: Felix Bünning, Benjamin Huber, Adrian Schalbetter, Ahmed Aboudonia, Mathias Hudoba de Badyn, Philipp Heer, Roy S. Smith, John Lygeros

Abstract: Because physics-based building models are difficult to obtain as each building is individual, there is an increasing interest in generating models suitable for building MPC directly from measurement data. Machine learning methods have been widely applied to this problem and validated mostly in simulation; there are, however, few studies on a direct comparison of different models or validation in r… ▽ More Because physics-based building models are difficult to obtain as each building is individual, there is an increasing interest in generating models suitable for building MPC directly from measurement data. Machine learning methods have been widely applied to this problem and validated mostly in simulation; there are, however, few studies on a direct comparison of different models or validation in real buildings to be found in the literature. Methods that are indeed validated in application often lead to computationally complex non-convex optimization problems. Here we compare physics-informed Autoregressive-Moving-Average with Exogenous Inputs (ARMAX) models to Machine Learning models based on Random Forests and Input Convex Neural Networks and the resulting convex MPC schemes in experiments on a practical building application with the goal of minimizing energy consumption while maintaining occupant comfort, and in a numerical case study. We demonstrate that Predictive Control in general leads to savings between 26% and 49% of heating and cooling energy, compared to the building's baseline hysteresis controller. Moreover, we show that all model types lead to satisfactory control performance in terms of constraint satisfaction and energy reduction. However, we also see that the physics-informed ARMAX models have a lower computational burden, and a superior sample efficiency compared to the Machine Learning based models. Moreover, even if abundant training data is available, the ARMAX models have a significantly lower prediction error than the Machine Learning models, which indicates that the encoded physics-based prior of the former cannot independently be found by the latter. △ Less

Submitted 26 January, 2022; v1 submitted 29 October, 2021; originally announced October 2021.

Comments: 17 pages, 11 Figures, submitted to Applied Energy

Journal ref: Applied Energy 310 (2020) 118491

Parallel and Flexible Dynamic Programming via the Randomized Mini-Batch Operator

Authors: Matilde Gargiani, Andrea Martinelli, Max Ruts Martinez, John Lygeros

Abstract: The Bellman operator constitutes the foundation of dynamic programming (DP). An alternative is presented by the Gauss-Seidel operator, whose evaluation, differently from that of the Bellman operator where the states are all processed at once, updates one state at a time, while incorporating into the computation the interim results. The provably better convergence rate of DP methods based on the Ga… ▽ More The Bellman operator constitutes the foundation of dynamic programming (DP). An alternative is presented by the Gauss-Seidel operator, whose evaluation, differently from that of the Bellman operator where the states are all processed at once, updates one state at a time, while incorporating into the computation the interim results. The provably better convergence rate of DP methods based on the Gauss-Seidel operator comes at the price of an inherent sequentiality, which prevents the exploitation of modern multi-core systems. In this work we propose a new operator for dynamic programming, namely, the randomized mini-batch operator, which aims at realizing the trade-off between the better convergence rate of the methods based on the Gauss-Seidel operator and the parallelization capability offered by the Bellman operator. After the introduction of the new operator, a theoretical analysis for validating its fundamental properties is conducted. Such properties allow one to successfully deploy the new operator in the main dynamic programming schemes, such as value iteration and modified policy iteration. We compare the convergence of the DP algorithm based on the new operator with its earlier counterparts, shedding light on the algorithmic advantages of the new formulation and the impact of the batch-size parameter on the convergence. Finally, an extensive numerical evaluation of the newly introduced operator is conducted. In accordance with the theoretical derivations, the numerical results show the competitive performance of the proposed operator and its superior flexibility, which allows one to adapt the efficiency of its iterations to different structures of MDPs and hardware setups. △ Less

Submitted 6 October, 2021; originally announced October 2021.

Distributed Model Predictive Control of Buildings and Energy Hubs

Authors: Nicolas Lefebure, Mohammad Khosravi, Mathias Hudoba de Badyn, Felix Bünning, John Lygeros, Colin Jones, Roy S. Smith

Abstract: Model predictive control (MPC) strategies can be applied to the coordination of energy hubs to reduce their energy consumption. Despite the effectiveness of these techniques, their potential for energy savings are potentially underutilized due to the fact that energy demands are often assumed to be fixed quantities rather than controlled dynamic variables. The joint optimization of energy hubs and… ▽ More Model predictive control (MPC) strategies can be applied to the coordination of energy hubs to reduce their energy consumption. Despite the effectiveness of these techniques, their potential for energy savings are potentially underutilized due to the fact that energy demands are often assumed to be fixed quantities rather than controlled dynamic variables. The joint optimization of energy hubs and buildings' energy management systems can result in higher energy savings. This paper investigates how different MPC strategies perform on energy management systems in buildings and energy hubs. We first discuss two MPC approaches; centralized and decentralized. While the centralized control strategy offers optimal performance, its implementation is computationally prohibitive and raises privacy concerns. On the other hand, the decentralized control approach, which offers ease of implementation, displays significantly lower performance. We propose a third strategy, distributed control based on dual decomposition, which has the advantages of both approaches. Numerical case studies and comparisons demonstrate that the performance of distributed control is close to the performance of the centralized case, while maintaining a significantly lower computational burden, especially in large-scale scenarios with many agents. Finally, we validate and verify the reliability of the proposed method through an experiment on a full-scale energy hub system in the NEST demonstrator in Dübendorf, Switzerland. △ Less

Submitted 4 October, 2021; originally announced October 2021.

Comments: 16 pages, 8 figures

On the Synthesis of Bellman Inequalities for Data-Driven Optimal Control

Authors: Andrea Martinelli, Matilde Gargiani, John Lygeros

Abstract: In the context of the linear programming (LP) approach to data-driven control, one assumes that the dynamical system is unknown but can be observed indirectly through data on its evolution. Both theoretical and empirical evidence suggest that a desired suboptimality gap is often only achieved with massive exploration of the state-space. In case of linear systems, we discuss how a relatively small… ▽ More In the context of the linear programming (LP) approach to data-driven control, one assumes that the dynamical system is unknown but can be observed indirectly through data on its evolution. Both theoretical and empirical evidence suggest that a desired suboptimality gap is often only achieved with massive exploration of the state-space. In case of linear systems, we discuss how a relatively small but sufficiently rich dataset can be exploited to generate new constraints offline and without observing the corresponding transitions. Moreover, we show how to reconstruct the associated unknown stage-costs and, when the system is stochastic, we offer insights on the related problem of estimating the expected value in the Bellman operator without re-initializing the dynamics in the same state-input pairs. △ Less

Submitted 27 September, 2021; originally announced September 2021.

A distributed framework for linear adaptive MPC

Authors: Anilkumar Parsi, Ahmed Aboudonia, Andrea Iannelli, John Lygeros, Roy S. Smith

Abstract: Adaptive model predictive control (MPC) robustly ensures safety while reducing uncertainty during operation. In this paper, a distributed version is proposed to deal with network systems featuring multiple agents and limited communication. To solve the problem in a distributed manner, structure is imposed on the control design ingredients without sacrificing performance. Decentralized and distribu… ▽ More Adaptive model predictive control (MPC) robustly ensures safety while reducing uncertainty during operation. In this paper, a distributed version is proposed to deal with network systems featuring multiple agents and limited communication. To solve the problem in a distributed manner, structure is imposed on the control design ingredients without sacrificing performance. Decentralized and distributed adaptation schemes that allow for a reduction of the uncertainty online compatibly with the network topology are also proposed. The algorithm ensures robust constraint satisfaction, recursive feasibility and finite gain $\ell_2$ stability, and yields lower closed-loop cost compared to robust distributed MPC in simulations. △ Less

Submitted 16 April, 2024; v1 submitted 13 September, 2021; originally announced September 2021.

Comments: This work has been accepted to the 60th IEEE Conference on Decision and Control, 2021

Journal ref: Proc. 60th IEEE Conference on Decision and Control, Austin, TX, USA, 2021, pp. 460-465

Discrete-Time Linear-Quadratic Regulation via Optimal Transport

Authors: Mathias Hudoba de Badyn, Erik Miehling, Dylan Janak, Behçet Açıkmeşe, Mehran Mesbahi, Tamer Başar, John Lygeros, Roy S. Smith

Abstract: In this paper, we consider a discrete-time stochastic control problem with uncertain initial and target states. We first discuss the connection between optimal transport and stochastic control problems of this form. Next, we formulate a linear-quadratic regulator problem where the initial and terminal states are distributed according to specified probability densities. A closed-form solution for t… ▽ More In this paper, we consider a discrete-time stochastic control problem with uncertain initial and target states. We first discuss the connection between optimal transport and stochastic control problems of this form. Next, we formulate a linear-quadratic regulator problem where the initial and terminal states are distributed according to specified probability densities. A closed-form solution for the optimal transport map in the case of linear-time varying systems is derived, along with an algorithm for computing the optimal map. Two numerical examples pertaining to swarm deployment demonstrate the practical applicability of the model, and performance of the numerical method. △ Less

Submitted 6 September, 2021; originally announced September 2021.

Comments: 8 pages, 6 figures. To be included in the Proceedings of the 60th Conference on Decision and Control. This version includes proofs

Passivity-based Decentralized Control for Discrete-time Large-scale Systems

Authors: Ahmed Aboudonia, Andrea Martinelli, John Lygeros

Abstract: Passivity theory has recently contributed to develo** decentralized control schemes for large-scale systems. Many decentralized passivity-based control schemes are designed in continuous-time. It is well-known, however, that the passivity properties of continuous-time systems may be lost under discretization. In this work, we present a novel stabilizing decentralized control scheme by ensuring p… ▽ More Passivity theory has recently contributed to develo** decentralized control schemes for large-scale systems. Many decentralized passivity-based control schemes are designed in continuous-time. It is well-known, however, that the passivity properties of continuous-time systems may be lost under discretization. In this work, we present a novel stabilizing decentralized control scheme by ensuring passivity for discrete-time systems directly and thus avoiding the issue of passivity preservation. The controller is synthesized by locally solving a semidefinite program offline for each subsystem in a decentralized fashion. This program comprises local conditions ensuring that the corresponding subsystem is locally passive. Passivity is ensured with respect to a local virtual output which is different from the local actual output. The program also comprises local conditions ensuring that the local passivity of all subsystems implies the asymptotic stability of the whole system. The performance of the proposed controller is evaluated on a case study in DC microgrids. △ Less

Submitted 15 July, 2021; originally announced July 2021.

Decentralized trajectory optimization for multi-agent exploration

Authors: Dimitris Gkouletsos, Andrea Iannelli, Mathias Hudoba de Badyn, John Lygeros

Abstract: Autonomous exploration is an application of growing importance in robotics. A promising strategy is ergodic trajectory planning, whereby an agent spends in each area a fraction of time which is proportional to its probability information density function. In this paper, a decentralized ergodic multi-agent trajectory planning algorithm featuring limited communication constraints is proposed. The ag… ▽ More Autonomous exploration is an application of growing importance in robotics. A promising strategy is ergodic trajectory planning, whereby an agent spends in each area a fraction of time which is proportional to its probability information density function. In this paper, a decentralized ergodic multi-agent trajectory planning algorithm featuring limited communication constraints is proposed. The agents' trajectories are designed by optimizing a weighted cost encompassing ergodicity, control energy and close-distance operation objectives. To solve the underlying optimal control problem, a second-order descent iterative method coupled with a projection operator in the form of an optimal feedback controller is used. Exhaustive numerical analyses show that the multi-agent solution allows a much more efficient exploration in terms of completion task time and control energy distribution by leveraging collaboration among agents. △ Less

Submitted 4 July, 2021; originally announced July 2021.

Comments: 8 pages, 9 figures. Accepted to the Robotics and Automation Letters and the 2021 International Conference on Intelligent Robots and Systems

Learning $Q$-function approximations for hybrid control problems

Authors: Sandeep Menta, Joseph Warrington, John Lygeros, Manfred Morari

Abstract: The main challenge in controlling hybrid systems arises from having to consider an exponential number of sequences of future modes to make good long-term decisions. Model predictive control (MPC) computes a control action through a finite-horizon optimisation problem. A key ingredient in this problem is a terminal cost, to account for the system's evolution beyond the chosen horizon. A good termin… ▽ More The main challenge in controlling hybrid systems arises from having to consider an exponential number of sequences of future modes to make good long-term decisions. Model predictive control (MPC) computes a control action through a finite-horizon optimisation problem. A key ingredient in this problem is a terminal cost, to account for the system's evolution beyond the chosen horizon. A good terminal cost can reduce the horizon length required for good control action and is often tuned empirically by observing performance. We build on the idea of using $N$-step $Q$-functions $(\mathcal{Q}^{(N)})$ in the MPC objective to avoid having to choose a terminal cost. We present a formulation incorporating the system dynamics and constraints to approximate the optimal $\mathcal{Q}^{(N)}$-function and algorithms to train the approximation parameters through an exploration of the state space. We test the control policy derived from the trained approximations on two benchmark problems through simulations and observe that our algorithms are able to learn good $\mathcal{Q}^{(N)}$-approximations for high dimensional hybrid systems based on a relatively small dataset. Finally, we compare our controller's performance against that of Hybrid MPC in terms of computation time and closed-loop cost. △ Less

Submitted 7 June, 2021; v1 submitted 27 May, 2021; originally announced May 2021.

Comments: 8 pages, 4 figures, 1 algorithm, 2 tables, 1 appendix with 5 matrices

MSC Class: 90C39; 49L20 ACM Class: I.2.8