-
Sensitivity Analysis for Piecewise-Affine Approximations of Nonlinear Programs with Polytopic Constraints
Authors:
Leila Gharavi,
Changrui Liu,
Bart De Schutter,
Simone Baldi
Abstract:
Nonlinear Programs (NLPs) are prevalent in optimization-based control of nonlinear systems. Solving general NLPs is computationally expensive, necessitating the development of fast hardware or tractable suboptimal approximations. This paper investigates the sensitivity of the solutions of NLPs with polytopic constraints when the nonlinear continuous objective function is approximated by a PieceWis…
▽ More
Nonlinear Programs (NLPs) are prevalent in optimization-based control of nonlinear systems. Solving general NLPs is computationally expensive, necessitating the development of fast hardware or tractable suboptimal approximations. This paper investigates the sensitivity of the solutions of NLPs with polytopic constraints when the nonlinear continuous objective function is approximated by a PieceWise-Affine (PWA) counterpart. By leveraging perturbation analysis using a convex modulus, we derive guaranteed bounds on the distance between the optimal solution of the original polytopically-constrained NLP and that of its approximated formulation. Our approach aids in determining criteria for achieving desired solution bounds. Two case studies on the Eggholder function and nonlinear model predictive control of an inverted pendulum demonstrate the theoretical results.
△ Less
Submitted 30 May, 2024;
originally announced May 2024.
-
Distributed MPC for PWA Systems Based on Switching ADMM
Authors:
Samuel Mallick,
Azita Dabiri,
Bart De Schutter
Abstract:
This paper presents a novel approach for distributed model predictive control (MPC) for piecewise affine (PWA) systems. Existing approaches rely on solving mixed-integer optimization problems, requiring significant computation power or time. We propose a distributed MPC scheme that requires solving only convex optimization problems. The key contribution is a novel method, based on the alternating…
▽ More
This paper presents a novel approach for distributed model predictive control (MPC) for piecewise affine (PWA) systems. Existing approaches rely on solving mixed-integer optimization problems, requiring significant computation power or time. We propose a distributed MPC scheme that requires solving only convex optimization problems. The key contribution is a novel method, based on the alternating direction method of multipliers, for solving the non-convex optimal control problem that arises due to the PWA dynamics. We present a distributed MPC scheme, leveraging this method, that explicitly accounts for the coupling between subsystems by reaching agreement on the values of coupled states. Stability and recursive feasibility are shown under additional assumptions on the underlying system. Two numerical examples are provided, in which the proposed controller is shown to significantly improve the CPU time and closed-loop performance over existing state-of-the-art approaches.
△ Less
Submitted 25 April, 2024;
originally announced April 2024.
-
Scenario Reduction with Guarantees for Stochastic Optimal Control of Linear Systems
Authors:
Francesco Cordiano,
Bart De Schutter
Abstract:
Scenario reduction algorithms can be an effective means to provide a tractable description of the uncertainty in optimal control problems. However, they might significantly compromise the performance of the controlled system. In this paper, we propose a method to compensate for the effect of scenario reduction on stochastic optimal control problems for chance-constrained linear systems with additi…
▽ More
Scenario reduction algorithms can be an effective means to provide a tractable description of the uncertainty in optimal control problems. However, they might significantly compromise the performance of the controlled system. In this paper, we propose a method to compensate for the effect of scenario reduction on stochastic optimal control problems for chance-constrained linear systems with additive uncertainty. We consider a setting in which the uncertainty has a discrete distribution, where the number of possible realizations is large. We then propose a reduction algorithm with a problem-dependent loss function, and we define sufficient conditions on the stochastic optimal control problem to ensure out-of-sample guarantees (i.e., against the original distribution of the uncertainty) for the controlled system in terms of performance and chance constraint satisfaction. Finally, we demonstrate the effectiveness of the approach on a numerical example.
△ Less
Submitted 11 April, 2024;
originally announced April 2024.
-
A Benchmark for the Application of Distributed Control Techniques to the Electricity Network of the European Economic Area
Authors:
A. Riccardi,
L. Laurenti,
B. De Schutter
Abstract:
The European Economic Area Electricity Network Benchmark (EEA-ENB) is a multi-area power system representing the European network of transmission systems for electricity to facilitate the application of distributed control techniques. In the EEA-ENB we consider the Load Frequency Control (LFC) problem in the presence of renewable energy sources (RESs), and energy storage systems (ESSs). RESs are k…
▽ More
The European Economic Area Electricity Network Benchmark (EEA-ENB) is a multi-area power system representing the European network of transmission systems for electricity to facilitate the application of distributed control techniques. In the EEA-ENB we consider the Load Frequency Control (LFC) problem in the presence of renewable energy sources (RESs), and energy storage systems (ESSs). RESs are known to cause instability in power networks due to their inertia-less and intermittent characteristics, while ESSs are introduced as a resource to mitigate the problem. In the EEA-ENB, particular attention is dedicated to Distributed Model Predictive Control (DMPC), whose application is often limited to small and homogeneous test cases due to the lack of standardized large-scale scenarios for testing, and due to the large computation time required to obtain a centralized MPC action for performance comparison with DMPC strategies under consideration. The second problem is exacerbated when the scale of the system grows. To address these challenges and to provide a real-world-based and control-independent benchmark, the EEA-ENB has been developed. The benchmark includes a centralized MPC strategy providing performance and computation time metrics to compare distributed control within a repeatable and realistic simulation environment.
△ Less
Submitted 26 May, 2024; v1 submitted 21 March, 2024;
originally announced March 2024.
-
A Comparison Benchmark for Distributed Hybrid MPC Control Methods: Distributed Vehicle Platooning
Authors:
Samuel Mallick,
Azita Dabiri,
Bart De Schutter
Abstract:
Distributed model predictive control (MPC) is currently being investigated as a solution to the important control challenge presented by networks of hybrid dynamical systems. However, a benchmark problem for distributed hybrid MPC is absent from the literature. We propose distributed control of a platoon of autonomous vehicles as a comparison benchmark problem. The problem provides a complex and a…
▽ More
Distributed model predictive control (MPC) is currently being investigated as a solution to the important control challenge presented by networks of hybrid dynamical systems. However, a benchmark problem for distributed hybrid MPC is absent from the literature. We propose distributed control of a platoon of autonomous vehicles as a comparison benchmark problem. The problem provides a complex and adaptable case study, upon which existing and future approaches to distributed MPC for hybrid systems can be evaluated. Two hybrid modeling frameworks are presented for the vehicle dynamics. Five hybrid MPC controllers are then evaluated and extensively assessed on the fleet of vehicles. Finally, we comment on the need for new efficient and high performing distributed MPC schemes for hybrid systems.
△ Less
Submitted 18 June, 2024; v1 submitted 18 January, 2024;
originally announced January 2024.
-
State-action control barrier functions: Imposing safety on learning-based control with low online computational costs
Authors:
Kanghui He,
Shengling Shi,
Ton van den Boom,
Bart De Schutter
Abstract:
Learning-based control with safety guarantees usually requires real-time safety certification and modifications of possibly unsafe learning-based policies. The control barrier function (CBF) method uses a safety filter containing a constrained optimization problem to produce safe policies. However, finding a valid CBF for a general nonlinear system requires a complex function parameterization, whi…
▽ More
Learning-based control with safety guarantees usually requires real-time safety certification and modifications of possibly unsafe learning-based policies. The control barrier function (CBF) method uses a safety filter containing a constrained optimization problem to produce safe policies. However, finding a valid CBF for a general nonlinear system requires a complex function parameterization, which in general, makes the policy optimization problem difficult to solve in real time. For nonlinear systems with nonlinear state constraints, this paper proposes the novel concept of state-action CBFs, which not only characterize the safety at each state but also evaluate the control inputs taken at each state. State-action CBFs, in contrast to CBFs, enable a flexible parameterization, resulting in a safety filter that involves a convex quadratic optimization problem. This, in turn, significantly alleviates the online computational burden. To synthesize state-action CBFs, we propose a learning-based approach exploiting Hamilton-Jacobi reachability. The effect of learning errors on the effectiveness of state-action CBFs is addressed by constraint tightening and introducing a new concept called contractive CBFs. These contributions ensure formal safety guarantees for learned CBFs and control policies, enhancing the applicability of learning-based control in real-time scenarios. Simulation results on an inverted pendulum with elastic walls validate the proposed CBFs in terms of constraint satisfaction and CPU time.
△ Less
Submitted 18 December, 2023;
originally announced December 2023.
-
Multi-Agent Reinforcement Learning via Distributed MPC as a Function Approximator
Authors:
Samuel Mallick,
Filippo Airaldi,
Azita Dabiri,
Bart De Schutter
Abstract:
This paper presents a novel approach to multi-agent reinforcement learning (RL) for linear systems with convex polytopic constraints. Existing work on RL has demonstrated the use of model predictive control (MPC) as a function approximator for the policy and value functions. The current paper is the first work to extend this idea to the multi-agent setting. We propose the use of a distributed MPC…
▽ More
This paper presents a novel approach to multi-agent reinforcement learning (RL) for linear systems with convex polytopic constraints. Existing work on RL has demonstrated the use of model predictive control (MPC) as a function approximator for the policy and value functions. The current paper is the first work to extend this idea to the multi-agent setting. We propose the use of a distributed MPC scheme as a function approximator, with a structure allowing for distributed learning and deployment. We then show that Q-learning updates can be performed distributively without introducing nonstationarity, by reconstructing a centralized learning update. The effectiveness of the approach is demonstrated on two numerical examples.
△ Less
Submitted 16 April, 2024; v1 submitted 8 December, 2023;
originally announced December 2023.
-
Reinforcement Learning with Model Predictive Control for Highway Ramp Metering
Authors:
Filippo Airaldi,
Bart De Schutter,
Azita Dabiri
Abstract:
In the backdrop of an increasingly pressing need for effective urban and highway transportation systems, this work explores the synergy between model-based and learning-based strategies to enhance traffic flow management by use of an innovative approach to the problem of ramp metering control that embeds Reinforcement Learning (RL) techniques within the Model Predictive Control (MPC) framework. Th…
▽ More
In the backdrop of an increasingly pressing need for effective urban and highway transportation systems, this work explores the synergy between model-based and learning-based strategies to enhance traffic flow management by use of an innovative approach to the problem of ramp metering control that embeds Reinforcement Learning (RL) techniques within the Model Predictive Control (MPC) framework. The control problem is formulated as an RL task by crafting a suitable stage cost function that is representative of the traffic conditions, variability in the control action, and violations of the constraint on the maximum number of vehicles in queue. An MPC-based RL approach, which leverages the MPC optimal problem as a function approximation for the RL algorithm, is proposed to learn to efficiently control an on-ramp and satisfy its constraints despite uncertainties in the system model and variable demands. Simulations are performed on a benchmark small-scale highway network to compare the proposed methodology against other state-of-the-art control approaches. Results show that, starting from an MPC controller that has an imprecise model and is poorly tuned, the proposed methodology is able to effectively learn to improve the control policy such that congestion in the network is reduced and constraints are satisfied, yielding an improved performance that is superior to the other controllers.
△ Less
Submitted 21 May, 2024; v1 submitted 15 November, 2023;
originally announced November 2023.
-
Regret Analysis of Learning-Based Linear Quadratic Gaussian Control with Additive Exploration
Authors:
Archith Athrey,
Othmane Mazhar,
Meichen Guo,
Bart De Schutter,
Shengling Shi
Abstract:
In this paper, we analyze the regret incurred by a computationally efficient exploration strategy, known as naive exploration, for controlling unknown partially observable systems within the Linear Quadratic Gaussian (LQG) framework. We introduce a two-phase control algorithm called LQG-NAIVE, which involves an initial phase of injecting Gaussian input signals to obtain a system model, followed by…
▽ More
In this paper, we analyze the regret incurred by a computationally efficient exploration strategy, known as naive exploration, for controlling unknown partially observable systems within the Linear Quadratic Gaussian (LQG) framework. We introduce a two-phase control algorithm called LQG-NAIVE, which involves an initial phase of injecting Gaussian input signals to obtain a system model, followed by a second phase of an interplay between naive exploration and control in an episodic fashion. We show that LQG-NAIVE achieves a regret growth rate of $\tilde{\mathcal{O}}(\sqrt{T})$, i.e., $\mathcal{O}(\sqrt{T})$ up to logarithmic factors after $T$ time steps, and we validate its performance through numerical simulations. Additionally, we propose LQG-IF2E, which extends the exploration signal to a `closed-loop' setting by incorporating the Fisher Information Matrix (FIM). We provide compelling numerical evidence of the competitive performance of LQG-IF2E compared to LQG-NAIVE.
△ Less
Submitted 24 November, 2023; v1 submitted 5 November, 2023;
originally announced November 2023.
-
Proactive Emergency Collision Avoidance for Automated Driving in Highway Scenarios
Authors:
Leila Gharavi,
Azita Dabiri,
Jelske Verkuijlen,
Bart De Schutter,
Simone Baldi
Abstract:
Uncertainty in the behavior of other traffic participants is a crucial factor in collision avoidance for automated driving; here, stochastic metrics should often be considered to avoid overly conservative decisions. This paper introduces a Stochastic Model Predictive Control (SMPC) planner for emergency collision avoidance in highway scenarios to proactively minimize collision risk while ensuring…
▽ More
Uncertainty in the behavior of other traffic participants is a crucial factor in collision avoidance for automated driving; here, stochastic metrics should often be considered to avoid overly conservative decisions. This paper introduces a Stochastic Model Predictive Control (SMPC) planner for emergency collision avoidance in highway scenarios to proactively minimize collision risk while ensuring safety through chance constraints. To address the challenge of guaranteeing the feasibility for the emergency trajectory, we incorporate nonlinear tire dynamics in the prediction model of the ego vehicle. Further, we exploit Max-Min-Plus-Scaling (MMPS) approximations of the nonlinearities to avoid conservatism, enforce proactive collision avoidance, and improve computational efficiency in terms of performance and speed. Consequently, our contributions include integrating a dynamic ego vehicle model into the SMPC planner, introducing the MMPS approximation for real-time implementation in emergency scenarios, and integrating SMPC with hybridized chance constraints and risk minimization. We evaluate our SMPC formulation in terms of proactivity and efficiency in various hazardous scenarios. Moreover, we demonstrate the effectiveness of our proposed approach by comparing it with a state-of-the-art SMPC planner and validate the feasibility of generated trajectories using a high-fidelity vehicle model in IPG CarMaker.
△ Less
Submitted 26 October, 2023;
originally announced October 2023.
-
A Behavioral Perspective on Models of Linear Dynamical Networks with Manifest Variables
Authors:
Shengling Shi,
Zhiyong Sun,
Bart De Schutter
Abstract:
Networks of dynamical systems play an important role in various domains and have motivated many studies on the control and analysis of linear dynamical networks. For linear network models considered in these studies, it is typically pre-determined what signal channels are inputs and what are outputs. These models do not capture the practical need to incorporate different experimental situations, w…
▽ More
Networks of dynamical systems play an important role in various domains and have motivated many studies on the control and analysis of linear dynamical networks. For linear network models considered in these studies, it is typically pre-determined what signal channels are inputs and what are outputs. These models do not capture the practical need to incorporate different experimental situations, where different selections of input and output channels are applied to the same network. Moreover, a unified view of different network models is lacking. This work makes an initial step towards addressing the above issues by taking a behavioral perspective, where input and output channels are not pre-determined. The focus of this work is on behavioral network models with only external variables. By exploiting the concept of hypergraphs, novel dual graphical representations, called system graphs and signal graphs, are introduced for behavioral networks. Moreover, connections between behavioral network models and structural vector autoregressive models are established. In addition to their connections in graphical representations, it is shown that the regularity of interconnections is an essential assumption when choosing a structural vector autoregressive model.
△ Less
Submitted 5 May, 2024; v1 submitted 24 October, 2023;
originally announced October 2023.
-
Efficient MPC for Emergency Evasive Maneuvers, Part II: Comparative Assessment for Hybrid Control
Authors:
Leila Gharavi,
Bart De Schutter,
Simone Baldi
Abstract:
Optimization-based approaches such as Model Predictive Control (MPC) are promising approaches in proactive control for safety-critical applications with changing environments such as automated driving systems. However, the computational complexity of the MPC optimization problem coupled with the need for real-time control in hazardous scenarios is the main bottleneck in realization of automation l…
▽ More
Optimization-based approaches such as Model Predictive Control (MPC) are promising approaches in proactive control for safety-critical applications with changing environments such as automated driving systems. However, the computational complexity of the MPC optimization problem coupled with the need for real-time control in hazardous scenarios is the main bottleneck in realization of automation levels four and five for driving systems. In this paper, we construct hybrid formulations of the nonlinear MPC problem for tracking control during emergency evasive maneuvers and assess their computational efficiency in terms of accuracy and solution time. To hybridize the MPC problem, we combine three hybrid approximations of the prediction model and four approximations of the nonlinear stability and tire saturation constraints and simulate the closed-loop behavior of the resulting controllers during five emergency maneuvers for different prediction horizons. Further, we compare the robustness of the controllers in the presence of friction uncertainty as well to assess the accuracy-time trade-off in cases where the friction of the road is either unknown or has an offset error with respect to the prediction model. This robustness is studied for different levels of friction uncertainty, as well as investigated with respect to the proximity to the vehicle handling limits. We show that the hybridization of the MPC problem is an efficient approach for real-time implementation of MPC during emergency evasive maneuvers, paving the way for implementation of high levels of automation.
△ Less
Submitted 1 October, 2023;
originally announced October 2023.
-
Efficient MPC for Emergency Evasive Maneuvers, Part I: Hybridization of the Nonlinear Problem
Authors:
Leila Gharavi,
Bart De Schutter,
Simone Baldi
Abstract:
Despite the extensive application of nonlinear Model Predictive Control (MPC) in automated driving, balancing its computational efficiency with respect to the control performance and constraint satisfaction remains a challenge in emergency scenarios: in such situations, sub-optimal but computationally fast responses are more valuable than optimal responses obtained after long computations. In this…
▽ More
Despite the extensive application of nonlinear Model Predictive Control (MPC) in automated driving, balancing its computational efficiency with respect to the control performance and constraint satisfaction remains a challenge in emergency scenarios: in such situations, sub-optimal but computationally fast responses are more valuable than optimal responses obtained after long computations. In this paper, we introduce a hybridization approach for efficient approximation of nonlinear vehicle dynamics and non-convex constraints using a hybrid systems modeling framework. Hybridization allows to reformulate the nonlinear MPC problem during emergency evasive maneuvers as a hybrid MPC problem. In this regard, Max-Min-Plus-Scaling (MMPS) hybrid modeling is used to approximate the nonlinear vehicle dynamics. Meanwhile, different formulations for constraint approximation are presented, and various grid-generation methods are compared to solve these approximation problems. Among these, two novel grid types are introduced to structurally include the influence of the system dynamics on the grid point distributions in the state domain. Overall, the work presents and compares three hybrid models and four hybrid constraints for efficient MPC synthesis and offers guidelines for implementation of the presented hybridization framework in other applications.
△ Less
Submitted 1 October, 2023;
originally announced October 2023.
-
Approximate Dynamic Programming for Constrained Piecewise Affine Systems with Stability and Safety Guarantees
Authors:
Kanghui He,
Shengling Shi,
Ton van den Boom,
Bart De Schutter
Abstract:
Infinite-horizon optimal control of constrained piecewise affine (PWA) systems has been approximately addressed by hybrid model predictive control (MPC), which, however, has computational limitations, both in offline design and online implementation. In this paper, we consider an alternative approach based on approximate dynamic programming (ADP), an important class of methods in reinforcement lea…
▽ More
Infinite-horizon optimal control of constrained piecewise affine (PWA) systems has been approximately addressed by hybrid model predictive control (MPC), which, however, has computational limitations, both in offline design and online implementation. In this paper, we consider an alternative approach based on approximate dynamic programming (ADP), an important class of methods in reinforcement learning. We accommodate non-convex union-of-polyhedra state constraints and linear input constraints into ADP by designing PWA penalty functions. PWA function approximation is used, which allows for a mixed-integer encoding to implement ADP. The main advantage of the proposed ADP method is its online computational efficiency. Particularly, we propose two control policies, which lead to solving a smaller-scale mixed-integer linear program than conventional hybrid MPC, or a single convex quadratic program, depending on whether the policy is implicitly determined online or explicitly computed offline. We characterize the stability and safety properties of the closed-loop systems, as well as the sub-optimality of the proposed policies, by quantifying the approximation errors of value functions and policies. We also develop an offline mixed-integer linear programming-based method to certify the reliability of the proposed method. Simulation results on an inverted pendulum with elastic walls and on an adaptive cruise control problem validate the control performance in terms of constraint satisfaction and CPU time.
△ Less
Submitted 6 January, 2024; v1 submitted 27 June, 2023;
originally announced June 2023.
-
Suboptimality analysis of receding horizon quadratic control with unknown linear systems and its applications in learning-based control
Authors:
Shengling Shi,
Anastasios Tsiamis,
Bart De Schutter
Abstract:
In this work, we aim to analyze how the trade-off between the modeling error, the terminal value function error, and the prediction horizon affects the performance of a nominal receding-horizon linear quadratic (LQ) controller. By develo** a novel perturbation result of the Riccati difference equation, a novel performance upper bound is obtained and suggests that for many cases, the prediction h…
▽ More
In this work, we aim to analyze how the trade-off between the modeling error, the terminal value function error, and the prediction horizon affects the performance of a nominal receding-horizon linear quadratic (LQ) controller. By develo** a novel perturbation result of the Riccati difference equation, a novel performance upper bound is obtained and suggests that for many cases, the prediction horizon can be either one or infinity to improve the control performance, depending on the relative difference between the modeling error and the terminal value function error. The result also shows that when an infinite horizon is desired, a finite prediction horizon that is larger than the controllability index can be sufficient for achieving a near-optimal performance, revealing a close relation between the prediction horizon and controllability. The obtained suboptimality performance bound is also applied to provide novel sample complexity and regret guarantees for nominal receding-horizon LQ controllers in a learning-based setting.
△ Less
Submitted 8 April, 2024; v1 submitted 18 January, 2023;
originally announced January 2023.
-
Learning safety in model-based Reinforcement Learning using MPC and Gaussian Processes
Authors:
Filippo Airaldi,
Bart De Schutter,
Azita Dabiri
Abstract:
We propose a method to encourage safety in Model Predictive Control (MPC)-based Reinforcement Learning (RL) via Gaussian Process (GP) regression. This framework consists of 1) a parametric MPC scheme that is employed as model-based controller with approximate knowledge on the real system's dynamics, 2) an episodic RL algorithm tasked with adjusting the MPC parametrization in order to increase its…
▽ More
We propose a method to encourage safety in Model Predictive Control (MPC)-based Reinforcement Learning (RL) via Gaussian Process (GP) regression. This framework consists of 1) a parametric MPC scheme that is employed as model-based controller with approximate knowledge on the real system's dynamics, 2) an episodic RL algorithm tasked with adjusting the MPC parametrization in order to increase its performance, and lastly, 3) GP regressors used to estimate, directly from data, constraints on the MPC parameters capable of predicting, up to some probability, whether the parametrization is likely to yield a safe or unsafe policy. These constraints are then enforced onto the RL updates in an effort to enhance the learning method with a probabilistic safety mechanism. Compared to other recent publications combining safe RL with MPC, our method does not require further assumptions on, e.g., the prediction model in order to retain computational tractability. We illustrate the results of our method in a numerical example on the control of a quadrotor drone in a safety-critical environment.
△ Less
Submitted 17 March, 2023; v1 submitted 3 November, 2022;
originally announced November 2022.
-
Approximate Dynamic Programming for Constrained Linear Systems: A Piecewise Quadratic Approximation Approach
Authors:
Kanghui He,
Shengling Shi,
Ton van den Boom,
Bart De Schutter
Abstract:
Approximate dynamic programming (ADP) faces challenges in dealing with constraints in control problems. Model predictive control (MPC) is, in comparison, well-known for its accommodation of constraints and stability guarantees, although its computation is sometimes prohibitive. This paper introduces an approach combining the two methodologies to overcome their individual limitations. The predictiv…
▽ More
Approximate dynamic programming (ADP) faces challenges in dealing with constraints in control problems. Model predictive control (MPC) is, in comparison, well-known for its accommodation of constraints and stability guarantees, although its computation is sometimes prohibitive. This paper introduces an approach combining the two methodologies to overcome their individual limitations. The predictive control law for constrained linear quadratic regulation (CLQR) problems has been proven to be piecewise affine (PWA) while the value function is piecewise quadratic. We exploit these formal results from MPC to design an ADP method for CLQR problems. A novel convex and piecewise quadratic neural network with a local-global architecture is proposed to provide an accurate approximation of the value function, which is used as the cost-to-go function in the online dynamic programming problem. An efficient decomposition algorithm is developed to speed up the online computation. Rigorous stability analysis of the closed-loop system is conducted for the proposed control scheme under the condition that a good approximation of the value function is achieved. Comparative simulations are carried out to demonstrate the potential of the proposed method in terms of online computation and optimality.
△ Less
Submitted 6 April, 2023; v1 submitted 20 May, 2022;
originally announced May 2022.
-
Finite-sample analysis of identification of switched linear systems with arbitrary or restricted switching
Authors:
Shengling Shi,
Othmane Mazhar,
Bart De Schutter
Abstract:
For the identification of switched systems with a measured switching signal, this work aims to analyze the effect of switching strategies on the estimation error. The data for identification is assumed to be collected from globally asymptotically or marginally stable switched systems under switches that are arbitrary or subject to an average dwell time constraint. Then the switched system is estim…
▽ More
For the identification of switched systems with a measured switching signal, this work aims to analyze the effect of switching strategies on the estimation error. The data for identification is assumed to be collected from globally asymptotically or marginally stable switched systems under switches that are arbitrary or subject to an average dwell time constraint. Then the switched system is estimated by the least-squares (LS) estimator. To capture the effect of the parameters of the switching strategies on the LS estimation error, finite-sample error bounds are developed in this work. The obtained error bounds show that the estimation error is logarithmic of the switching parameters when there are only stable modes; however, when there are unstable modes, the estimation error bound can increase linearly as the switching parameter changes. This suggests that in the presence of unstable modes, the switching strategy should be properly designed to avoid the significant increase of the estimation error.
△ Less
Submitted 28 June, 2022; v1 submitted 18 March, 2022;
originally announced March 2022.
-
Max-algebraic hybrid automata: Modelling and equivalences
Authors:
A. Gupta,
B. De Schutter,
J. van der Woude,
T. van den Boom
Abstract:
This article introduces the novel framework of max-algebraic hybrid automata as a hybrid modelling language in the max-plus algebra. We show that the modelling framework unifies and extends the switching max-plus linear systems framework and is analogous to the discrete hybrid automata framework in conventional algebra. In addition, we show that the framework serves as a bridge between automata-th…
▽ More
This article introduces the novel framework of max-algebraic hybrid automata as a hybrid modelling language in the max-plus algebra. We show that the modelling framework unifies and extends the switching max-plus linear systems framework and is analogous to the discrete hybrid automata framework in conventional algebra. In addition, we show that the framework serves as a bridge between automata-theoretic models in max-plus algebra and switching max-plus linear systems. In doing so, we formalise the relationship between max-plus automata and switching max-plus linear systems in a behavioural sense. This also serves as another step towards importing tools for analysis and optimal control from conventional time-driven hybrid systems to discrete-event systems in max-plus algebra.
△ Less
Submitted 19 November, 2021;
originally announced November 2021.
-
Error-free approximation of explicit linear MPC through lattice piecewise affine expression
Authors:
Jun Xu,
Yunjiang Lou,
Bart De Schutter,
Zhenhua Xiong
Abstract:
In this paper, the disjunctive and conjunctive lattice piecewise affine (PWA) approximations of explicit linear model predictive control (MPC) are proposed. The training data are generated uniformly in the domain of interest, consisting of the state samples and corresponding affine control laws, based on which the lattice PWA approximations are constructed. Re-sampling of data is also proposed to…
▽ More
In this paper, the disjunctive and conjunctive lattice piecewise affine (PWA) approximations of explicit linear model predictive control (MPC) are proposed. The training data are generated uniformly in the domain of interest, consisting of the state samples and corresponding affine control laws, based on which the lattice PWA approximations are constructed. Re-sampling of data is also proposed to guarantee that the lattice PWA approximations are identical to explicit MPC control law in the unique order (UO) regions containing the sample points as interior points. Additionally, under mild assumptions, the equivalence of the two lattice PWA approximations guarantees that the approximations are error-free in the domain of interest. The algorithms for deriving statistically error-free approximation to the explicit linear MPC are proposed and the complexity of the entire procedure is analyzed, which is polynomial with respect to the number of samples. The performance of the proposed approximation strategy is tested through two simulation examples, and the result shows that with a moderate number of sample points, we can construct lattice PWA approximations that are equivalent to optimal control law of the explicit linear MPC.
△ Less
Submitted 20 July, 2022; v1 submitted 1 October, 2021;
originally announced October 2021.
-
A real-time distributed post-disaster restoration planning strategy for distribution networks
Authors:
Jianfeng Fu,
Alfredo Nunez,
Bart De Schutter
Abstract:
After disasters, distribution networks have to be restored by repair, reconfiguration, and power dispatch. During the restoration process, changes can occur in real time that deviate from the situations considered in pre-designed planning strategies. That may result in the pre-designed plan to become far from optimal or even unimplementable. This paper proposes a centralized-distributed bi-level o…
▽ More
After disasters, distribution networks have to be restored by repair, reconfiguration, and power dispatch. During the restoration process, changes can occur in real time that deviate from the situations considered in pre-designed planning strategies. That may result in the pre-designed plan to become far from optimal or even unimplementable. This paper proposes a centralized-distributed bi-level optimization method to solve the real-time restoration planning problem. The first level determines integer variables related to routing of the crews and the status of the switches using a genetic algorithm (GA), while the second level determines the dispatch of active/reactive power by using distributed model predictive control (DMPC). A novel Aitken- DMPC solver is proposed to accelerate convergence and to make the method suitable for real-time decision making. A case study based on the IEEE 123-bus system is considered, and the acceleration performance of the proposed Aitken-DMPC solver is evaluated and compared with the standard DMPC method.
△ Less
Submitted 18 May, 2021;
originally announced May 2021.
-
Scenario-based Nonlinear Model Predictive Control for Building Heating Systems
Authors:
Tomas Pippia,
Jesus Lago,
Roel De Coninck,
Bart De Schutter
Abstract:
State-of-the-art Model Predictive Control (MPC) applications for building heating adopt either a deterministic controller together with a nonlinear model or a linearized model with a stochastic MPC controller. However, deterministic MPC only considers one single realization of the disturbances and its performance strongly depends on the quality of the forecast of the disturbances, which can lead t…
▽ More
State-of-the-art Model Predictive Control (MPC) applications for building heating adopt either a deterministic controller together with a nonlinear model or a linearized model with a stochastic MPC controller. However, deterministic MPC only considers one single realization of the disturbances and its performance strongly depends on the quality of the forecast of the disturbances, which can lead to low performance. In fact, inadequate building energy management can lead to high energy costs and CO$_2$ emissions. On the other hand, a linearized model can fail to capture some dynamics and behavior of the building under control. In this article, we combine a stochastic scenario-based MPC (SBMPC) controller together with a nonlinear Modelica model that is able to provide a richer building description and to capture the dynamics of the building more accurately than linear models. The adopted SBMPC controller considers multiple realizations of the external disturbances obtained through a statistically accurate model, so as to consider different possible disturbance evolutions and to robustify the control action. To this purpose, we present a scenario generation method for building temperature control that can be applied to several exogenous perturbations, e.g.\ solar irradiance, outside temperature, and that satisfies several important stastistical properties, in contrast with simpler and less accurate methods adopted in the literature. We show the benefits of our proposed approach through several simulations in which we compare our method against the standard ones from the literature, for several combinations of a trade-off parameter between comfort and energy cost. We show how our SBMPC controller approach outperforms the standard controllers available in the literature.
△ Less
Submitted 20 May, 2021; v1 submitted 3 December, 2020;
originally announced December 2020.
-
Framework for Studying Stability of Switching Max-Plus Linear Systems
Authors:
Abhimanyu Gupta,
Ton van den Boom,
Jacob van der Woude,
Bart De Schutter
Abstract:
We propose a framework for studying the stability of discrete-event systems modelled as switching max-plus linear systems. In this framework, we propose a set of notions of stability for generic discrete-event systems in the max-plus algebra. Then we show the loss of equivalence of these notions for switching max-plus linear systems due to the lack of global monotonicity and the accompanying diffi…
▽ More
We propose a framework for studying the stability of discrete-event systems modelled as switching max-plus linear systems. In this framework, we propose a set of notions of stability for generic discrete-event systems in the max-plus algebra. Then we show the loss of equivalence of these notions for switching max-plus linear systems due to the lack of global monotonicity and the accompanying difficulty in rigorous analysis. This serves as a motivation to relax the assumption on monotonicity of the dynamics to positive invariance of max-plus cones. Then we proceed to generalise the notions of stability when the dynamics is restricted to such cones. The stability analysis approach presented in this paper serves as a first step to study the stability of a general class of switching max-plus linear systems.
△ Less
Submitted 6 July, 2020;
originally announced July 2020.
-
A Directed Spanning Tree Adaptive Control Framework for Time-Varying Formations
Authors:
Dongdong Yue,
Simone Baldi,
**de Cao,
Qi Li,
Bart De Schutter
Abstract:
In this paper, the time-varying formation and time-varying formation tracking problems are solved for linear multi-agent systems over digraphs without the knowledge of the eigenvalues of the Laplacian matrix associated to the digraph. The solution to these problems relies on a framework that generalizes the directed spanning tree adaptive method, which was originally limited to consensus problems.…
▽ More
In this paper, the time-varying formation and time-varying formation tracking problems are solved for linear multi-agent systems over digraphs without the knowledge of the eigenvalues of the Laplacian matrix associated to the digraph. The solution to these problems relies on a framework that generalizes the directed spanning tree adaptive method, which was originally limited to consensus problems. Necessary and sufficient conditions for the existence of solutions to the formation problems are derived. Asymptotic convergence of the formation errors is proved via graph theory and Lyapunov analysis.
△ Less
Submitted 18 February, 2021; v1 submitted 4 May, 2020;
originally announced May 2020.
-
Integrated Intelligent and Predictive Control: A Multi-Agent Adaptive Type-2 Fuzzy Control Architecture
Authors:
Anahita Jamshidnejad,
Emilio Frazzoli,
Mohammad J. Mahjoob,
Bart De Schutter
Abstract:
We propose a novel two-layer multi-agent architecture aimed at efficient real-time control of large-scale and complex-dynamics systems. The proposed architecture integrates intelligent control approaches (which have a low computation time and fit real-time applications) with model-predictive control (which takes care of the optimality requirements of control). The bottom control layer (intelligent…
▽ More
We propose a novel two-layer multi-agent architecture aimed at efficient real-time control of large-scale and complex-dynamics systems. The proposed architecture integrates intelligent control approaches (which have a low computation time and fit real-time applications) with model-predictive control (which takes care of the optimality requirements of control). The bottom control layer (intelligent-control module) includes several distributed intelligent-control agents, the design parameters of which are tuned by the top layer (model-predictive control module). The model-predictive control module fulfills two significant roles: looking ahead to the effects of the control decisions, and coordinating the intelligent-control agents of the lower control layer. The resulting multi-agent control system has a very low computation time, and provides adaptivity, control coordination, and aims at excellent performance. Additionally, we give a general treatment of type-2 fuzzy membership functions, and introduce two categories for them: probabilistic-fuzzy (which is a novel concept introduced in this paper) and fuzzy-fuzzy (which is a new treatment of the existing type-2 fuzzy membership functions). The performance of the proposed modeling and control approaches are assessed via a case study involving a simple urban traffic network: the results show that the novel concept of probabilistic-fuzzy membership function outperforms the type-1 and type-2 membership functions that have already been introduced in the literature. Furthermore, the proposed two-layer integrated multi-agent control architecture significantly outperforms a multi-agent decentralized fuzzy control system (without coordination among the agents), while requiring a comparable computation time.
△ Less
Submitted 28 August, 2019;
originally announced August 2019.
-
Comparison of Bounds for Optimal PMU Placement for State Estimation in Distribution Grids
Authors:
Miguel Picallo,
Adolfo Anta,
Bart De Schutter
Abstract:
The lack of measurements in distribution grids poses a severe challenge for their monitoring: since there may not be enough sensors to achieve numerical observability, load forecasts (pseudo-measurements) are typically used, and thus an accurate state estimation is not guaranteed. However, an estimation is required to control distribution grids given the increasing amount of distributed generation…
▽ More
The lack of measurements in distribution grids poses a severe challenge for their monitoring: since there may not be enough sensors to achieve numerical observability, load forecasts (pseudo-measurements) are typically used, and thus an accurate state estimation is not guaranteed. However, an estimation is required to control distribution grids given the increasing amount of distributed generation. Therefore, we consider the problem of optimal sensor placement to improve the state estimation accuracy in large-scale, 3-phase coupled, unbalanced distribution grids. This is a combinatorial optimization problem whose optimal solution is unpractical to obtain for large networks. We explore the properties of different metrics in the context of optimal experimental design, like convexity and modularity, to propose and compare several tight lower and upper bounds on the performance of the optimal solution. Moreover, we show how to use these bounds to choose near-optimal solutions. We test the method on two IEEE benchmark test feeders, the 123-bus and the 8500-node feeders, to show the effectiveness of the approach.
△ Less
Submitted 7 August, 2019;
originally announced August 2019.
-
Stochastic Optimal Power Flow in Distribution Grids under Uncertainty from State Estimation
Authors:
Miguel Picallo,
Adolfo Anta,
Bart De Schutter
Abstract:
The increasing amount of controllable generation and consumption in distribution grids poses a severe challenge in kee** voltage values within admissible ranges. Existing approaches have considered different optimal power flow formulations to regulate distributed generation and other controllable elements. Nevertheless, distribution grids are characterized by an insufficient number of sensors, a…
▽ More
The increasing amount of controllable generation and consumption in distribution grids poses a severe challenge in kee** voltage values within admissible ranges. Existing approaches have considered different optimal power flow formulations to regulate distributed generation and other controllable elements. Nevertheless, distribution grids are characterized by an insufficient number of sensors, and state estimation algorithms are required to monitor the grid status. We consider in this paper the combined problem of optimal power flow under state estimation, where the estimation uncertainty results into stochastic constraints for the voltage magnitude levels instead of deterministic ones. To solve the given problem efficiently and to bypass the lack of load measurements, we use a linear approximation of the power flow equations. Moreover, we derive a transformation of the stochastic constraints to make them tractable without being too conservative. A case study shows the success of our approach at kee** voltage within limits, and also shows how ignoring the uncertainty in the estimation can lead to voltage level violations.
△ Less
Submitted 24 July, 2019;
originally announced July 2019.
-
Integrated Offline and Online Optimization-Based Control in a Base-Parallel Architecture
Authors:
Anahita Jamshidnejad,
Gabriel Gomes,
Alexandre M. Bayen,
Bart De Schutter
Abstract:
We propose an integrated control architecture to address the gap that currently exists for efficient real-time implementation of MPC-based control approaches for highly nonlinear systems with fast dynamics and a large number of control constraints. The proposed architecture contains two types of controllers: base controllers that are tuned or optimized offline, and parallel controllers that solve…
▽ More
We propose an integrated control architecture to address the gap that currently exists for efficient real-time implementation of MPC-based control approaches for highly nonlinear systems with fast dynamics and a large number of control constraints. The proposed architecture contains two types of controllers: base controllers that are tuned or optimized offline, and parallel controllers that solve an optimization-based control problem online. The control inputs computed by the base controllers provide starting points for the optimization problem of the parallel controllers, which operate in parallel within a limited time budget that does not exceed the control sampling time. The resulting control system is very flexible and its architecture can easily be modified or changed online, e.g., by adding or eliminating controllers, for online improvement of the performance of the controlled system. In a case study, the proposed control architecture is implemented for highway traffic, which is characterized by nonlinear, fast dynamics with multiple control constraints, to minimize the overall travel time of the vehicles, while increasing their total traveled distance within the fixed simulation time window. The results of the simulation show the excellent real-time (i.e., within the given time budget) performance of the proposed control architecture, with the least realized value of the overall cost function. Moreover, among the online control approaches considered for the case study, the average cost per vehicle for the base-parallel control approach is the closest to the online MPC-based controllers, which have excellent performance but may involve computation times that exceed the given time budget.
△ Less
Submitted 11 July, 2019;
originally announced July 2019.
-
Efficient convex optimization for optimal PMU placement in large distribution grids
Authors:
Miguel Picallo,
Adolfo Anta,
Bart De Schutter
Abstract:
The small amount of measurements in distribution grids makes their monitoring more difficult. Topological observability may not be possible, and thus, pseudo-measurements are needed to perform state estimation, which is required to control elements such as distributed generation or transformers at distribution grids. Therefore, we consider the problem of optimal sensor placement to improve the sta…
▽ More
The small amount of measurements in distribution grids makes their monitoring more difficult. Topological observability may not be possible, and thus, pseudo-measurements are needed to perform state estimation, which is required to control elements such as distributed generation or transformers at distribution grids. Therefore, we consider the problem of optimal sensor placement to improve the state estimation accuracy in large-scale, 3-phase coupled, unbalanced distribution grids. This is an NP-hard optimization problem whose optimal solution is unpractical to obtain for large networks. Therefore, we develop a computationally efficient convex optimization algorithm to compute a lower bound on the possible value of the optimal solution, and thus check the gap between the bound and heuristic solutions. We test the method on a large test feeder, the standard IEEE 8500-node, to show the effectiveness of the approach.
△ Less
Submitted 23 July, 2019; v1 submitted 29 October, 2018;
originally announced October 2018.
-
A Two-Step Distribution System State Estimator with Grid Constraints and Mixed Measurements
Authors:
Miguel Picallo Cruz,
Adolfo Anta,
Ara Panosyan,
Bart De Schutter
Abstract:
In this work we study the problem of State Estimation(SE) in large-scale, 3-phase coupled, unbalanced distribution systems. More specifically, we address the problem of including mixed real-time measurements, synchronized and unsynchronized, from phasor measurement units and smart meters, into existing solutions. We propose a computationally efficient two-step method to update a prior solution usi…
▽ More
In this work we study the problem of State Estimation(SE) in large-scale, 3-phase coupled, unbalanced distribution systems. More specifically, we address the problem of including mixed real-time measurements, synchronized and unsynchronized, from phasor measurement units and smart meters, into existing solutions. We propose a computationally efficient two-step method to update a prior solution using the measurements, while taking into account physical constraint caused by buses with no loads. We test the method on a benchmark test feeder to illustrate the effectiveness of the approach.
△ Less
Submitted 15 May, 2018; v1 submitted 31 March, 2017;
originally announced March 2017.
-
Residential Demand Response Applications Using Batch Reinforcement Learning
Authors:
Frederik Ruelens,
Bert Claessens,
Stijn Vandael,
Bart De Schutter,
Robert Babuska,
Ronnie Belmans
Abstract:
Driven by recent advances in batch Reinforcement Learning (RL), this paper contributes to the application of batch RL to demand response. In contrast to conventional model-based approaches, batch RL techniques do not require a system identification step, which makes them more suitable for a large-scale implementation. This paper extends fitted Q-iteration, a standard batch RL technique, to the sit…
▽ More
Driven by recent advances in batch Reinforcement Learning (RL), this paper contributes to the application of batch RL to demand response. In contrast to conventional model-based approaches, batch RL techniques do not require a system identification step, which makes them more suitable for a large-scale implementation. This paper extends fitted Q-iteration, a standard batch RL technique, to the situation where a forecast of the exogenous data is provided. In general, batch RL techniques do not rely on expert knowledge on the system dynamics or the solution. However, if some expert knowledge is provided, it can be incorporated by using our novel policy adjustment method. Finally, we tackle the challenge of finding an open-loop schedule required to participate in the day-ahead market. We propose a model-free Monte-Carlo estimator method that uses a metric to construct artificial trajectories and we illustrate this method by finding the day-ahead schedule of a heat-pump thermostat. Our experiments show that batch RL techniques provide a valuable alternative to model-based controllers and that they can be used to construct both closed-loop and open-loop policies.
△ Less
Submitted 8 April, 2015;
originally announced April 2015.
-
A distributed accelerated gradient algorithm for distributed model predictive control of a hydro power valley
Authors:
Minh Dang Doan,
Pontus Giselsson,
Tamás Keviczky,
Bart De Schutter,
Anders Rantzer
Abstract:
A distributed model predictive control (DMPC) approach based on distributed optimization is applied to the power reference tracking problem of a hydro power valley (HPV) system. The applied optimization algorithm is based on accelerated gradient methods and achieves a convergence rate of O(1/k^2), where k is the iteration number. Major challenges in the control of the HPV include a nonlinear and l…
▽ More
A distributed model predictive control (DMPC) approach based on distributed optimization is applied to the power reference tracking problem of a hydro power valley (HPV) system. The applied optimization algorithm is based on accelerated gradient methods and achieves a convergence rate of O(1/k^2), where k is the iteration number. Major challenges in the control of the HPV include a nonlinear and large-scale model, nonsmoothness in the power-production functions, and a globally coupled cost function that prevents distributed schemes to be applied directly. We propose a linearization and approximation approach that accommodates the proposed the DMPC framework and provides very similar performance compared to a centralized solution in simulations. The provided numerical studies also suggest that for the sparsely interconnected system at hand, the distributed algorithm we propose is faster than a centralized state-of-the-art solver such as CPLEX.
△ Less
Submitted 8 February, 2013;
originally announced February 2013.
-
Synchronization of a class of cyclic discrete-event systems describing legged locomotion
Authors:
G. A. D. Lopes,
B. Kersbergen,
B. De Schutter,
T. J. J. van den Boom,
R. Babuska
Abstract:
It has been shown that max-plus linear systems are well suited for applications in synchronization and scheduling, such as the generation of train timetables, manufacturing, or traffic. In this paper we show that the same is true for multi-legged locomotion. In this framework, the max-plus eigenvalue of the system matrix represents the total cycle time, whereas the max-plus eigenvector dictates th…
▽ More
It has been shown that max-plus linear systems are well suited for applications in synchronization and scheduling, such as the generation of train timetables, manufacturing, or traffic. In this paper we show that the same is true for multi-legged locomotion. In this framework, the max-plus eigenvalue of the system matrix represents the total cycle time, whereas the max-plus eigenvector dictates the steady-state behavior. Uniqueness of the eigenstructure also indicates uniqueness of the resulting behavior. For the particular case of legged locomotion, the movement of each leg is abstracted to two-state circuits: swing and stance (leg in flight and on the ground, respectively). The generation of a gait (a manner of walking) for a multiple legged robot is then achieved by synchronizing the multiple discrete-event cycles via the max-plus framework. By construction, different gaits and gait parameters can be safely interleaved by using different system matrices. In this paper we address both the transient and steady-state behavior for a class of gaits by presenting closed-form expressions for the max-plus eigenvalue and max-plus eigenvector of the system matrix and the coupling time. The significance of this result is in showing guaranteed robustness to perturbations and gait switching, and also a systematic methodology for synthesizing controllers that allow for legged robots to change rhythms fast.
△ Less
Submitted 21 December, 2012;
originally announced December 2012.
-
A distributed optimization-based approach for hierarchical model predictive control of large-scale systems with coupled dynamics and constraints
Authors:
Minh Dang Doan,
Tamás Keviczky,
Bart De Schutter
Abstract:
We present a hierarchical model predictive control approach for large-scale systems based on dual decomposition. The proposed scheme allows coupling in both dynamics and constraints between the subsystems and generates a primal feasible solution within a finite number of iterations, using primal averaging and a constraint tightening approach. The primal update is performed in a distributed way and…
▽ More
We present a hierarchical model predictive control approach for large-scale systems based on dual decomposition. The proposed scheme allows coupling in both dynamics and constraints between the subsystems and generates a primal feasible solution within a finite number of iterations, using primal averaging and a constraint tightening approach. The primal update is performed in a distributed way and does not require exact solutions, while the dual problem uses an approximate subgradient method. Stability of the scheme is established using bounded suboptimality.
△ Less
Submitted 7 September, 2011; v1 submitted 6 September, 2011;
originally announced September 2011.
