Showing 1–50 of 71 results for author: Belta, C

Interpretable Generative Adversarial Imitation Learning

Authors: Wenliang Liu, Danyang Li, Erfan Aasi, Roberto Tron, Calin Belta

Abstract: Imitation learning methods have demonstrated considerable success in teaching autonomous systems complex tasks through expert demonstrations. However, a limitation of these methods is their lack of interpretability, particularly in understanding the specific task the learning agent aims to accomplish. In this paper, we propose a novel imitation learning method that combines Signal Temporal Logic (… ▽ More Imitation learning methods have demonstrated considerable success in teaching autonomous systems complex tasks through expert demonstrations. However, a limitation of these methods is their lack of interpretability, particularly in understanding the specific task the learning agent aims to accomplish. In this paper, we propose a novel imitation learning method that combines Signal Temporal Logic (STL) inference and control synthesis, enabling the explicit representation of the task as an STL formula. This approach not only provides a clear understanding of the task but also allows for the incorporation of human knowledge and adaptation to new scenarios through manual adjustments of the STL formulae. Additionally, we employ a Generative Adversarial Network (GAN)-inspired training approach for both the inference and the control policy, effectively narrowing the gap between the expert and learned policies. The effectiveness of our algorithm is demonstrated through two case studies, showcasing its practical applicability and adaptability. △ Less

Submitted 15 February, 2024; originally announced February 2024.

Comments: Submitted to L4DC 2024 (under review)

Control-Based Planning over Probability Mass Function Measurements via Robust Linear Programming

Authors: Mehdi Kermanshah, Calin Belta, Roberto Tron

Abstract: We propose an approach to synthesize linear feedback controllers for linear systems in polygonal environments. Our method focuses on designing a robust controller that can account for uncertainty in measurements. Its inputs are provided by a perception module that generates probability mass functions (PMFs) for predefined landmarks in the environment, such as distinguishable geometric features. We… ▽ More We propose an approach to synthesize linear feedback controllers for linear systems in polygonal environments. Our method focuses on designing a robust controller that can account for uncertainty in measurements. Its inputs are provided by a perception module that generates probability mass functions (PMFs) for predefined landmarks in the environment, such as distinguishable geometric features. We formulate an optimization problem with Control Lyapunov Function (CLF) and Control Barrier Function (CBF) constraints to derive a stable and safe controller. Using the strong duality of linear programs (LPs) and robust optimization, we convert the optimization problem to a linear program that can be efficiently solved offline. At a high level, our approach partially combines perception, planning, and real-time control into a single design problem. An additional advantage of our method is the ability to produce controllers capable of exhibiting nonlinear behavior while relying solely on an offline LP for control synthesis. △ Less

Submitted 12 October, 2023; originally announced October 2023.

Feasibility-Guaranteed Safety-Critical Control with Applications to Heterogeneous Platoons

Authors: Shuo Liu, Wei Xiao, Calin A. Belta

Abstract: This paper studies safety and feasibility guarantees for systems with tight control bounds. It has been shown that stabilizing an affine control system while optimizing a quadratic cost and satisfying state and control constraints can be mapped to a sequence of Quadratic Programs (QPs) using Control Barrier Functions (CBF) and Control Lyapunov Functions (CLF). One of the main challenges in this me… ▽ More This paper studies safety and feasibility guarantees for systems with tight control bounds. It has been shown that stabilizing an affine control system while optimizing a quadratic cost and satisfying state and control constraints can be mapped to a sequence of Quadratic Programs (QPs) using Control Barrier Functions (CBF) and Control Lyapunov Functions (CLF). One of the main challenges in this method is that the QP could easily become infeasible under safety constraints of high relative degree, especially under tight control bounds. Recent work focused on deriving sufficient conditions for guaranteeing feasibility. The existing results are case-dependent. In this paper, we consider the general case. We define a feasibility constraint and propose a new type of CBF to enforce it. Our method guarantees the feasibility of the above mentioned QPs, while satisfying safety requirements. We demonstrate the proposed method on an Adaptive Cruise Control (ACC) problem for a heterogeneous platoon with tight control bounds, and compare our method to existing CBF-CLF approaches. The results show that our proposed approach can generate gradually transitioned control (without abrupt changes) with guaranteed feasibility and safety. △ Less

Submitted 31 March, 2024; v1 submitted 29 September, 2023; originally announced October 2023.

Comments: 8 pages, 2 figures. arXiv admin note: text overlap with arXiv:2304.00372

Learning Robust and Correct Controllers from Signal Temporal Logic Specifications Using BarrierNet

Authors: Wenliang Liu, Wei Xiao, Calin Belta

Abstract: In this paper, we consider the problem of learning a neural network controller for a system required to satisfy a Signal Temporal Logic (STL) specification. We exploit STL quantitative semantics to define a notion of robust satisfaction. Guaranteeing the correctness of a neural network controller, i.e., ensuring the satisfaction of the specification by the controlled system, is a difficult problem… ▽ More In this paper, we consider the problem of learning a neural network controller for a system required to satisfy a Signal Temporal Logic (STL) specification. We exploit STL quantitative semantics to define a notion of robust satisfaction. Guaranteeing the correctness of a neural network controller, i.e., ensuring the satisfaction of the specification by the controlled system, is a difficult problem that received a lot of attention recently. We provide a general procedure to construct a set of trainable High Order Control Barrier Functions (HOCBFs) enforcing the satisfaction of formulas in a fragment of STL. We use the BarrierNet, implemented by a differentiable Quadratic Program (dQP) with HOCBF constraints, as the last layer of the neural network controller, to guarantee the satisfaction of the STL formulas. We train the HOCBFs together with other neural network parameters to further improve the robustness of the controller. Simulation results demonstrate that our approach ensures satisfaction and outperforms existing algorithms. △ Less

Submitted 12 April, 2023; originally announced April 2023.

Comments: Submitted to CDC 2023

Uncertainty Quantification for Recursive Estimation in Adaptive Safety-Critical Control

Authors: Max H. Cohen, Makai Mann, Kevin Leahy, Calin Belta

Abstract: In this paper, we present a framework for online parameter estimation and uncertainty quantification in the context of adaptive safety-critical control. The key insight enabling our approach is that the parameter estimate generated by the continuous-time recursive least squares (RLS) algorithm at any point in time is an affine transformation of the initial parameter estimate. This property allows… ▽ More In this paper, we present a framework for online parameter estimation and uncertainty quantification in the context of adaptive safety-critical control. The key insight enabling our approach is that the parameter estimate generated by the continuous-time recursive least squares (RLS) algorithm at any point in time is an affine transformation of the initial parameter estimate. This property allows for parameterizing such estimates using objects that are closed under affine transformation, such as zonotopes, and enables the efficient propagation of such set-based estimates as time progresses. We illustrate how such an approach facilitates the synthesis of safety-critical controllers for systems with parametric uncertainty and additive disturbances using control barrier functions, and demonstrate the utility of our approach through illustrative examples. △ Less

Submitted 12 March, 2024; v1 submitted 4 April, 2023; originally announced April 2023.

Comments: To appear at the 2024 American Control Conference

LQR-CBF-RRT*: Safe and Optimal Motion Planning

Authors: Guang Yang, Mingyu Cai, Ahmad Ahmad, Amanda Prorok, Roberto Tron, Calin Belta

Abstract: We present LQR-CBF-RRT*, an incremental sampling-based algorithm for offline motion planning. Our framework leverages the strength of Control Barrier Functions (CBFs) and Linear Quadratic Regulators (LQR) to generate safety-critical and optimal trajectories for a robot with dynamics described by an affine control system. CBFs are used for safety guarantees, while LQRs are employed for optimal cont… ▽ More We present LQR-CBF-RRT*, an incremental sampling-based algorithm for offline motion planning. Our framework leverages the strength of Control Barrier Functions (CBFs) and Linear Quadratic Regulators (LQR) to generate safety-critical and optimal trajectories for a robot with dynamics described by an affine control system. CBFs are used for safety guarantees, while LQRs are employed for optimal control synthesis during edge extensions. Popular CBF-based formulations for safety critical control require solving Quadratic Programs (QPs), which can be computationally expensive. Moreover, LQR-based controllers require repetitive applications of first-order Taylor approximations for nonlinear systems, which can also create an additional computational burden. To improve the motion planning efficiency, we verify the satisfaction of the CBF constraints directly in edge extension to avoid the burden of solving the QPs. We store computed optimal LQR gain matrices in a hash table to avoid re-computation during the local linearization of the rewiring procedure. Lastly, we utilize the Cross-Entropy Method for importance sampling to improve sampling efficiency. Our results show that the proposed planner surpasses its counterparts in computational efficiency and performs well in an experimental setup. △ Less

Submitted 27 September, 2023; v1 submitted 3 April, 2023; originally announced April 2023.

Auxiliary-Variable Adaptive Control Barrier Functions for Safety Critical Systems

Authors: Shuo Liu, Wei Xiao, Calin A. Belta

Abstract: This paper studies safety guarantees for systems with time-varying control bounds. It has been shown that optimizing quadratic costs subject to state and control constraints can be reduced to a sequence of Quadratic Programs (QPs) using Control Barrier Functions (CBFs). One of the main challenges in this method is that the CBF-based QP could easily become infeasible under tight control bounds, esp… ▽ More This paper studies safety guarantees for systems with time-varying control bounds. It has been shown that optimizing quadratic costs subject to state and control constraints can be reduced to a sequence of Quadratic Programs (QPs) using Control Barrier Functions (CBFs). One of the main challenges in this method is that the CBF-based QP could easily become infeasible under tight control bounds, especially when the control bounds are time-varying. The recently proposed adaptive CBFs have addressed such infeasibility issues, but require extensive and non-trivial hyperparameter tuning for the CBF-based QP and may introduce overshooting control near the boundaries of safe sets. To address these issues, we propose a new type of adaptive CBFs called Auxiliary-Variable Adaptive CBFs (AVCBFs). Specifically, we introduce an auxiliary variable that multiplies each CBF itself, and define dynamics for the auxiliary variable to adapt it in constructing the corresponding CBF constraint. In this way, we can improve the feasibility of the CBF-based QP while avoiding extensive parameter tuning with non-overshooting control since the formulation is identical to classical CBF methods. We demonstrate the advantages of using AVCBFs and compare them with existing techniques on an Adaptive Cruise Control (ACC) problem with time-varying control bounds. △ Less

Submitted 19 April, 2024; v1 submitted 1 April, 2023; originally announced April 2023.

Comments: 8 pages, 4 figures. arXiv admin note: text overlap with arXiv:2310.00238

Learning Feasibility Constraints for Control Barrier Functions

Authors: Wei Xiao, Christos G. Cassandras, Calin A. Belta

Abstract: It has been shown that optimizing quadratic costs while stabilizing affine control systems to desired (sets of) states subject to state and control constraints can be reduced to a sequence of Quadratic Programs (QPs) by using Control Barrier Functions (CBFs) and Control Lyapunov Functions (CLFs). In this paper, we employ machine learning techniques to ensure the feasibility of these QPs, which is… ▽ More It has been shown that optimizing quadratic costs while stabilizing affine control systems to desired (sets of) states subject to state and control constraints can be reduced to a sequence of Quadratic Programs (QPs) by using Control Barrier Functions (CBFs) and Control Lyapunov Functions (CLFs). In this paper, we employ machine learning techniques to ensure the feasibility of these QPs, which is a challenging problem, especially for high relative degree constraints where High Order CBFs (HOCBFs) are required. To this end, we propose a sampling-based learning approach to learn a new feasibility constraint for CBFs; this constraint is then enforced by another HOCBF added to the QPs. The accuracy of the learned feasibility constraint is recursively improved by a recurrent training algorithm. We demonstrate the advantages of the proposed learning approach to constrained optimal control problems with specific focus on a robot control problem and on autonomous driving in an unknown environment. △ Less

Submitted 10 March, 2023; originally announced March 2023.

Comments: 8 pages, to appear in ECC 2023

Modular Adaptive Safety-Critical Control

Authors: Max Cohen, Calin Belta

Abstract: This paper presents an adaptive control approach for uncertain nonlinear systems subject to safety constraints that allows for modularity in the selection of the parameter estimation algorithm. Such modularity is achieved by unifying the concepts of input-to-state stability (ISS) and input-to-state safety (ISSf) via control Lyapunov functions (CLFs) and control barrier functions (CBFs), respective… ▽ More This paper presents an adaptive control approach for uncertain nonlinear systems subject to safety constraints that allows for modularity in the selection of the parameter estimation algorithm. Such modularity is achieved by unifying the concepts of input-to-state stability (ISS) and input-to-state safety (ISSf) via control Lyapunov functions (CLFs) and control barrier functions (CBFs), respectively. In particular, we propose a class of exponential ISS-CLFs and ISSf high order CBFs that can be combined with a general class of parameter estimation algorithms akin to those found in the literature on concurrent learning adaptive control. We demonstrate that the unification of ISS and ISSf in an adaptive control setting allows for maintaining a single set of parameter estimates for both the CLF and CBF that can be generated by a class of update laws satisfying a few general properties. The modularity of our approach is demonstrated via numerical examples by comparing performance in terms of stability and safety across different parameter estimation algorithms. △ Less

Submitted 7 March, 2023; originally announced March 2023.

Comments: To appear at the 2023 American Control Conference

CatlNet: Learning Communication and Coordination Policies from CaTL+ Specifications

Authors: Wenliang Liu, Kevin Leahy, Zachary Serlin, Calin Belta

Abstract: In this paper, we propose a learning-based framework to simultaneously learn the communication and distributed control policies for a heterogeneous multi-agent system (MAS) under complex mission requirements from Capability Temporal Logic plus (CaTL+) specifications. Both policies are trained, implemented, and deployed using a novel neural network model called CatlNet. Taking advantage of the robu… ▽ More In this paper, we propose a learning-based framework to simultaneously learn the communication and distributed control policies for a heterogeneous multi-agent system (MAS) under complex mission requirements from Capability Temporal Logic plus (CaTL+) specifications. Both policies are trained, implemented, and deployed using a novel neural network model called CatlNet. Taking advantage of the robustness measure of CaTL+, we train CatlNet centrally to maximize it where network parameters are shared among all agents, allowing CatlNet to scale to large teams easily. CatlNet can then be deployed distributedly. A plan repair algorithm is also introduced to guide CatlNet's training and improve both training efficiency and the overall performance of CatlNet. The CatlNet approach is tested in simulation and results show that, after training, CatlNet can steer the decentralized MAS system online to satisfy a CaTL+ specification with a high success rate. △ Less

Submitted 30 November, 2022; originally announced December 2022.

Iterative Convex Optimization for Model Predictive Control with Discrete-Time High-Order Control Barrier Functions

Authors: Shuo Liu, Jun Zeng, Koushil Sreenath, Calin A. Belta

Abstract: Safety is one of the fundamental challenges in control theory. Recently, multi-step optimal control problems for discrete-time dynamical systems were formulated to enforce stability, while subject to input constraints as well as safety-critical requirements using discrete-time control barrier functions within a model predictive control (MPC) framework. Existing work usually focus on the feasibilit… ▽ More Safety is one of the fundamental challenges in control theory. Recently, multi-step optimal control problems for discrete-time dynamical systems were formulated to enforce stability, while subject to input constraints as well as safety-critical requirements using discrete-time control barrier functions within a model predictive control (MPC) framework. Existing work usually focus on the feasibility or the safety for the optimization problem, and the majority of the existing work restrict the discussions to relative-degree one control barrier functions. Additionally, the real-time computation is challenging when a large horizon is considered in the MPC problem for relative-degree one or high-order control barrier functions. In this paper, we propose a framework that solves the safety-critical MPC problem in an iterative optimization, which is applicable for any relative-degree control barrier functions. In the proposed formulation, the nonlinear system dynamics as well as the safety constraints modeled as discrete-time high-order control barrier functions (DHOCBF) are linearized at each time step. Our formulation is generally valid for any control barrier function with an arbitrary relative-degree. The advantages of fast computational performance with safety guarantee are analyzed and validated with numerical results. △ Less

Submitted 13 July, 2023; v1 submitted 9 October, 2022; originally announced October 2022.

Comments: The open source code is added and the paper is accepted to American Control Conference (ACC) 2023 (8 pages)

Robust Multi-Agent Coordination from CaTL+ Specifications

Authors: Wenliang Liu, Kevin Leahy, Zachary Serlin, Calin Belta

Abstract: We consider the problem of controlling a heterogeneous multi-agent system required to satisfy temporal logic requirements. Capability Temporal Logic (CaTL) was recently proposed to formalize such specifications for deploying a team of autonomous agents with different capabilities and cooperation requirements. In this paper, we extend CaTL to a new logic CaTL+, which is more expressive than CaTL an… ▽ More We consider the problem of controlling a heterogeneous multi-agent system required to satisfy temporal logic requirements. Capability Temporal Logic (CaTL) was recently proposed to formalize such specifications for deploying a team of autonomous agents with different capabilities and cooperation requirements. In this paper, we extend CaTL to a new logic CaTL+, which is more expressive than CaTL and has semantics over a continuous workspace shared by all agents. We define two novel robustness metrics for CaTL+: the traditional robustness and the exponential robustness. The latter is sound, differentiable almost everywhere and eliminates masking, which is one of the main limitations of the traditional robustness metric. We formulate a control synthesis problem to maximize CaTL+ robustness and propose a two-step optimization method to solve this problem. Simulation results are included to illustrate the increased expressivity of CaTL+ and the efficacy of the proposed control synthesis approach. △ Less

Submitted 12 April, 2023; v1 submitted 4 October, 2022; originally announced October 2022.

Comments: Accepted to ACC 2023

Robust Control Barrier Functions for Nonlinear Control Systems with Uncertainty: A Duality-based Approach

Authors: Max H. Cohen, Calin Belta, Roberto Tron

Abstract: This paper studies the design of controllers that guarantee stability and safety of nonlinear control affine systems with parametric uncertainty in both the drift and control vector fields. To this end, we introduce novel classes of robust control barrier functions (RCBF) and robust control Lyapunov functions (RCLF) that facilitate the synthesis of safety-critical controllers in the presence of pa… ▽ More This paper studies the design of controllers that guarantee stability and safety of nonlinear control affine systems with parametric uncertainty in both the drift and control vector fields. To this end, we introduce novel classes of robust control barrier functions (RCBF) and robust control Lyapunov functions (RCLF) that facilitate the synthesis of safety-critical controllers in the presence of parametric uncertainty using quadratic programming. Since the initial bounds on the system uncertainty may be highly conservative, we present a data-driven approach to reducing such bounds using input-output data collected online. In particular, we leverage an integral set-membership identification algorithm that iteratively shrinks the set of possible system parameters online and guarantees stability and safety during learning. The efficacy of the developed approach is illustrated on two numerical examples. △ Less

Submitted 11 August, 2022; originally announced August 2022.

Comments: To appear at the 2022 IEEE Conference on Decision and Control

Compositional Synthesis for Linear Systems via Convex Optimization of Assume-Guarantee Contracts

Authors: Kasra Ghasemi, Sadra Sadraddini, Calin Belta

Abstract: We take a divide and conquer approach to design controllers for reachability problems given large-scale linear systems with polyhedral constraints on states, controls, and disturbances. Such systems are made of small subsystems with coupled dynamics. We treat the couplings as additional disturbances and use assume-guarantee (AG) contracts to characterize these disturbance sets. For each subsystem,… ▽ More We take a divide and conquer approach to design controllers for reachability problems given large-scale linear systems with polyhedral constraints on states, controls, and disturbances. Such systems are made of small subsystems with coupled dynamics. We treat the couplings as additional disturbances and use assume-guarantee (AG) contracts to characterize these disturbance sets. For each subsystem, we design and implement a robust controller locally, subject to its own constraints and contracts. The main contribution of this paper is a method to derive the contracts via a novel parameterization and a corresponding potential function that characterizes the distance to the correct composition of controllers and contracts, where all contracts are held. We show that the potential function is convex in the contract parameters. This enables the subsystems to negotiate the contracts with the gradient information from the dual of their local synthesis optimization problems in a distributed way, facilitating compositional control synthesis that scales to large systems. We present numerical examples, including a scalability study on a system with tens of thousands of dimensions, and a case study on applying our method to a distributed Model Predictive Control (MPC) problem in a power system. △ Less

Submitted 2 August, 2022; originally announced August 2022.

Decentralized Signal Temporal Logic Control for Perturbed Interconnected Systems via Assume-Guarantee Contract Optimization

Authors: Kasra Ghasemi, Sadra Sadraddini, Calin Belta

Abstract: We develop a novel decentralized control method for a network of perturbed linear systems with dynamical couplings subject to Signal Temporal Logic (STL) specifications. We first transform the STL requirements into set containment problems and then we develop controllers to solve these problems. Our approach is based on treating the couplings between subsystems as disturbances, which are bounded s… ▽ More We develop a novel decentralized control method for a network of perturbed linear systems with dynamical couplings subject to Signal Temporal Logic (STL) specifications. We first transform the STL requirements into set containment problems and then we develop controllers to solve these problems. Our approach is based on treating the couplings between subsystems as disturbances, which are bounded sets that the subsystems negotiate in the form of parametric assume-guarantee contracts. The set containment requirements and parameterized contracts are added to the subsystems' constraints. We introduce a centralized optimization problem to derive the contracts, reachability tubes, and decentralized closed-loop control laws. We show that, when the STL formula is separable with respect to the subsystems, the centralized optimization problem can be solved in a distributed way, which scales to large systems. We present formal theoretical guarantees on robustness of STL satisfaction. The effectiveness of the proposed method is demonstrated via a power network case study. △ Less

Submitted 30 June, 2022; originally announced July 2022.

Adaptive Sampling-based Motion Planning with Control Barrier Functions

Authors: Ahmad Ahmad, Calin Belta, Roberto Tron

Abstract: Sampling-based algorithms, such as Rapidly Exploring Random Trees (RRT) and its variants, have been used extensively for motion planning. Control barrier functions (CBFs) have been recently proposed to synthesize controllers for safety-critical systems. In this paper, we combine the effectiveness of RRT-based algorithms with the safety guarantees provided by CBFs in a method called CBF-RRT$^\ast$.… ▽ More Sampling-based algorithms, such as Rapidly Exploring Random Trees (RRT) and its variants, have been used extensively for motion planning. Control barrier functions (CBFs) have been recently proposed to synthesize controllers for safety-critical systems. In this paper, we combine the effectiveness of RRT-based algorithms with the safety guarantees provided by CBFs in a method called CBF-RRT$^\ast$. CBFs are used for local trajectory planning for RRT$^\ast$, avoiding explicit collision checking of the extended paths. We prove that CBF-RRT$^\ast$ preserves the probabilistic completeness of RRT$^\ast$. Furthermore, in order to improve the sampling efficiency of the algorithm, we equip the algorithm with an adaptive sampling procedure, which is based on the cross-entropy method (CEM) for importance sampling (IS). The procedure exploits the tree of samples to focus the sampling in promising regions of the configuration space. We demonstrate the efficacy of the proposed algorithms through simulation examples. △ Less

Submitted 1 June, 2022; originally announced June 2022.

Comments: Submitted to CDC2022

Control Barrier Functions for Systems with Multiple Control Inputs

Authors: Wei Xiao, Christos G. Cassandras, Calin A. Belta, Daniela Rus

Abstract: Control Barrier Functions (CBFs) are becoming popular tools in guaranteeing safety for nonlinear systems and constraints, and they can reduce a constrained optimal control problem into a sequence of Quadratic Programs (QPs) for affine control systems. The recently proposed High Order Control Barrier Functions (HOCBFs) work for arbitrary relative degree constraints. One of the challenges in a HOCBF… ▽ More Control Barrier Functions (CBFs) are becoming popular tools in guaranteeing safety for nonlinear systems and constraints, and they can reduce a constrained optimal control problem into a sequence of Quadratic Programs (QPs) for affine control systems. The recently proposed High Order Control Barrier Functions (HOCBFs) work for arbitrary relative degree constraints. One of the challenges in a HOCBF is to address the relative degree problem when a system has multiple control inputs, i.e., the relative degree could be defined with respect to different components of the control vector. This paper proposes two methods for HOCBFs to deal with systems with multiple control inputs: a general integral control method and a method which is simpler but limited to specific classes of physical systems. When control bounds are involved, the feasibility of the above mentioned QPs can also be significantly improved with the proposed methods. We illustrate our approaches on a unicyle model with two control inputs, and compare the two proposed methods to demonstrate their effectiveness and performance. △ Less

Submitted 15 March, 2022; originally announced March 2022.

Comments: To appear in ACC2022

Distributed Control using Reinforcement Learning with Temporal-Logic-Based Reward Sha**

Authors: Ningyuan Zhang, Wenliang Liu, Calin Belta

Abstract: We present a computational framework for synthesis of distributed control strategies for a heterogeneous team of robots in a partially observable environment. The goal is to cooperatively satisfy specifications given as Truncated Linear Temporal Logic (TLTL) formulas. Our approach formulates the synthesis problem as a stochastic game and employs a policy graph method to find a control strategy wit… ▽ More We present a computational framework for synthesis of distributed control strategies for a heterogeneous team of robots in a partially observable environment. The goal is to cooperatively satisfy specifications given as Truncated Linear Temporal Logic (TLTL) formulas. Our approach formulates the synthesis problem as a stochastic game and employs a policy graph method to find a control strategy with memory for each agent. We construct the stochastic game on the product between the team transition system and a finite state automaton (FSA) that tracks the satisfaction of the TLTL formula. We use the quantitative semantics of TLTL as the reward of the game, and further reshape it using the FSA to guide and accelerate the learning process. Simulation results demonstrate the efficacy of the proposed solution under demanding task specifications and the effectiveness of reward sha** in significantly accelerating the speed of learning. △ Less

Submitted 6 April, 2022; v1 submitted 8 March, 2022; originally announced March 2022.

Comments: 12 pages, 4 figures, accepted by L4DC 2022

High Order Robust Adaptive Control Barrier Functions and Exponentially Stabilizing Adaptive Control Lyapunov Functions

Authors: Max H. Cohen, Calin Belta

Abstract: This paper studies the problem of utilizing data-driven adaptive control techniques to guarantee stability and safety of uncertain nonlinear systems with high relative degree. We first introduce the notion of a High Order Robust Adaptive Control Barrier Function (HO-RaCBF) as a means to compute control policies guaranteeing satisfaction of high relative degree safety constraints in the face of par… ▽ More This paper studies the problem of utilizing data-driven adaptive control techniques to guarantee stability and safety of uncertain nonlinear systems with high relative degree. We first introduce the notion of a High Order Robust Adaptive Control Barrier Function (HO-RaCBF) as a means to compute control policies guaranteeing satisfaction of high relative degree safety constraints in the face of parametric model uncertainty. The developed approach guarantees safety by initially accounting for all possible parameter realizations but adaptively reduces uncertainty in the parameter estimates leveraging data recorded online. We then introduce the notion of an Exponentially Stabilizing Adaptive Control Lyapunov Function (ES-aCLF) that leverages the same data as the HO-RaCBF controller to guarantee exponential convergence of the system trajectory. The developed HO-RaCBF and ES-aCLF are unified in a quadratic programming framework, whose efficacy is showcased via two numerical examples that, to our knowledge, cannot be addressed by existing adaptive control barrier function techniques. △ Less

Submitted 3 March, 2022; originally announced March 2022.

Comments: Accepted to the 2022 American Control Conference

Learning Spatio-Temporal Specifications for Dynamical Systems

Authors: Suhail Alsalehi, Erfan Aasi, Ron Weiss, Calin Belta

Abstract: Learning dynamical systems properties from data provides important insights that help us understand such systems and mitigate undesired outcomes. In this work, we propose a framework for learning spatio-temporal (ST) properties as formal logic specifications from data. We introduce SVM-STL, an extension of Signal Signal Temporal Logic (STL), capable of specifying spatial and temporal properties of… ▽ More Learning dynamical systems properties from data provides important insights that help us understand such systems and mitigate undesired outcomes. In this work, we propose a framework for learning spatio-temporal (ST) properties as formal logic specifications from data. We introduce SVM-STL, an extension of Signal Signal Temporal Logic (STL), capable of specifying spatial and temporal properties of a wide range of dynamical systems that exhibit time-varying spatial patterns. Our framework utilizes machine learning techniques to learn SVM-STL specifications from system executions given by sequences of spatial patterns. We present methods to deal with both labeled and unlabeled data. In addition, given system requirements in the form of SVM-STL specifications, we provide an approach for parameter synthesis to find parameters that maximize the satisfaction of such specifications. Our learning framework and parameter synthesis approach are showcased in an example of a reaction-diffusion system. △ Less

Submitted 20 December, 2021; originally announced December 2021.

Comments: 12 pages, submitted to L4DC 2021

MSC Class: I.5.3; I.5.4; B.1.0

Journal ref: PMLR 168:968-980, 2022

Rule-based Evaluation and Optimal Control for Autonomous Driving

Authors: Wei Xiao, Noushin Mehdipour, Anne Collin, Amitai Y. Bin-Nun, Emilio Frazzoli, Radboud Duintjer Tebbens, Calin Belta

Abstract: We develop optimal control strategies for autonomous vehicles (AVs) that are required to meet complex specifications imposed as rules of the road (ROTR) and locally specific cultural expectations of reasonable driving behavior. We formulate these specifications as rules, and specify their priorities by constructing a priority structure, called \underline{T}otal \underline{OR}der over e\underline{Q… ▽ More We develop optimal control strategies for autonomous vehicles (AVs) that are required to meet complex specifications imposed as rules of the road (ROTR) and locally specific cultural expectations of reasonable driving behavior. We formulate these specifications as rules, and specify their priorities by constructing a priority structure, called \underline{T}otal \underline{OR}der over e\underline{Q}uivalence classes (TORQ). We propose a recursive framework, in which the satisfaction of the rules in the priority structure are iteratively relaxed in reverse order of priority. Central to this framework is an optimal control problem, where convergence to desired states is achieved using Control Lyapunov Functions (CLFs) and clearance with other road users is enforced through Control Barrier Functions (CBFs). We present offline and online approaches to this problem. In the latter, the AV has limited sensing range that affects the activation of the rules, and the control is generated using a receding horizon (Model Predictive Control, MPC) approach. We also show how the offline method can be used for after-the-fact (offline) pass/fail evaluation of trajectories - a given trajectory is rejected if we can find a controller producing a trajectory that leads to less violation of the rule priority structure. We present case studies with multiple driving scenarios to demonstrate the effectiveness of the algorithms, and to compare the offline and online versions of our proposed framework. △ Less

Submitted 15 July, 2021; originally announced July 2021.

Comments: under review in TAC, 16 pages. arXiv admin note: substantial text overlap with arXiv:2101.05709

Safe Exploration in Model-based Reinforcement Learning using Control Barrier Functions

Authors: Max H. Cohen, Calin Belta

Abstract: This paper develops a model-based reinforcement learning (MBRL) framework for learning online the value function of an infinite-horizon optimal control problem while obeying safety constraints expressed as control barrier functions (CBFs). Our approach is facilitated by the development of a novel class of CBFs, termed Lyapunov-like CBFs (LCBFs), that retain the beneficial properties of CBFs for de… ▽ More This paper develops a model-based reinforcement learning (MBRL) framework for learning online the value function of an infinite-horizon optimal control problem while obeying safety constraints expressed as control barrier functions (CBFs). Our approach is facilitated by the development of a novel class of CBFs, termed Lyapunov-like CBFs (LCBFs), that retain the beneficial properties of CBFs for develo** minimally-invasive safe control policies while also possessing desirable Lyapunov-like qualities such as positive semi-definiteness. We show how these LCBFs can be used to augment a learning-based control policy to guarantee safety and then leverage this approach to develop a safe exploration framework in a MBRL setting. We demonstrate that our approach can handle more general safety constraints than comparative methods via numerical examples. △ Less

Submitted 19 September, 2022; v1 submitted 16 April, 2021; originally announced April 2021.

Comments: Accepted for publication in Automatica

Journal ref: Automatica, vol. 147, p. 110684, 2023

Neural Network-based Control for Multi-Agent Systems from Spatio-Temporal Specifications

Authors: Suhail Alsalehi, Noushin Mehdipour, Ezio Bartocci, Calin Belta

Abstract: We propose a framework for solving control synthesis problems for multi-agent networked systems required to satisfy spatio-temporal specifications. We use Spatio-Temporal Reach and Escape Logic (STREL) as a specification language. For this logic, we define smooth quantitative semantics, which captures the degree of satisfaction of a formula by a multi-agent team. We use the novel quantitative sema… ▽ More We propose a framework for solving control synthesis problems for multi-agent networked systems required to satisfy spatio-temporal specifications. We use Spatio-Temporal Reach and Escape Logic (STREL) as a specification language. For this logic, we define smooth quantitative semantics, which captures the degree of satisfaction of a formula by a multi-agent team. We use the novel quantitative semantics to map control synthesis problems with STREL specifications to optimization problems and propose a combination of heuristic and gradient-based methods to solve such problems. As this method might not meet the requirements of a real-time implementation, we develop a machine learning technique that uses the results of the off-line optimizations to train a neural network that gives the control inputs at current states. We illustrate the effectiveness of the proposed framework by applying it to a model of a robotic team required to satisfy a spatial-temporal specification under communication constraints. △ Less

Submitted 6 April, 2021; originally announced April 2021.

Comments: 8 pages. Submitted to the CDC 2021

Safe Model-based Control from Signal Temporal Logic Specifications Using Recurrent Neural Networks

Authors: Wenliang Liu, Mirai Nishioka, Calin Belta

Abstract: We propose a policy search approach to learn controllers from specifications given as Signal Temporal Logic (STL) formulae. The system model, which is unknown but assumed to be an affine control system, is learned together with the control policy. The model is implemented as two feedforward neural networks (FNNs) - one for the drift, and one for the control directions. To capture the history depen… ▽ More We propose a policy search approach to learn controllers from specifications given as Signal Temporal Logic (STL) formulae. The system model, which is unknown but assumed to be an affine control system, is learned together with the control policy. The model is implemented as two feedforward neural networks (FNNs) - one for the drift, and one for the control directions. To capture the history dependency of STL specifications, we use a recurrent neural network (RNN) to implement the control policy. In contrast to prevalent model-free methods, the learning approach proposed here takes advantage of the learned model and is more efficient. We use control barrier functions (CBFs) with the learned model to improve the safety of the system. We validate our algorithm via simulations and experiments. The results show that our approach can satisfy the given specification within very few system runs, and can be used for on-line control. △ Less

Submitted 16 November, 2022; v1 submitted 29 March, 2021; originally announced March 2021.

Comments: Submitted to ICRA 2023

Event-Triggered Safety-Critical Control for Systems with Unknown Dynamics

Authors: Wei Xiao, Calin Belta, Christos G. Cassandras

Abstract: This paper addresses the problem of safety-critical control for systems with unknown dynamics. It has been shown that stabilizing affine control systems to desired (sets of) states while optimizing quadratic costs subject to state and control constraints can be reduced to a sequence of quadratic programs (QPs) by using Control Barrier Functions (CBFs) and Control Lyapunov Functions (CLFs). Our rec… ▽ More This paper addresses the problem of safety-critical control for systems with unknown dynamics. It has been shown that stabilizing affine control systems to desired (sets of) states while optimizing quadratic costs subject to state and control constraints can be reduced to a sequence of quadratic programs (QPs) by using Control Barrier Functions (CBFs) and Control Lyapunov Functions (CLFs). Our recently proposed High Order CBFs (HOCBFs) can accommodate constraints of arbitrary relative degree. One of the main challenges in this approach is obtaining accurate system dynamics, which is especially difficult for systems that require online model identification given limited computational resources and system data. In order to approximate the real unmodelled system dynamics, we define adaptive affine control dynamics which are updated based on the error states obtained by real-time sensor measurements. We define a HOCBF for a safety requirement on the unmodelled system based on the adaptive dynamics and error states, and reformulate the safety-critical control problem as the above mentioned QP. Then, we determine the events required to solve the QP in order to guarantee safety. We also derive a condition that guarantees the satisfaction of the HOCBF constraint between events. We illustrate the effectiveness of the proposed framework on an adaptive cruise control problem and compare it with the classical time-driven approach. △ Less

Submitted 29 March, 2021; originally announced March 2021.

Comments: 8 pages, submitted to CDC2021. arXiv admin note: text overlap with arXiv:2011.08248

Experimental Validation of Linear and Nonlinear MPC on an Articulated Unmanned Ground Vehicle

Authors: Erkan Kayacan, Wouter Saeys, Herman Ramon, Calin Belta, Joshua M. Peschel

Abstract: This paper focuses on the trajectory tracking control problem for an articulated unmanned ground vehicle. We propose and compare two approaches in terms of performance and computational complexity. The first uses a nonlinear mathematical model derived from first principles and combines a nonlinear model predictive controller (NMPC) with a nonlinear moving horizon estimator (NMHE) to produce a cont… ▽ More This paper focuses on the trajectory tracking control problem for an articulated unmanned ground vehicle. We propose and compare two approaches in terms of performance and computational complexity. The first uses a nonlinear mathematical model derived from first principles and combines a nonlinear model predictive controller (NMPC) with a nonlinear moving horizon estimator (NMHE) to produce a control strategy. The second is based on an input-state linearization (ISL) of the original model followed by linear model predictive control (LMPC). A fast real-time iteration scheme is proposed, implemented for the NMHE-NMPC framework and benchmarked against the ISL-LMPC framework, which is a traditional and cheap method. The experimental results for a time-based trajectory show that the NMHE-NMPC framework with the proposed real-time iteration scheme gives better trajectory tracking performance than the ISL-LMPC framework and the required computation time is feasible for real-time applications. Moreover, the ISL-LMPC produces results of a quality comparable to the NMHE-NMPC framework at a significantly reduced computational cost. △ Less

Submitted 25 March, 2021; originally announced March 2021.

Journal ref: IEEE/ASME Transactions on Mechatronics, vol. 23, issue 5, 2018

High Order Control Lyapunov-Barrier Functions for Temporal Logic Specifications

Authors: Wei Xiao, Calin A. Belta, Christos G. Cassandras

Abstract: Recent work has shown that stabilizing an affine control system to a desired state while optimizing a quadratic cost subject to state and control constraints can be reduced to a sequence of Quadratic Programs (QPs) by using Control Barrier Functions (CBFs) and Control Lyapunov Functions (CLFs). In our own recent work, we defined High Order CBFs (HOCBFs) for systems and constraints with arbitrary r… ▽ More Recent work has shown that stabilizing an affine control system to a desired state while optimizing a quadratic cost subject to state and control constraints can be reduced to a sequence of Quadratic Programs (QPs) by using Control Barrier Functions (CBFs) and Control Lyapunov Functions (CLFs). In our own recent work, we defined High Order CBFs (HOCBFs) for systems and constraints with arbitrary relative degrees. In this paper, in order to accommodate initial states that do not satisfy the state constraints and constraints with arbitrary relative degree, we generalize HOCBFs to High Order Control Lyapunov-Barrier Functions (HOCLBFs). We also show that the proposed HOCLBFs can be used to guarantee the Boolean satisfaction of Signal Temporal Logic (STL) formulae over the state of the system. We illustrate our approach on a safety-critical optimal control problem (OCP) for a unicycle. △ Less

Submitted 12 February, 2021; originally announced February 2021.

Comments: 9 pages, accepted in ACC 2021

Model-Based Reinforcement Learning for Approximate Optimal Control with Temporal Logic Specifications

Authors: Max Cohen, Calin Belta

Abstract: In this paper we study the problem of synthesizing optimal control policies for uncertain continuous-time nonlinear systems from syntactically co-safe linear temporal logic (scLTL) formulas. We formulate this problem as a sequence of reach-avoid optimal control sub-problems. We show that the resulting hybrid optimal control policy guarantees the satisfaction of a given scLTL formula by constructin… ▽ More In this paper we study the problem of synthesizing optimal control policies for uncertain continuous-time nonlinear systems from syntactically co-safe linear temporal logic (scLTL) formulas. We formulate this problem as a sequence of reach-avoid optimal control sub-problems. We show that the resulting hybrid optimal control policy guarantees the satisfaction of a given scLTL formula by constructing a barrier certificate. Since solving each optimal control problem may be computationally intractable, we take a learning-based approach to approximately solve this sequence of optimal control problems online without requiring full knowledge of the system dynamics. Using Lyapunov-based tools, we develop sufficient conditions under which our approximate solution maintains correctness. Finally, we demonstrate the efficacy of the developed method with a numerical example. △ Less

Submitted 14 April, 2021; v1 submitted 18 January, 2021; originally announced January 2021.

Comments: To appear at the 24th ACM International Conference on Hybrid Systems: Computation and Control

Rule-based Optimal Control for Autonomous Driving

Authors: Wei Xiao, Noushin Mehdipour, Anne Collin, Amitai Bin-Nun, Emilio Frazzoli, Radboud Duintjer Tebbens, Calin Belta

Abstract: We develop optimal control strategies for Autonomous Vehicles (AVs) that are required to meet complex specifications imposed by traffic laws and cultural expectations of reasonable driving behavior. We formulate these specifications as rules, and specify their priorities by constructing a priority structure. We propose a recursive framework, in which the satisfaction of the rules in the priority s… ▽ More We develop optimal control strategies for Autonomous Vehicles (AVs) that are required to meet complex specifications imposed by traffic laws and cultural expectations of reasonable driving behavior. We formulate these specifications as rules, and specify their priorities by constructing a priority structure. We propose a recursive framework, in which the satisfaction of the rules in the priority structure are iteratively relaxed based on their priorities. Central to this framework is an optimal control problem, where convergence to desired states is achieved using Control Lyapunov Functions (CLFs), and safety is enforced through Control Barrier Functions (CBFs). We also show how the proposed framework can be used for after-the-fact, pass / fail evaluation of trajectories - a given trajectory is rejected if we can find a controller producing a trajectory that leads to less violation of the rule priority structure. We present case studies with multiple driving scenarios to demonstrate the effectiveness of the proposed framework. △ Less

Submitted 14 January, 2021; originally announced January 2021.

Comments: accepted in ICCPS2021

Sufficient Conditions for Feasibility of Optimal Control Problems Using Control Barrier Functions

Authors: Wei Xiao, Calin Belta, Christos G. Cassandras

Abstract: It has been shown that satisfying state and control constraints while optimizing quadratic costs subject to desired (sets of) state convergence for affine control systems can be reduced to a sequence of quadratic programs (QPs) by using Control Barrier Functions (CBFs) and Control Lyapunov Functions (CLFs). One of the main challenges in this approach is ensuring the feasibility of these QPs, espec… ▽ More It has been shown that satisfying state and control constraints while optimizing quadratic costs subject to desired (sets of) state convergence for affine control systems can be reduced to a sequence of quadratic programs (QPs) by using Control Barrier Functions (CBFs) and Control Lyapunov Functions (CLFs). One of the main challenges in this approach is ensuring the feasibility of these QPs, especially under tight control bounds and safety constraints of high relative degree. In this paper, we provide sufficient conditions for guranteed feasibility. The sufficient conditions are captured by a single constraint that is enforced by a CBF, which is added to the QPs such that their feasibility is always guaranteed. The additional constraint is designed to be always compatible with the existing constraints, therefore, it cannot make a feasible set of constraints infeasible - it can only increase the overall feasibility. We illustrate the effectiveness of the proposed approach on an adaptive cruise control problem. △ Less

Submitted 16 November, 2020; originally announced November 2020.

Comments: 10 pages, submitted to Automatica

Specifying User Preferences using Weighted Signal Temporal Logic

Authors: Noushin Mehdipour, Cristian-Ioan Vasile, Calin Belta

Abstract: We extend Signal Temporal Logic (STL) to enable the specification of importance and priorities. The extension, called Weighted STL (wSTL), has the same qualitative (Boolean) semantics as STL, but additionally defines weights associated with Boolean and temporal operators that modulate its quantitative semantics (robustness). We show that the robustness of wSTL can be defined as weighted generaliza… ▽ More We extend Signal Temporal Logic (STL) to enable the specification of importance and priorities. The extension, called Weighted STL (wSTL), has the same qualitative (Boolean) semantics as STL, but additionally defines weights associated with Boolean and temporal operators that modulate its quantitative semantics (robustness). We show that the robustness of wSTL can be defined as weighted generalizations of all known compatible robustness functionals (i.e., robustness scores that are recursively defined over formulae) that can take into account the weights in wSTL formulae. We utilize this weighted robustness to distinguish signals with respect to a desired wSTL formula that has sub-formulae with different importance or priorities and time preferences, and demonstrate its usefulness in problems with conflicting tasks where satisfaction of all tasks cannot be achieved. We also employ wSTL robustness in an optimization framework to synthesize controllers that maximize satisfaction of a specification with user specified preferences. △ Less

Submitted 1 October, 2020; originally announced October 2020.

Comments: Submitted to IEEE Control Systems Letters (L-CSS)

Recurrent Neural Network Controllers for Signal Temporal Logic Specifications Subject to Safety Constraints

Authors: Wenliang Liu, Noushin Mehdipour, Calin Belta

Abstract: We propose a framework based on Recurrent Neural Networks (RNNs) to determine an optimal control strategy for a discrete-time system that is required to satisfy specifications given as Signal Temporal Logic (STL) formulae. RNNs can store information of a system over time, thus, enable us to determine satisfaction of the dynamic temporal requirements specified in STL formulae. Given a STL formula,… ▽ More We propose a framework based on Recurrent Neural Networks (RNNs) to determine an optimal control strategy for a discrete-time system that is required to satisfy specifications given as Signal Temporal Logic (STL) formulae. RNNs can store information of a system over time, thus, enable us to determine satisfaction of the dynamic temporal requirements specified in STL formulae. Given a STL formula, a dataset of satisfying system executions and corresponding control policies, we can use RNNs to predict a control policy at each time based on the current and previous states of system. We use Control Barrier Functions (CBFs) to guarantee the safety of the predicted control policy. We validate our theoretical formulation and demonstrate its performance in an optimal control problem subject to partially unknown safety constraints through simulations. △ Less

Submitted 23 September, 2020; originally announced September 2020.

Comments: 7 pages, 4 figures, submitted to IEEE Control Systems Letters (L-CSS) with the option to present it to the ACC 2021

Bridging the Gap between Optimal Trajectory Planning and Safety-Critical Control with Applications to Autonomous Vehicles

Authors: Wei Xiao, Christos G. Cassandras, Calin A. Belta

Abstract: We address the problem of optimizing the performance of a dynamic system while satisfying hard safety constraints at all times. Implementing an optimal control solution is limited by the computational cost required to derive it in real time, especially when constraints become active, as well as the need to rely on simple linear dynamics, simple objective functions, and ignoring noise. The recently… ▽ More We address the problem of optimizing the performance of a dynamic system while satisfying hard safety constraints at all times. Implementing an optimal control solution is limited by the computational cost required to derive it in real time, especially when constraints become active, as well as the need to rely on simple linear dynamics, simple objective functions, and ignoring noise. The recently proposed Control Barrier Function (CBF) method may be used for safety-critical control at the expense of sub-optimal performance. In this paper, we develop a real-time control framework that combines optimal trajectories generated through optimal control with the computationally efficient CBF method providing safety guarantees. We use Hamiltonian analysis to obtain a tractable optimal solution for a linear or linearized system, then employ High Order CBFs (HOCBFs) and Control Lyapunov Functions (CLFs) to account for constraints with arbitrary relative degrees and to track the optimal state, respectively. We further show how to deal with noise in arbitrary relative degree systems. The proposed framework is then applied to the optimal traffic merging problem for Connected and Automated Vehicles (CAVs) where the objective is to jointly minimize the travel time and energy consumption of each CAV subject to speed, acceleration, and speed-dependent safety constraints. In addition, when considering more complex objective functions, nonlinear dynamics and passenger comfort requirements for which analytical optimal control solutions are unavailable, we adapt the HOCBF method to such problems. Simulation examples are included to compare the performance of the proposed framework to optimal solutions (when available) and to a baseline provided by human-driven vehicles with results showing significant improvements in all metrics. △ Less

Submitted 17 August, 2020; originally announced August 2020.

Comments: 15 pages, Provisionally accepted in Automatica

Approximate Optimal Control for Safety-Critical Systems with Control Barrier Functions

Authors: Max Cohen, Calin Belta

Abstract: Control Barrier Functions (CBFs) have become a popular tool for enforcing set invariance in safety-critical control systems. While guaranteeing safety, most CBF approaches are myopic in the sense that they solve an optimization problem at each time step rather than over a long time horizon. This approach may allow a system to get too close to the unsafe set where the optimization problem can becom… ▽ More Control Barrier Functions (CBFs) have become a popular tool for enforcing set invariance in safety-critical control systems. While guaranteeing safety, most CBF approaches are myopic in the sense that they solve an optimization problem at each time step rather than over a long time horizon. This approach may allow a system to get too close to the unsafe set where the optimization problem can become infeasible. Some of these issues can be mitigated by introducing relaxation variables into the optimization problem; however, this compromises convergence to the desired equilibrium point. To address these challenges, we develop an approximate optimal approach to the safety-critical control problem in which the cost of violating safety constraints is directly embedded within the value function. We show that our method is capable of guaranteeing both safety and convergence to a desired equilibrium. Finally, we compare the performance of our method with that of the traditional quadratic programming approach through numerical examples. △ Less

Submitted 10 August, 2020; originally announced August 2020.

Comments: Accepted to IEEE Conference on Decision and Control 2020

Non-uniform Sampled Motion Planning for Continuous-time STL

Authors: Guang Yang, Calin Belta, Roberto Tron

Abstract: This paper presents an offline motion planner for linear cyber-physical systems that satisfy a continuous-time Signal Temporal Logic (STL) specification, in which controls are applied in a Zeroth-order Hold (ZOH) manner. The motion planning problem is formulated as a Mixed-integer Program (MIP) with nonuniform control updates. We develop a novel method to obtain bounds of Control Barrier Functions… ▽ More This paper presents an offline motion planner for linear cyber-physical systems that satisfy a continuous-time Signal Temporal Logic (STL) specification, in which controls are applied in a Zeroth-order Hold (ZOH) manner. The motion planning problem is formulated as a Mixed-integer Program (MIP) with nonuniform control updates. We develop a novel method to obtain bounds of Control Barrier Functions (CBF) and linear predicates to render both spatial and temporal requirements. The theoretical results are validated in numerical examples. △ Less

Submitted 3 June, 2020; originally announced June 2020.

Decentralized Optimal Control in Multi-lane Merging for Connected and Automated Vehicles

Authors: Wei Xiao, Christos G. Cassandras, Calin Belta

Abstract: We address the problem of optimally controlling Connected and Automated Vehicles (CAVs) arriving from two multi-lane roads and merging at multiple points where the objective is to jointly minimize the travel time and energy consumption of each CAV subject to speed-dependent safety constraints, as well as speed and acceleration constraints. This problem was solved in prior work for two single-lane… ▽ More We address the problem of optimally controlling Connected and Automated Vehicles (CAVs) arriving from two multi-lane roads and merging at multiple points where the objective is to jointly minimize the travel time and energy consumption of each CAV subject to speed-dependent safety constraints, as well as speed and acceleration constraints. This problem was solved in prior work for two single-lane roads. A direct extension to multi-lane roads is limited by the computational complexity required to obtain an explicit optimal control solution. Instead, we propose a general framework that converts a multi-lane merging problem into a decentralized optimal control problem for each CAV in a less-conservative way. To accomplish this, we employ a joint optimal control and barrier function method to efficiently get an optimal control for each CAV with guaranteed satisfaction of all constraints. Simulation examples are included to compare the performance of the proposed framework to a baseline provided by human-driven vehicles with results showing significant improvements in both time and energy metrics. △ Less

Submitted 30 May, 2020; originally announced June 2020.

Comments: To appear in ITSC2020, complete version with 8 pages

Compositional Synthesis via a Convex Parameterization of Assume-Guarantee Contracts

Authors: Kasra Ghasemi, Sadra Sadraddini, Calin Belta

Abstract: We develop an assume-guarantee framework for control of large scale linear (time-varying) systems from finite-time reach and avoid or infinite-time invariance specifications. The contracts describe the admissible set of states and controls for individual subsystems. A set of contracts compose correctly if mutual assumptions and guarantees match in a way that we formalize. We propose a rich paramet… ▽ More We develop an assume-guarantee framework for control of large scale linear (time-varying) systems from finite-time reach and avoid or infinite-time invariance specifications. The contracts describe the admissible set of states and controls for individual subsystems. A set of contracts compose correctly if mutual assumptions and guarantees match in a way that we formalize. We propose a rich parameterization of contracts such that the set of parameters that compose correctly is convex. Moreover, we design a potential function of parameters that describes the distance of contracts from a correct composition. Thus, the verification and synthesis for the aggregate system are broken to solving small convex programs for individual subsystems, where correctness is ultimately achieved in a compositional way. Illustrative examples demonstrate the scalability of our method. △ Less

Submitted 15 February, 2020; originally announced February 2020.

Adaptive Control Barrier Functions for Safety-Critical Systems

Authors: Wei Xiao, Calin Belta, Christos G. Cassandras

Abstract: Recent work showed that stabilizing affine control systems to desired (sets of) states while optimizing quadratic costs and observing state and control constraints can be reduced to quadratic programs (QP) by using control barrier functions (CBF) and control Lyapunov functions. In our own recent work, we defined high order CBFs (HOCBFs) to accommodating systems and constraints with arbitrary relat… ▽ More Recent work showed that stabilizing affine control systems to desired (sets of) states while optimizing quadratic costs and observing state and control constraints can be reduced to quadratic programs (QP) by using control barrier functions (CBF) and control Lyapunov functions. In our own recent work, we defined high order CBFs (HOCBFs) to accommodating systems and constraints with arbitrary relative degrees, and a penalty method to increase the feasibility of the corresponding QPs. In this paper, we introduce adaptive CBF (AdaCBFs) that can accommodate time-varying control bounds and dynamics noise, and also address the feasibility problem. Central to our approach is the introduction of penalty functions in the definition of an AdaCBF and the definition of auxiliary dynamics for these penalty functions that are HOCBFs and are stabilized by CLFs. We demonstrate the advantages of the proposed method by applying it to a cruise control problem with different road surfaces, tires slip**, and dynamics noise. △ Less

Submitted 11 February, 2020; originally announced February 2020.

Comments: 11 pages, 7 figures, submitted to CDC2020

Feasibility-Guided Learning for Robust Control in Constrained Optimal Control Problems

Authors: Wei Xiao, Calin A. Belta, Christos G. Cassandras

Abstract: Optimal control problems with constraints ensuring safety and convergence to desired states can be mapped onto a sequence of real time optimization problems through the use of Control Barrier Functions (CBFs) and Control Lyapunov Functions (CLFs). One of the main challenges in these approaches is ensuring the feasibility of the resulting quadratic programs (QPs) if the system is affine in controls… ▽ More Optimal control problems with constraints ensuring safety and convergence to desired states can be mapped onto a sequence of real time optimization problems through the use of Control Barrier Functions (CBFs) and Control Lyapunov Functions (CLFs). One of the main challenges in these approaches is ensuring the feasibility of the resulting quadratic programs (QPs) if the system is affine in controls. The recently proposed penalty method has the potential to improve the existence of feasible solutions to such problems. In this paper, we further improve the feasibility robustness (i.e., feasibility maintenance in the presence of time-varying and unknown unsafe sets) through the definition of a High Order CBF (HOCBF) that works for arbitrary relative degree constraints; this is achieved by a proposed feasibility-guided learning approach. Specifically, we apply machine learning techniques to classify the parameter space of a HOCBF into feasible and infeasible sets, and get a differentiable classifier that is then added to the learning process. The proposed feasibility-guided learning approach is compared with the gradient-descent method on a robot control problem. The simulation results show an improved ability of the feasibility-guided learning approach over the gradient-decent method to determine the optimal parameters in the definition of a HOCBF for the feasibility robustness, as well as show the potential of the CBF method for robot safe navigation in an unknown environment. △ Less

Submitted 6 December, 2019; originally announced December 2019.

Comments: 8 pages, submitted to L-CSS

Compositional Synthesis of Decentralized Robust Set-Invariance Controllers for Large-scale Linear Systems

Authors: Kasra Ghasemi, Sadra Sadraddini, Calin Belta

Abstract: Ensuring constraint satisfaction in large-scale systems with hard constraints is vital in many safety critical systems. The challenge is to design controllers that are efficiently synthesized offline, easily implementable online, and provide formal correctness guarantees. In this paper, we provide a method to compute correct-by-construction controllers for a network of coupled linear systems with… ▽ More Ensuring constraint satisfaction in large-scale systems with hard constraints is vital in many safety critical systems. The challenge is to design controllers that are efficiently synthesized offline, easily implementable online, and provide formal correctness guarantees. In this paper, we provide a method to compute correct-by-construction controllers for a network of coupled linear systems with additive bounded disturbances such that i) the design of the controllers is fully compositional - we use an optimization-based approach that iteratively computes subsystem-level assume-guarantee contracts in the form of robust control invariant sets; and ii) the controllers are decentralized hence online implementation requires only the local state information. We present illustrative examples, including a case study on a system with 1000 dimensions. △ Less

Submitted 17 September, 2019; v1 submitted 13 September, 2019; originally announced September 2019.

Control from Signal Temporal Logic Specifications with Smooth Cumulative Quantitative Semantics

Authors: Iman Haghighi, Noushin Mehdipour, Ezio Bartocci, Calin Belta

Abstract: We present a framework to synthesize control policies for nonlinear dynamical systems from complex temporal constraints specified in a rich temporal logic called Signal Temporal Logic (STL). We propose a novel smooth and differentiable STL quantitative semantics called cumulative robustness, and efficiently compute control policies through a series of smooth optimization problems that are solved u… ▽ More We present a framework to synthesize control policies for nonlinear dynamical systems from complex temporal constraints specified in a rich temporal logic called Signal Temporal Logic (STL). We propose a novel smooth and differentiable STL quantitative semantics called cumulative robustness, and efficiently compute control policies through a series of smooth optimization problems that are solved using gradient ascent algorithms. Furthermore, we demonstrate how these techniques can be incorporated in a model predictive control framework in order to synthesize control policies over long time horizons. The advantages of combining the cumulative robustness function with smooth optimization methods as well as model predictive control are illustrated in case studies. △ Less

Submitted 25 April, 2019; originally announced April 2019.

Control Barrier Functions for Systems with High Relative Degree

Authors: Wei Xiao, Calin Belta

Abstract: This paper extends control barrier functions (CBFs) to high order control barrier functions (HOCBFs) that can be used for high relative degree constraints. The proposed HOCBFs are more general than recently proposed (exponential) HOCBFs. We introduce high order barrier functions (HOBF), and show that their satisfaction of Lyapunov-like conditions implies the forward invariance of the intersection… ▽ More This paper extends control barrier functions (CBFs) to high order control barrier functions (HOCBFs) that can be used for high relative degree constraints. The proposed HOCBFs are more general than recently proposed (exponential) HOCBFs. We introduce high order barrier functions (HOBF), and show that their satisfaction of Lyapunov-like conditions implies the forward invariance of the intersection of a series of sets. We then introduce HOCBF, and show that any control input that satisfies the HOCBF constraints renders the intersection of a series of sets forward invariant. We formulate optimal control problems with constraints given by HOCBF and control Lyapunov functions (CLF) and analyze the influence of the choice of the class $\mathcal{K}$ functions used in the definition of the HOCBF on the size of the feasible control region. We also provide a promising method to address the conflict between HOCBF constraints and control limitations by penalizing the class $\mathcal{K}$ functions. We illustrate the proposed method on an adaptive cruise control problem. △ Less

Submitted 13 March, 2019; v1 submitted 11 March, 2019; originally announced March 2019.

Comments: 9 pages, 7 figures, submitted to CDC19

Continuous-time Signal Temporal Logic Planning with Control Barrier Function

Authors: Guang Yang, Roberto Tron, Calin Belta

Abstract: Temporal Logic (TL) guided control problems have gained interests in recent years. By using the TL, one can specify a wide range of temporal constraints on the system and is widely used in cyber-physical systems. On the other hand, Control Barrier Functions have also gained interests in the context of safety critical applications. However, most of the existing approaches only focus on discrete-tim… ▽ More Temporal Logic (TL) guided control problems have gained interests in recent years. By using the TL, one can specify a wide range of temporal constraints on the system and is widely used in cyber-physical systems. On the other hand, Control Barrier Functions have also gained interests in the context of safety critical applications. However, most of the existing approaches only focus on discrete-time dynamical systems. In this paper, we propose an offline trajectory planner for linear systems subject to safety and temporal specifications. Such specifications can be expressed as logical junctions or disjunctions of linear CBFs, or as STL specifications with linear predicates. Our planner produces trajectories that are valid in continuous time, while assuming only discrete-time control updates and arbitrary time interval in the STL formula. Our planner is based on a Mixed Integer Quadratic Programming (MIQP) formulation, where the linear STL predicates are encoded as set of linear constraints to guarantee satisfaction at on a finite discrete set of time instants, while we use CBFs to derive constraints that guarantee continuous satisfaction between time instants. Moreover, we have shown the predicates can be encoded as time-based CBF constraints for system with any relative degrees. We validate our theoretical results and formulation through numerical simulations. △ Less

Submitted 9 March, 2019; originally announced March 2019.

Self-triggered Control for Safety Critical Systems using Control Barrier Functions

Authors: Guang Yang, Calin Belta, Roberto Tron

Abstract: We propose a real-time control strategy that combines self-triggered control with Control Lyapunov Functions (CLF) and Control Barrier Functions (CBF). Similar to related works proposing CLF-CBF-based controllers, the computation of the controller is achieved by solving a Quadratic Program (QP). However, we propose a Zeroth-Order Hold (ZOH) implementation of the controller that overcomes the main… ▽ More We propose a real-time control strategy that combines self-triggered control with Control Lyapunov Functions (CLF) and Control Barrier Functions (CBF). Similar to related works proposing CLF-CBF-based controllers, the computation of the controller is achieved by solving a Quadratic Program (QP). However, we propose a Zeroth-Order Hold (ZOH) implementation of the controller that overcomes the main limitations of traditional approaches based on periodic controllers, i.e., unnecessary controller updates and potential violations of the safety constraints. Central to our approach is the novel notion of safe period, which enforces a strong safety guarantee for implementing ZOH control. In addition, we prove that the system does not exhibit a Zeno behavior as it approaches the desired equilibrium. △ Less

Submitted 8 March, 2019; originally announced March 2019.

Distributed Robust Set-Invariance for Interconnected Linear Systems

Authors: Sadra Sadraddini, Calin Belta

Abstract: We introduce a class of distributed control policies for networks of discrete-time linear systems with polytopic additive disturbances. The objective is to restrict the network-level state and controls to user-specified polyhedral sets for all times. This problem arises in many safety-critical applications. We consider two problems. First, given a communication graph characterizing the structure o… ▽ More We introduce a class of distributed control policies for networks of discrete-time linear systems with polytopic additive disturbances. The objective is to restrict the network-level state and controls to user-specified polyhedral sets for all times. This problem arises in many safety-critical applications. We consider two problems. First, given a communication graph characterizing the structure of the information flow in the network, we find the optimal distributed control policy by solving a single linear program. Second, we find the sparsest communication graph required for the existence of a distributed invariance-inducing control policy. Illustrative examples, including one on platooning, are presented. △ Less

Submitted 28 September, 2017; originally announced September 2017.

Comments: 8 Pages. Submitted to American Control Conference (ACC), 2018

Provably Safe Cruise Control of Vehicular Platoons

Authors: Sadra Sadraddini, Sivaranjani S, Vijay Gupta, Calin Belta

Abstract: We synthesize performance-aware safe cruise control policies for longitudinal motion of platoons of autonomous vehicles. Using set-invariance theories, we guarantee infinite-time collision avoidance in the presence of bounded additive disturbances, while ensuring that the length and the cruise speed of the platoon are bounded within specified ranges. We propose (i) a centralized control policy, an… ▽ More We synthesize performance-aware safe cruise control policies for longitudinal motion of platoons of autonomous vehicles. Using set-invariance theories, we guarantee infinite-time collision avoidance in the presence of bounded additive disturbances, while ensuring that the length and the cruise speed of the platoon are bounded within specified ranges. We propose (i) a centralized control policy, and (ii) a distributed control policy, where each vehicle's control decision depends solely on its relative kinematics with respect to the platoon leader. Numerical examples are included. △ Less

Submitted 5 June, 2017; originally announced June 2017.

Comments: 6 Pages. Accepted to appear in IEEE Control Systems Letters (L-CSS)

Formal Methods for Adaptive Control of Dynamical Systems

Authors: Sadra Sadraddini, Calin Belta

Abstract: We develop a method to control discrete-time systems with constant but initially unknown parameters from linear temporal logic (LTL) specifications. We introduce the notions of (non-deterministic) parametric and adaptive transition systems and show how to use tools from formal methods to compute adaptive control strategies for finite systems. For infinite systems, we first compute abstractions in… ▽ More We develop a method to control discrete-time systems with constant but initially unknown parameters from linear temporal logic (LTL) specifications. We introduce the notions of (non-deterministic) parametric and adaptive transition systems and show how to use tools from formal methods to compute adaptive control strategies for finite systems. For infinite systems, we first compute abstractions in the form of parametric finite quotient transition systems and then apply the techniques for finite systems. Unlike traditional adaptive control methods, our approach is correct by design, does not require a reference model, and can deal with a much wider range of systems and specifications. Illustrative case studies are included. △ Less

Submitted 22 March, 2017; originally announced March 2017.

Comments: 8 Pages. Submitted to CDC 2017

Formal Synthesis of Control Strategies for Positive Monotone Systems

Authors: Sadra Sadraddini, Calin Belta

Abstract: We design controllers from formal specifications for positive discrete-time monotone systems that are subject to bounded disturbances. Such systems are widely used to model the dynamics of transportation and biological networks. The specifications are described using signal temporal logic (STL), which can express a broad range of temporal properties. We formulate the problem as a mixed-integer lin… ▽ More We design controllers from formal specifications for positive discrete-time monotone systems that are subject to bounded disturbances. Such systems are widely used to model the dynamics of transportation and biological networks. The specifications are described using signal temporal logic (STL), which can express a broad range of temporal properties. We formulate the problem as a mixed-integer linear program (MILP) and show that under the assumptions made in this paper, which are not restrictive for traffic applications, the existence of open-loop control policies is sufficient and almost necessary to ensure the satisfaction of STL formulas. We establish a relation between satisfaction of STL formulas in infinite time and set-invariance theories and provide an efficient method to compute robust control invariant sets in high dimensions. We also develop a robust model predictive framework to plan controls optimally while ensuring the satisfaction of the specification. Illustrative examples and a traffic management case study are included. △ Less

Submitted 18 March, 2018; v1 submitted 27 February, 2017; originally announced February 2017.

Comments: To appear in IEEE Transactions on Automatic Control (TAC) (2018), 16 pages, double column

Q-Learning for Robust Satisfaction of Signal Temporal Logic Specifications

Authors: Derya Aksaray, Austin Jones, Zhaodan Kong, Mac Schwager, Calin Belta

Abstract: This paper addresses the problem of learning optimal policies for satisfying signal temporal logic (STL) specifications by agents with unknown stochastic dynamics. The system is modeled as a Markov decision process, in which the states represent partitions of a continuous space and the transition probabilities are unknown. We formulate two synthesis problems where the desired STL specification is… ▽ More This paper addresses the problem of learning optimal policies for satisfying signal temporal logic (STL) specifications by agents with unknown stochastic dynamics. The system is modeled as a Markov decision process, in which the states represent partitions of a continuous space and the transition probabilities are unknown. We formulate two synthesis problems where the desired STL specification is enforced by maximizing the probability of satisfaction, and the expected robustness degree, that is, a measure quantifying the quality of satisfaction. We discuss that Q-learning is not directly applicable to these problems because, based on the quantitative semantics of STL, the probability of satisfaction and expected robustness degree are not in the standard objective form of Q-learning. To resolve this issue, we propose an approximation of STL synthesis problems that can be solved via Q-learning, and we derive some performance bounds for the policies obtained by the approximate approach. The performance of the proposed method is demonstrated via simulations. △ Less

Submitted 23 September, 2016; originally announced September 2016.

Comments: This paper is accepted to IEEE CDC 2016

Robotic Swarm Control from Spatio-Temporal Specifications

Authors: Iman Haghighi, Sadra Sadraddini, Calin Belta

Abstract: In this paper, we study the problem of controlling a two-dimensional robotic swarm with the purpose of achieving high level and complex spatio-temporal patterns. We use a rich spatio-temporal logic that is capable of describing a wide range of time varying and complex spatial configurations, and develop a method to encode such formal specifications as a set of mixed integer linear constraints, whi… ▽ More In this paper, we study the problem of controlling a two-dimensional robotic swarm with the purpose of achieving high level and complex spatio-temporal patterns. We use a rich spatio-temporal logic that is capable of describing a wide range of time varying and complex spatial configurations, and develop a method to encode such formal specifications as a set of mixed integer linear constraints, which are incorporated into a mixed integer linear programming problem. We plan trajectories for each individual robot such that the whole swarm satisfies the spatio-temporal requirements, while optimizing total robot movement and/or a metric that shows how strongly the swarm trajectory resembles given spatio-temporal behaviors. An illustrative case study is included. △ Less

Submitted 20 September, 2016; originally announced September 2016.

Comments: A shorter version of this paper is going to be published at the proceedings of the 55th international conference on decision and control (CDC 2016)