Showing 1–50 of 113 results for author: Cassandras, C G

Robust Optimal Lane-changing Control for Connected Autonomous Vehicles in Mixed Traffic

Authors: Anni Li, Andres S. Chavez Armijos, Christos G. Cassandras

Abstract: We derive time and energy-optimal policies for a Connected Autonomous Vehicle (CAV) to execute lane change maneuvers in mixed traffic, i.e., in the presence of both CAVs and Human Driven Vehicles (HDVs). These policies are also shown to be robust with respect to the unpredictable behavior of HDVs by exploiting CAV cooperation which can eliminate or greatly reduce the interaction between CAVs and H… ▽ More We derive time and energy-optimal policies for a Connected Autonomous Vehicle (CAV) to execute lane change maneuvers in mixed traffic, i.e., in the presence of both CAVs and Human Driven Vehicles (HDVs). These policies are also shown to be robust with respect to the unpredictable behavior of HDVs by exploiting CAV cooperation which can eliminate or greatly reduce the interaction between CAVs and HDVs. We derive a simple threshold-based criterion on the initial relative distance between two cooperating CAVs based on which an optimal policy is selected such that the lane-changing CAV merges ahead of a cooperating CAV in the target lane; in this case, the lane-changing CAV's trajectory becomes independent of HDV behavior. Otherwise, the interaction between CAVs and neighboring HDVs is formulated as a bilevel optimization problem with an appropriate behavioral model for an HDV, and an iterated best response (IBR) method is used to determine an equilibrium. We demonstrate the convergence of the IBR process under certain conditions. Furthermore, Control Barrier Functions (CBFs) are implemented to ensure the robustness of lane-changing behaviors by guaranteeing safety in both longitudinal and lateral directions despite HDV disturbances. Simulation results validate the effectiveness of our CAV controllers in terms of cost, safety guarantees, and limited disruption to traffic flow. Additionally, we demonstrate the robustness of the lane-changing behaviors in the presence of uncontrollable HDVs. △ Less

Submitted 15 March, 2024; originally announced June 2024.

Comments: arXiv admin note: text overlap with arXiv:2303.16948

Towards Achieving Cooperation Compliance of Human Drivers in Mixed Traffic

Authors: Anni Li, Christos G. Cassandras

Abstract: We consider a mixed-traffic environment in transportation systems, where Connected and Automated Vehicles (CAVs) coexist with potentially non-cooperative Human-Driven Vehicles (HDVs). We develop a cooperation compliance control framework to incentivize HDVs to align their behavior with socially optimal objectives using a ``refundable toll'' scheme so as to achieve a desired compliance probability… ▽ More We consider a mixed-traffic environment in transportation systems, where Connected and Automated Vehicles (CAVs) coexist with potentially non-cooperative Human-Driven Vehicles (HDVs). We develop a cooperation compliance control framework to incentivize HDVs to align their behavior with socially optimal objectives using a ``refundable toll'' scheme so as to achieve a desired compliance probability for all non-compliant HDVs through a feedback control mechanism combining global with local (individual) components. We apply this scheme to the lane-changing problem, where a ``Social Planner'' provides references to the HDVs, measures their state errors, and induces cooperation compliance for safe lane-changing through a refundable toll approach. Simulation results are included to show the effectiveness of our cooperation compliance controller in terms of improved compliance and lane-changing maneuver safety and efficiency when non-cooperative HDVs are present. △ Less

Submitted 27 May, 2024; originally announced May 2024.

Scalable Adaptive Traffic Light Control Over a Traffic Network Including Turns, Transit Delays, and Blocking

Authors: Yingqing Chen, Christos G. Cassandras

Abstract: We develop adaptive data-driven traffic light controllers for a grid-like traffic network considering straight, left-turn, and right-turn traffic flows. The analysis incorporates transit delays and blocking effects on vehicle movements between neighboring intersections. Using a stochastic hybrid system model with parametric traffic light controllers, we use Infinitesimal Perturbation Analysis (IPA… ▽ More We develop adaptive data-driven traffic light controllers for a grid-like traffic network considering straight, left-turn, and right-turn traffic flows. The analysis incorporates transit delays and blocking effects on vehicle movements between neighboring intersections. Using a stochastic hybrid system model with parametric traffic light controllers, we use Infinitesimal Perturbation Analysis (IPA) to derive a data-driven cost gradient estimator with respect to controllable parameters. We then iteratively adjust them through an online gradient-based algorithm to improve performance metrics. By integrating a flexible modeling framework to represent diverse intersection and traffic network configurations with event-driven IPA-based adaptive controllers, we develop a general scalable, adaptive framework for real-time traffic light control in multi-intersection traffic networks. △ Less

Submitted 26 April, 2024; originally announced April 2024.

Comments: arXiv admin note: substantial text overlap with arXiv:2305.09024

Online Trajectory Optimization for Persistent Monitoring Problems in Partitioned Environments

Authors: Jonas Hall, Christos G. Cassandras, Sean B. Andersson

Abstract: We consider the problem of using an autonomous agent to persistently monitor a collection of dynamic targets distributed in an environment. We generalize existing work by allowing the agent's dynamics to vary throughout the environment, leading to a hybrid dynamical system. This introduces an additional layer of complexity towards the planning portion of the problem: we must not only identify in w… ▽ More We consider the problem of using an autonomous agent to persistently monitor a collection of dynamic targets distributed in an environment. We generalize existing work by allowing the agent's dynamics to vary throughout the environment, leading to a hybrid dynamical system. This introduces an additional layer of complexity towards the planning portion of the problem: we must not only identify in which order to visit the points of interest, but also in which order to traverse the regions. We design an offline high-level sequence planner together with an online trajectory optimizer realizing the computed visiting sequence. We provide numerical experiments to illustrate the performance of our approach. △ Less

Submitted 28 March, 2024; originally announced March 2024.

Comments: all code available at https://github.com/hallfjonas/hytoperm

Reinforcement Learning-based Receding Horizon Control using Adaptive Control Barrier Functions for Safety-Critical Systems

Authors: Ehsan Sabouni, H. M. Sabbir Ahmad, Vittorio Giammarino, Christos G. Cassandras, Ioannis Ch. Paschalidis, Wenchao Li

Abstract: Optimal control methods provide solutions to safety-critical problems but easily become intractable. Control Barrier Functions (CBFs) have emerged as a popular technique that facilitates their solution by provably guaranteeing safety, through their forward invariance property, at the expense of some performance loss. This approach involves defining a performance objective alongside CBF-based safet… ▽ More Optimal control methods provide solutions to safety-critical problems but easily become intractable. Control Barrier Functions (CBFs) have emerged as a popular technique that facilitates their solution by provably guaranteeing safety, through their forward invariance property, at the expense of some performance loss. This approach involves defining a performance objective alongside CBF-based safety constraints that must always be enforced. Unfortunately, both performance and solution feasibility can be significantly impacted by two key factors: (i) the selection of the cost function and associated parameters, and (ii) the calibration of parameters within the CBF-based constraints, which capture the trade-off between performance and conservativeness. %as well as infeasibility. To address these challenges, we propose a Reinforcement Learning (RL)-based Receding Horizon Control (RHC) approach leveraging Model Predictive Control (MPC) with CBFs (MPC-CBF). In particular, we parameterize our controller and use bilevel optimization, where RL is used to learn the optimal parameters while MPC computes the optimal control input. We validate our method by applying it to the challenging automated merging control problem for Connected and Automated Vehicles (CAVs) at conflicting roadways. Results demonstrate improved performance and a significant reduction in the number of infeasible cases compared to traditional heuristic approaches used for tuning CBF-based controllers, showcasing the effectiveness of the proposed method. △ Less

Submitted 25 March, 2024; originally announced March 2024.

Spline Trajectory Tracking and Obstacle Avoidance for Mobile Agents via Convex Optimization

Authors: Akua Dickson, Christos G. Cassandras, Roberto Tron

Abstract: We propose an output feedback control-based motion planning technique for agents to enable them to converge to a specified polynomial trajectory while imposing a set of safety constraints on our controller to avoid collisions within the free configuration space (polygonal environment). To achieve this, we 1) decompose our polygonal environment into different overlap** cells 2) write out our poly… ▽ More We propose an output feedback control-based motion planning technique for agents to enable them to converge to a specified polynomial trajectory while imposing a set of safety constraints on our controller to avoid collisions within the free configuration space (polygonal environment). To achieve this, we 1) decompose our polygonal environment into different overlap** cells 2) write out our polynomial trajectories as the output of a reference dynamical system with given initial conditions 3) formulate convergence and safety constraints as Linear Matrix Inequalities (LMIs) on our controller using Control Lyapunov Functions (CLFs) and Control Barrier Functions (CBFs) and 4) solve a semi-definite programming (SDP) problem with convergence and safety constraints imposed to synthesize a controller for each convex cell. Extensive simulations are included to test our motion planning method under different initial conditions and different reference trajectories. The synthesized controller is robust to changes in initial conditions and is always safe relative to the boundaries of the polygonal environment. △ Less

Submitted 25 March, 2024; originally announced March 2024.

Performance-Guaranteed Solutions for Multi-Agent Optimal Coverage Problems using Submodularity, Curvature, and Greedy Algorithms

Authors: Shirantha Welikala, Christos G. Cassandras

Abstract: We consider a class of multi-agent optimal coverage problems in which the goal is to determine the optimal placement of a group of agents in a given mission space so that they maximize a coverage objective that represents a blend of individual and collaborative event detection capabilities. This class of problems is extremely challenging due to the non-convex nature of the mission space and of the… ▽ More We consider a class of multi-agent optimal coverage problems in which the goal is to determine the optimal placement of a group of agents in a given mission space so that they maximize a coverage objective that represents a blend of individual and collaborative event detection capabilities. This class of problems is extremely challenging due to the non-convex nature of the mission space and of the coverage objective. With this motivation, greedy algorithms are often used as means of getting feasible coverage solutions efficiently. Even though such greedy solutions are suboptimal, the submodularity (diminishing returns) property of the coverage objective can be exploited to provide performance bound guarantees. Moreover, we show that improved performance bound guarantees (beyond the standard (1-1/e) performance bound) can be established using various curvature measures of the coverage problem. In particular, we provide a brief review of all existing popular applicable curvature measures, including a recent curvature measure that we proposed, and discuss their effectiveness and computational complexity, in the context of optimal coverage problems. We also propose novel computationally efficient techniques to estimate some curvature measures. Finally, we provide several numerical results to support our findings and propose several potential future research directions. △ Less

Submitted 22 March, 2024; v1 submitted 20 March, 2024; originally announced March 2024.

Comments: Will be submitted to CDC 2024

Optimal Sequencing and Motion Control in a Roundabout with Safety Guarantees

Authors: Yingqing Chen, Christos G. Cassandras, Kaiyuan Xu

Abstract: This paper develops a controller for Connected and Automated Vehicles (CAVs) traversing a single-lane roundabout. The controller simultaneously determines the optimal sequence and associated optimal motion control jointly minimizing travel time and energy consumption while providing speed-dependent safety guarantees, as well as satisfying velocity and acceleration constraints. This is achieved by… ▽ More This paper develops a controller for Connected and Automated Vehicles (CAVs) traversing a single-lane roundabout. The controller simultaneously determines the optimal sequence and associated optimal motion control jointly minimizing travel time and energy consumption while providing speed-dependent safety guarantees, as well as satisfying velocity and acceleration constraints. This is achieved by integrating (a) Model Predictive Control (MPC) to enable receding horizon optimization with (b) Control Lyapunov-Barrier Functions (CLBFs) to guarantee convergence to a safe set in finite time, thus providing an extended stability region compared to the use of classic Control Barrier Functions (CBFs). The proposed MPC-CLBF framework addresses both infeasibility and myopic control issues commonly encountered when controlling CAVs over multiple interconnected control zones in a traffic network, which has been a limitation of prior work on CAVs going through roundabouts, while still providing safety guarantees. Simulations under varying traffic demands demonstrate the controller's effectiveness and stability. △ Less

Submitted 19 March, 2024; v1 submitted 14 March, 2024; originally announced March 2024.

Secure Control of Connected and Automated Vehicles Using Trust-Aware Robust Event-Triggered Control Barrier Functions

Authors: H M Sabbir Ahmad, Ehsan Sabouni, Akua Dickson, Wei Xiao, Christos G. Cassandras, Wenchao Li

Abstract: We address the security of a network of Connected and Automated Vehicles (CAVs) cooperating to safely navigate through a conflict area (e.g., traffic intersections, merging roadways, roundabouts). Previous studies have shown that such a network can be targeted by adversarial attacks causing traffic jams or safety violations ending in collisions. We focus on attacks targeting the V2X communication… ▽ More We address the security of a network of Connected and Automated Vehicles (CAVs) cooperating to safely navigate through a conflict area (e.g., traffic intersections, merging roadways, roundabouts). Previous studies have shown that such a network can be targeted by adversarial attacks causing traffic jams or safety violations ending in collisions. We focus on attacks targeting the V2X communication network used to share vehicle data and consider as well uncertainties due to noise in sensor measurements and communication channels. To combat these, motivated by recent work on the safe control of CAVs, we propose a trust-aware robust event-triggered decentralized control and coordination framework that can provably guarantee safety. We maintain a trust metric for each vehicle in the network computed based on their behavior and used to balance the tradeoff between conservativeness (when deeming every vehicle as untrustworthy) and guaranteed safety and security. It is important to highlight that our framework is invariant to the specific choice of the trust framework. Based on this framework, we propose an attack detection and mitigation scheme which has twofold benefits: (i) the trust framework is immune to false positives, and (ii) it provably guarantees safety against false positive cases. We use extensive simulations (in SUMO and CARLA) to validate the theoretical guarantees and demonstrate the efficacy of our proposed scheme to detect and mitigate adversarial attacks. △ Less

Submitted 25 March, 2024; v1 submitted 4 January, 2024; originally announced January 2024.

Comments: arXiv admin note: substantial text overlap with arXiv:2305.16818

Safe Optimal Interactions Between Automated and Human-Driven Vehicles in Mixed Traffic with Event-triggered Control Barrier Functions

Authors: Anni Li, Christos G. Cassandras, Wei Xiao

Abstract: This paper studies safe driving interactions between Human-Driven Vehicles (HDVs) and Connected and Automated Vehicles (CAVs) in mixed traffic where the dynamics and control policies of HDVs are unknown and hard to predict. In order to address this challenge, we employ event-triggered Control Barrier Functions (CBFs) to estimate the HDV model online, construct data-driven and state-feedback safety… ▽ More This paper studies safe driving interactions between Human-Driven Vehicles (HDVs) and Connected and Automated Vehicles (CAVs) in mixed traffic where the dynamics and control policies of HDVs are unknown and hard to predict. In order to address this challenge, we employ event-triggered Control Barrier Functions (CBFs) to estimate the HDV model online, construct data-driven and state-feedback safety controllers, and transform constrained optimal control problems for CAVs into a sequence of event-triggered quadratic programs. We show that we can ensure collision-free between HDVs and CAVs and demonstrate the robustness and flexibility of our framework on different types of human drivers in lane-changing scenarios while guaranteeing safety with human-in-the-loop interactions. △ Less

Submitted 30 September, 2023; originally announced October 2023.

Scaling up the Optimal Safe Control of Connected and Automated Vehicles to a Traffic Network: A Hierarchical Framework of Modular Control Zones

Authors: Kaiyuan Xu, Christos G. Cassandras

Abstract: We consider the problem of scaling up optimal and safe controllers for Connected and Automated Vehicles (CAVs) from a single Control Zone (CZ) around a traffic conflict area to an entire network. The goal is to jointly minimize travel time and energy consumption for all CAVs, while providing speed-dependent safety guarantees within a CZ and satisfying velocity and acceleration constraints. A hiera… ▽ More We consider the problem of scaling up optimal and safe controllers for Connected and Automated Vehicles (CAVs) from a single Control Zone (CZ) around a traffic conflict area to an entire network. The goal is to jointly minimize travel time and energy consumption for all CAVs, while providing speed-dependent safety guarantees within a CZ and satisfying velocity and acceleration constraints. A hierarchical modular CZ framework is developed consisting of a lower level where decentralized controllers are used that combine Optimal control and Control Barrier Functions (OCBF) and a higher level where a feedback flow controller is proposed to coordinate adjacent CZs. The flow controller is parameterized by a terminal velocity constraint that serves as the interface between CZs. Simulation results show that the proposed modular control zone framework outperforms a direct extension of the OCBF framework to multiple CZs without any flow control. △ Less

Submitted 6 August, 2023; originally announced August 2023.

Comments: arXiv admin note: substantial text overlap with arXiv:2203.04348

Optimal Control of Connected Automated Vehicles with Event-Triggered Control Barrier Functions: a Test Bed for Safe Optimal Merging

Authors: Ehsan Sabouni, H. M. Sabbir Ahmad, Wei Xiao, Christos G. Cassandras, Wenchao Li

Abstract: We address the problem of controlling Connected and Automated Vehicles (CAVs) in conflict areas of a traffic network subject to hard safety constraints. It has been shown that such problems can be solved through a combination of tractable optimal control problems and Control Barrier Functions (CBFs) that guarantee the satisfaction of all constraints. These solutions can be reduced to a sequence of… ▽ More We address the problem of controlling Connected and Automated Vehicles (CAVs) in conflict areas of a traffic network subject to hard safety constraints. It has been shown that such problems can be solved through a combination of tractable optimal control problems and Control Barrier Functions (CBFs) that guarantee the satisfaction of all constraints. These solutions can be reduced to a sequence of Quadratic Programs (QPs) which are efficiently solved on line over discrete time steps. However, guaranteeing the feasibility of the CBF-based QP method within each discretized time interval requires the careful selection of time steps which need to be sufficiently small. This creates computational requirements and communication rates between agents which may hinder the controller's application to real CAVs. In this paper, we overcome this limitation by adopting an event-triggered approach for CAVs in a conflict area such that the next QP is triggered by properly defined events with a safety guarantee. We present a laboratory-scale test bed we have developed to emulate merging roadways using mobile robots as CAVs which can be used to demonstrate how the event-triggered scheme is computationally efficient and can handle measurement uncertainties and noise compared to time-driven control while guaranteeing safety. △ Less

Submitted 2 June, 2023; originally announced June 2023.

Comments: arXiv admin note: substantial text overlap with arXiv:2203.12089, arXiv:2209.13053

Maximizing Safety and Efficiency for Cooperative Lane-Changing: A Minimally Disruptive Approach

Authors: Andres S. Chavez Armijos, Anni Li, Christos G. Cassandras

Abstract: This paper addresses cooperative lane-changing maneuvers in mixed traffic, aiming to minimize traffic flow disruptions while accounting for uncooperative vehicles. The proposed approach adopts controllers combining Optimal control with Control Barrier Functions (OCBF controllers) which guarantee spatio-temporal constraints through the use of fixed-time convergence. Additionally, we introduce robus… ▽ More This paper addresses cooperative lane-changing maneuvers in mixed traffic, aiming to minimize traffic flow disruptions while accounting for uncooperative vehicles. The proposed approach adopts controllers combining Optimal control with Control Barrier Functions (OCBF controllers) which guarantee spatio-temporal constraints through the use of fixed-time convergence. Additionally, we introduce robustness to disturbances by deriving a method for handling worst-case disturbances using the dual of a linear programming problem. We present a near-optimal solution that ensures safety, optimality, and robustness to changing behavior of uncooperative vehicles. Simulations demonstrate the effectiveness of the proposed approach in enhancing efficiency and safety. △ Less

Submitted 30 May, 2023; v1 submitted 29 May, 2023; originally announced May 2023.

Trust-Aware Resilient Control and Coordination of Connected and Automated Vehicles

Authors: H M Sabbir Ahmad, Ehsan Sabouni, Wei Xiao, Christos G. Cassandras, Wenchao Li

Abstract: We address the security of a network of Connected and Automated Vehicles (CAVs) cooperating to navigate through a conflict area. Adversarial attacks such as Sybil attacks can cause safety violations resulting in collisions and traffic jams. In addition, uncooperative (but not necessarily adversarial) CAVs can also induce similar adversarial effects on the traffic network. We propose a decentralize… ▽ More We address the security of a network of Connected and Automated Vehicles (CAVs) cooperating to navigate through a conflict area. Adversarial attacks such as Sybil attacks can cause safety violations resulting in collisions and traffic jams. In addition, uncooperative (but not necessarily adversarial) CAVs can also induce similar adversarial effects on the traffic network. We propose a decentralized resilient control and coordination scheme that mitigates the effects of adversarial attacks and uncooperative CAVs by utilizing a trust framework. Our trust-aware scheme can guarantee safe collision free coordination and mitigate traffic jams. Simulation results validate the theoretical guarantee of our proposed scheme, and demonstrate that it can effectively mitigate adversarial effects across different traffic scenarios. △ Less

Submitted 2 June, 2023; v1 submitted 26 May, 2023; originally announced May 2023.

Comments: Keywords: Resilient control and coordination, Cybersecurity, Safety guaranteed coordination, Connected And Autonomous Vehicles

Merging control in mixed traffic with safety guarantees: a safe sequencing policy with optimal motion control

Authors: Ehsan Sabouni, H. M. Sabbir Ahmad, Christos G. Cassandras, Wenchao Li

Abstract: We address the problem of merging traffic from two roadways consisting of both Connected Autonomous Vehicles (CAVs) and Human Driven Vehicles (HDVs). Guaranteeing safe merging in such mixed traffic settings is challenging due to the unpredictability of possibly uncooperative HDVs. We develop a hierarchical controller where at each discrete time step first a coordinator determines the best possible… ▽ More We address the problem of merging traffic from two roadways consisting of both Connected Autonomous Vehicles (CAVs) and Human Driven Vehicles (HDVs). Guaranteeing safe merging in such mixed traffic settings is challenging due to the unpredictability of possibly uncooperative HDVs. We develop a hierarchical controller where at each discrete time step first a coordinator determines the best possible Safe Sequence (SS) which can be realized without any knowledge of human driving behavior. Then, a lower-level decentralized motion controller for each CAV jointly minimizes travel time and energy over a prediction horizon, subject to hard safety constraints dependent on the given safe sequence. This is accomplished using a Model Predictive Controller (MPC) subject to constraints based on Control Barrier Functions (CBFs) which render it computationally efficient. Extensive simulation results are included showing that this hierarchical controller outperforms the commonly adopted Shortest Distance First (SDF) passing sequence over the full range of CAV penetration rates, while also providing safe merging guarantees. △ Less

Submitted 26 May, 2023; originally announced May 2023.

Scalable Adaptive Traffic Light Control Over a Traffic Network Including Transit Delays

Authors: Yingqing Chen, Christos G. Cassandras

Abstract: We study the Traffic Light Control (TLC) problem for a traffic network with multiple intersections in an artery, including the effect of transit delays for vehicles moving from one intersection to the next. The goal is to minimize the overall mean waiting time and improve the ``green wave'' properties in such systems. Using a stochastic hybrid system model with parametric traffic light controllers… ▽ More We study the Traffic Light Control (TLC) problem for a traffic network with multiple intersections in an artery, including the effect of transit delays for vehicles moving from one intersection to the next. The goal is to minimize the overall mean waiting time and improve the ``green wave'' properties in such systems. Using a stochastic hybrid system model with parametric traffic light controllers, we use Infinitesimal Perturbation Analysis (IPA) to derive a data-driven cost gradient estimator with respect to these parameters. We then iteratively adjust them through an online gradient-based algorithm. We show that the event-driven nature of the IPA estimators driving the controllers leads to scalable computationally efficient controllers as the dimensionality of the traffic network increases. △ Less

Submitted 15 May, 2023; originally announced May 2023.

Comments: arXiv admin note: text overlap with arXiv:2303.08173

Safe Control Synthesis Using Environmentally Robust Control Barrier Functions

Authors: Vahid Hamdipoor, Nader Meskin, Christos G. Cassandras

Abstract: In this paper, we study a safe control design for dynamical systems in the presence of uncertainty in a dynamical environment. The worst-case error approach is considered to formulate robust Control Barrier Functions (CBFs) in an optimization-based control synthesis framework. It is first shown that environmentally robust CBF formulations result in second-order cone programs (SOCPs). Then, a novel… ▽ More In this paper, we study a safe control design for dynamical systems in the presence of uncertainty in a dynamical environment. The worst-case error approach is considered to formulate robust Control Barrier Functions (CBFs) in an optimization-based control synthesis framework. It is first shown that environmentally robust CBF formulations result in second-order cone programs (SOCPs). Then, a novel scheme is presented to formulate robust CBFs which takes the nominally safe control as its desired control input in optimization-based control design and then tries to minimally modify it whenever the robust CBF constraint is violated. This proposed scheme leads to quadratic programs (QPs) which can be easily solved. Finally, the effectiveness of the proposed approach is demonstrated on an adaptive cruise control example. △ Less

Submitted 23 April, 2023; originally announced April 2023.

A Bilevel Optimization Scheme for Persistent Monitoring

Authors: Jonas Hall, Logan E. Beaver, Christos G. Cassandras, Sean B. Andersson

Abstract: In this paper we study an infinite-horizon persistent monitoring problem in a two-dimensional mission space containing a finite number of statically placed targets, at each of which we assume a constant rate of uncertainty accumulation. Equipped with a sensor of finite range, the agent is capable of reducing the uncertainty of nearby targets. We derive a steady-state minimum time periodic trajecto… ▽ More In this paper we study an infinite-horizon persistent monitoring problem in a two-dimensional mission space containing a finite number of statically placed targets, at each of which we assume a constant rate of uncertainty accumulation. Equipped with a sensor of finite range, the agent is capable of reducing the uncertainty of nearby targets. We derive a steady-state minimum time periodic trajectory over which each of the target uncertainties is driven down to zero during each visit. A hierarchical decomposition leads to purely local optimal control problems, coupled via boundary conditions. We optimize both the local trajectory segments as well as the boundary conditions in an on-line bilevel optimization scheme. △ Less

Submitted 7 April, 2023; originally announced April 2023.

Cooperative Lane Changing in Mixed Traffic can be Robust to Human Driver Behavior

Authors: Anni Li, Andres S. Chavez Armijos, Christos G. Cassandras

Abstract: We derive time and energy-optimal control policies for a Connected Autonomous Vehicle (CAV) to complete lane change maneuvers in mixed traffic. The interaction between CAVs and Human-Driven Vehicles (HDVs) requires designing the best possible response of a CAV to actions by its neighboring HDVs. This interaction is formulated using a bilevel optimization setting with an appropriate behavioral mode… ▽ More We derive time and energy-optimal control policies for a Connected Autonomous Vehicle (CAV) to complete lane change maneuvers in mixed traffic. The interaction between CAVs and Human-Driven Vehicles (HDVs) requires designing the best possible response of a CAV to actions by its neighboring HDVs. This interaction is formulated using a bilevel optimization setting with an appropriate behavioral model for an HDV's. Then, an iterated best response (IBR) method is used to determine a Nash equilibrium. However, we also show that when a common and simple-to-detect condition applies, the optimal lane-changing policy is in fact independent of HDV behavior with a CAV changing lanes by cooperating with another CAV in the target lane and always merging ahead of it. Thus, the dependence on the interaction between CAVs and HDVs may be eliminated in such cases. Simulation results are included to show the effectiveness of our controllers in terms of cost, safety guarantees, and disruption to the traffic flow when uncontrollable HDVs are present. △ Less

Submitted 29 March, 2023; originally announced March 2023.

Learning Feasibility Constraints for Control Barrier Functions

Authors: Wei Xiao, Christos G. Cassandras, Calin A. Belta

Abstract: It has been shown that optimizing quadratic costs while stabilizing affine control systems to desired (sets of) states subject to state and control constraints can be reduced to a sequence of Quadratic Programs (QPs) by using Control Barrier Functions (CBFs) and Control Lyapunov Functions (CLFs). In this paper, we employ machine learning techniques to ensure the feasibility of these QPs, which is… ▽ More It has been shown that optimizing quadratic costs while stabilizing affine control systems to desired (sets of) states subject to state and control constraints can be reduced to a sequence of Quadratic Programs (QPs) by using Control Barrier Functions (CBFs) and Control Lyapunov Functions (CLFs). In this paper, we employ machine learning techniques to ensure the feasibility of these QPs, which is a challenging problem, especially for high relative degree constraints where High Order CBFs (HOCBFs) are required. To this end, we propose a sampling-based learning approach to learn a new feasibility constraint for CBFs; this constraint is then enforced by another HOCBF added to the QPs. The accuracy of the learned feasibility constraint is recursively improved by a recurrent training algorithm. We demonstrate the advantages of the proposed learning approach to constrained optimal control problems with specific focus on a robot control problem and on autonomous driving in an unknown environment. △ Less

Submitted 10 March, 2023; originally announced March 2023.

Comments: 8 pages, to appear in ECC 2023

Adaptive Traffic Light Control for Competing Vehicle and Pedestrian Flows

Authors: Yingqing Chen, Christos G. Cassandras

Abstract: We study the Traffic Light Control (TLC) problem for a single intersection, considering both straight driving vehicle flows and corresponding crossing pedestrian flows with the goal of achieving a fair jointly optimal sharing policy in terms of average waiting times. Using a stochastic hybrid system model, we design a quasi-dynamic policy controlling the traffic light cycles with several threshold… ▽ More We study the Traffic Light Control (TLC) problem for a single intersection, considering both straight driving vehicle flows and corresponding crossing pedestrian flows with the goal of achieving a fair jointly optimal sharing policy in terms of average waiting times. Using a stochastic hybrid system model, we design a quasi-dynamic policy controlling the traffic light cycles with several threshold parameters applied to the light cycles and the partially observed contents of vehicle and pedestrian queues. Infinitesimal Perturbation Analysis (IPA) is then used to derive a data-driven gradient estimator of a cost metric with respect to the policy parameters and to iteratively adjust these parameters through an online gradient-based algorithm in order to improve overall performance on this intersection and adapt the policy to changing traffic conditions. The controller is applied to a simulated intersection in the town of Veberöd, Sweden, to illustrate the performance of this approach using real traffic data from this intersection. △ Less

Submitted 14 March, 2023; originally announced March 2023.

Minimally Disruptive Cooperative Lane-change Maneuvers

Authors: Behdad Chalaki, Vaishnav Tadiparthi, Hossein Nourkhiz Mahjoub, Jovin D'sa, Ehsan Moradi-Pari, Andres S. Chavez Armijos, Anni Li, Christos G. Cassandras

Abstract: A lane-change maneuver on a congested highway could be severely disruptive or even infeasible without the cooperation of neighboring cars. However, cooperation with other vehicles does not guarantee that the performed maneuver will not have a negative impact on traffic flow unless it is explicitly considered in the cooperative controller design. In this letter, we present a socially compliant fram… ▽ More A lane-change maneuver on a congested highway could be severely disruptive or even infeasible without the cooperation of neighboring cars. However, cooperation with other vehicles does not guarantee that the performed maneuver will not have a negative impact on traffic flow unless it is explicitly considered in the cooperative controller design. In this letter, we present a socially compliant framework for cooperative lane-change maneuvers for an arbitrary number of CAVs on highways that aims to interrupt traffic flow as minimally as possible. Moreover, we explicitly impose feasibility constraints in the optimization formulation by using reachability set theory, leading to a unified design that removes the need for an iterative procedure used in prior work. We quantitatively evaluate the effectiveness of our framework and compare it against previously offered approaches in terms of maneuver time and incurred throughput disruption. △ Less

Submitted 10 March, 2023; originally announced March 2023.

Comments: 6 pages, 2 figures

Journal ref: IEEE Control Systems Letters, vol. 7, pp. 1766-1771, 2023

Optimal Transport Perturbations for Safe Reinforcement Learning with Robustness Guarantees

Authors: James Queeney, Erhan Can Ozcan, Ioannis Ch. Paschalidis, Christos G. Cassandras

Abstract: Robustness and safety are critical for the trustworthy deployment of deep reinforcement learning. Real-world decision making applications require algorithms that can guarantee robust performance and safety in the presence of general environment disturbances, while making limited assumptions on the data collection process during training. In order to accomplish this goal, we introduce a safe reinfo… ▽ More Robustness and safety are critical for the trustworthy deployment of deep reinforcement learning. Real-world decision making applications require algorithms that can guarantee robust performance and safety in the presence of general environment disturbances, while making limited assumptions on the data collection process during training. In order to accomplish this goal, we introduce a safe reinforcement learning framework that incorporates robustness through the use of an optimal transport cost uncertainty set. We provide an efficient implementation based on applying Optimal Transport Perturbations to construct worst-case virtual state transitions, which does not impact data collection during training and does not require detailed simulator access. In experiments on continuous control tasks with safety constraints, our approach demonstrates robust performance while significantly improving safety at deployment time compared to standard safe reinforcement learning. △ Less

Submitted 28 March, 2024; v1 submitted 30 January, 2023; originally announced January 2023.

Comments: Transactions on Machine Learning Research (TMLR), 2024

Cooperative Energy and Time-Optimal Lane Change Maneuvers with Minimal Highway Traffic Disruption

Authors: Andres S. Chavez Armijos, Anni Li, Christos G. Cassandras, Yasir K. Al-Nadawi, Hidekazu Araki, Behdad Chalaki, Ehsan Moradi-Pari, Hossein Nourkhiz Mahjoub, Vaishnav Tadiparthi

Abstract: We derive optimal control policies for a Connected Automated Vehicle (CAV) and cooperating neighboring CAVs to carry out a lane change maneuver consisting of a longitudinal phase where the CAV properly positions itself relative to the cooperating neighbors and a lateral phase where it safely changes lanes. In contrast to prior work on this problem, where the CAV "selfishly" only seeks to minimize… ▽ More We derive optimal control policies for a Connected Automated Vehicle (CAV) and cooperating neighboring CAVs to carry out a lane change maneuver consisting of a longitudinal phase where the CAV properly positions itself relative to the cooperating neighbors and a lateral phase where it safely changes lanes. In contrast to prior work on this problem, where the CAV "selfishly" only seeks to minimize its maneuver time, we seek to ensure that the fast-lane traffic flow is minimally disrupted (through a properly defined metric). Additionally, when performing lane-changing maneuvers, we optimally select the cooperating vehicles from a set of feasible neighboring vehicles and experimentally show that the highway throughput is improved compared to the baseline case of human-driven vehicles changing lanes with no cooperation. When feasible solutions do not exist for a given maximal allowable disruption, we include a time relaxation method trading off a longer maneuver time with reduced disruption. Our analysis is also extended to multiple sequential maneuvers. Simulation results show the effectiveness of our controllers in terms of safety guarantees and up to 16% and 90% average throughput and maneuver time improvement respectively when compared to maneuvers with no cooperation. △ Less

Submitted 15 November, 2022; originally announced November 2022.

Comments: arXiv admin note: substantial text overlap with arXiv:2203.17102

A Graph-Based Approach to Generate Energy-Optimal Robot Trajectories in Polygonal Environments

Authors: Logan E. Beaver, Roberto Tron, Christos G. Cassandras

Abstract: As robotic systems continue to address emerging issues in areas such as logistics, mobility, manufacturing, and disaster response, it is increasingly important to rapidly generate safe and energy-efficient trajectories. In this article, we present a new approach to plan energy-optimal trajectories through cluttered environments containing polygonal obstacles. In particular, we develop a method to… ▽ More As robotic systems continue to address emerging issues in areas such as logistics, mobility, manufacturing, and disaster response, it is increasingly important to rapidly generate safe and energy-efficient trajectories. In this article, we present a new approach to plan energy-optimal trajectories through cluttered environments containing polygonal obstacles. In particular, we develop a method to quickly generate optimal trajectories for a double-integrator system, and we show that optimal path planning reduces to an integer program. To find an efficient solution, we present a distance-informed prefix search to efficiently generate optimal trajectories for a large class of environments. We demonstrate that our approach, while matching the performance of RRT* and Probabilistic Road Maps in terms of path length, outperforms both in terms of energy cost and computational time by up to an order of magnitude. We also demonstrate that our approach yields implementable trajectories in an experiment with a Crazyflie quadrotor. △ Less

Submitted 11 November, 2022; v1 submitted 9 November, 2022; originally announced November 2022.

Comments: 9 pages, 7 figures

Optimal Merging Control of an Autonomous Vehicle in Mixed Traffic: an Optimal Index Policy

Authors: Ehsan Sabouni, Christos G. Cassandras

Abstract: We consider the problem of a single Autonomous Vehicle (AV) merging into traffic consisting only of Human Driven Vehicles (HDVs) with the goal of minimizing both the travel time and energy consumption of the entire group of vehicles involved in the merging process. This is done by controlling only the AV and determining both the optimal merging sequence and the optimal AV trajectory associated wit… ▽ More We consider the problem of a single Autonomous Vehicle (AV) merging into traffic consisting only of Human Driven Vehicles (HDVs) with the goal of minimizing both the travel time and energy consumption of the entire group of vehicles involved in the merging process. This is done by controlling only the AV and determining both the optimal merging sequence and the optimal AV trajectory associated with it. We derive an optimal index policy which prescribes the merging position of the AV within the group of HDVs. We also specify conditions under which the optimal index corresponds to the AV merging before all HDVs or after all HDVs, in which case no interaction of the AV with the HDVs is required. Simulation results are included to validate the optimal index policy and demonstrate cases where optimal merging can be achieved without requiring any explicit assumptions regarding human driving behavior. △ Less

Submitted 7 November, 2022; originally announced November 2022.

Optimal Persistent Monitoring of Mobile Targets in One Dimension

Authors: Jonas Hall, Sean B. Andersson, Christos G. Cassandras

Abstract: This work shows the existence of optimal control laws for persistent monitoring of mobile targets in a one-dimensional mission space and derives explicit solutions. The underlying performance metric consists of minimizing the total uncertainty accumulated over a finite mission time. We first demonstrate that the corresponding optimal control problem can be reduced to a finite-dimensional optimizat… ▽ More This work shows the existence of optimal control laws for persistent monitoring of mobile targets in a one-dimensional mission space and derives explicit solutions. The underlying performance metric consists of minimizing the total uncertainty accumulated over a finite mission time. We first demonstrate that the corresponding optimal control problem can be reduced to a finite-dimensional optimization problem, and then establish existence of an optimal solution. Motivated by this result, we construct a parametric reformulation for which an event based gradient descent method is utilized with the goal of deriving (locally optimal) solutions. We additionally provide a more practical parameterization that has attractive properties such as simplicity, flexibility, and robustness. Both parameterizations are validated through simulation. △ Less

Submitted 3 October, 2022; originally announced October 2022.

Comments: Submitted to ACC 2023

Optimal Control of Connected Automated Vehicles with Event/Self-Triggered Control Barrier Functions

Authors: Ehsan Sabouni, Christos G. Cassandras, Wei Xiao, Nader Meskin

Abstract: We address the problem of controlling Connected and Automated Vehicles (CAVs) in conflict areas of a traffic network subject to hard safety constraints. It has been shown that such problems can be solved through a combination of tractable optimal control problem formulations and the use of Control Barrier Functions (CBFs) that guarantee the satisfaction of all constraints. These solutions can be r… ▽ More We address the problem of controlling Connected and Automated Vehicles (CAVs) in conflict areas of a traffic network subject to hard safety constraints. It has been shown that such problems can be solved through a combination of tractable optimal control problem formulations and the use of Control Barrier Functions (CBFs) that guarantee the satisfaction of all constraints. These solutions can be reduced to a sequence of Quadratic Programs (QPs) which are efficiently solved on-line over discrete time steps. However, the feasibility of each such QP cannot be guaranteed over every time step. To overcome this limitation, we develop both an event-triggered approach and a self-triggered approach such that the next QP is triggered by properly defined events. We show that both approaches, each in a different way, eliminate infeasible cases due to time-driven inter-sampling effects, thus also eliminating the need for selecting the size of time steps. Simulation examples are included to compare the two new schemes and to illustrate how overall infeasibilities can be significantly reduced while at the same time reducing the need for communication among CAVs without compromising performance. △ Less

Submitted 26 September, 2022; originally announced September 2022.

Comments: arXiv admin note: substantial text overlap with arXiv:2203.12089

Generalized Policy Improvement Algorithms with Theoretically Supported Sample Reuse

Authors: James Queeney, Ioannis Ch. Paschalidis, Christos G. Cassandras

Abstract: Data-driven, learning-based control methods offer the potential to improve operations in complex systems, and model-free deep reinforcement learning represents a popular approach to data-driven control. However, existing classes of algorithms present a trade-off between two important deployment requirements for real-world control: (i) practical performance guarantees and (ii) data efficiency. Off-… ▽ More Data-driven, learning-based control methods offer the potential to improve operations in complex systems, and model-free deep reinforcement learning represents a popular approach to data-driven control. However, existing classes of algorithms present a trade-off between two important deployment requirements for real-world control: (i) practical performance guarantees and (ii) data efficiency. Off-policy algorithms make efficient use of data through sample reuse but lack theoretical guarantees, while on-policy algorithms guarantee approximate policy improvement throughout training but suffer from high sample complexity. In order to balance these competing goals, we develop a class of Generalized Policy Improvement algorithms that combines the policy improvement guarantees of on-policy methods with the efficiency of sample reuse. We demonstrate the benefits of this new class of algorithms through extensive experimental analysis on a variety of continuous control tasks from the DeepMind Control Suite. △ Less

Submitted 13 April, 2023; v1 submitted 27 June, 2022; originally announced June 2022.

Sequential Cooperative Energy and Time-Optimal Lane Change Maneuvers for Highway Traffic

Authors: Andres S. Chavez Armijos, Rui Chen, Christos G. Cassandras, Yasir K. Al-Nadawi, Hossein Noukhiz Mahjoub, Hidekazu Araki

Abstract: We derive optimal control policies for a Connected Automated Vehicle (CAV) and cooperating neighboring CAVs to carry out a lane change maneuver consisting of a longitudinal phase where the CAV properly positions itself relative to the cooperating neighbors and a lateral phase where it safely changes lanes. In contrast to prior work on this problem, where the CAV "selfishly" seeks to minimize its m… ▽ More We derive optimal control policies for a Connected Automated Vehicle (CAV) and cooperating neighboring CAVs to carry out a lane change maneuver consisting of a longitudinal phase where the CAV properly positions itself relative to the cooperating neighbors and a lateral phase where it safely changes lanes. In contrast to prior work on this problem, where the CAV "selfishly" seeks to minimize its maneuver time, we seek to ensure that the fast-lane traffic flow is minimally disrupted (through a properly defined metric) and that highway throughput is improved by optimally selecting the cooperating vehicles. We show that analytical solutions for the optimal trajectories can be derived and are guaranteed to satisfy safety constraints for all vehicles involved in the maneuver. When feasible solutions do not exist, we include a time relaxation method trading off a longer maneuver time with reduced disruption. Our analysis is also extended to multiple sequential maneuvers. Simulation results where the controllers are implemented show their effectiveness in terms of safety guarantees and up to 35% throughput improvement compared to maneuvers with no vehicle cooperation. △ Less

Submitted 31 March, 2022; originally announced March 2022.

Self-Triggered Coordination Control of Connected Automated Vehicles in Traffic Networks

Authors: Nader Meskin, Ehsan Sabouni, Wei Xiao, Christos G. Cassandras

Abstract: In this paper, a self-triggered scheme is proposed to optimally control the traffic flow of Connected and Automated Vehicles (CAVs) at conflict areas of a traffic network with the main aim of reducing the data exchange among CAVs in the control zone and at the same to minimize the travel time and energy consumption. The safety constraints and the vehicle limitations are considered using the Contro… ▽ More In this paper, a self-triggered scheme is proposed to optimally control the traffic flow of Connected and Automated Vehicles (CAVs) at conflict areas of a traffic network with the main aim of reducing the data exchange among CAVs in the control zone and at the same to minimize the travel time and energy consumption. The safety constraints and the vehicle limitations are considered using the Control Barrier Function (CBF) framework and a self-triggered scheme is proposed using the CBF constraints. Moreover, modified CBF constraints are developed to ensure a minimum inter-event interval for the proposed self-triggered schemes. Finally, it is shown through a simulation study that the number of data exchanges among CAVs is significantly reduced using the proposed self-triggered schemes in comparison with the standard time-triggered framework. △ Less

Submitted 8 April, 2022; v1 submitted 24 March, 2022; originally announced March 2022.

Optimal Control of Connected Automated Vehicles with Event-Triggered Control Barrier Functions

Authors: Ehsan Sabouni, Christos G. Cassandras, Wei Xiao, Nader Meskin

Abstract: We address the problem of controlling Connected and Automated Vehicles (CAVs) in conflict areas of a traffic network subject to hard safety constraints. It has been shown that such problems can be solved through a combination of tractable optimal control problem formulations and the use of Control Barrier Functions (CBFs) that guarantee the satisfaction of all constraints. These solutions can be r… ▽ More We address the problem of controlling Connected and Automated Vehicles (CAVs) in conflict areas of a traffic network subject to hard safety constraints. It has been shown that such problems can be solved through a combination of tractable optimal control problem formulations and the use of Control Barrier Functions (CBFs) that guarantee the satisfaction of all constraints. These solutions can be reduced to a sequence of Quadratic Programs (QPs) which are efficiently solved on line over discrete time steps. However, the feasibility of each such QP cannot be guaranteed over every time step. To overcome this limitation, we develop an event-driven approach such that the next QP is triggered by properly defined events and show that this approach can eliminate infeasible cases due to time-driven inter-sampling effects. Simulation examples show how overall infeasibilities can be significantly reduced with the proposed event-triggering scheme, while also reducing the need for communication among CAVs without compromising performance. △ Less

Submitted 22 March, 2022; originally announced March 2022.

Control Barrier Functions for Systems with Multiple Control Inputs

Authors: Wei Xiao, Christos G. Cassandras, Calin A. Belta, Daniela Rus

Abstract: Control Barrier Functions (CBFs) are becoming popular tools in guaranteeing safety for nonlinear systems and constraints, and they can reduce a constrained optimal control problem into a sequence of Quadratic Programs (QPs) for affine control systems. The recently proposed High Order Control Barrier Functions (HOCBFs) work for arbitrary relative degree constraints. One of the challenges in a HOCBF… ▽ More Control Barrier Functions (CBFs) are becoming popular tools in guaranteeing safety for nonlinear systems and constraints, and they can reduce a constrained optimal control problem into a sequence of Quadratic Programs (QPs) for affine control systems. The recently proposed High Order Control Barrier Functions (HOCBFs) work for arbitrary relative degree constraints. One of the challenges in a HOCBF is to address the relative degree problem when a system has multiple control inputs, i.e., the relative degree could be defined with respect to different components of the control vector. This paper proposes two methods for HOCBFs to deal with systems with multiple control inputs: a general integral control method and a method which is simpler but limited to specific classes of physical systems. When control bounds are involved, the feasibility of the above mentioned QPs can also be significantly improved with the proposed methods. We illustrate our approaches on a unicyle model with two control inputs, and compare the two proposed methods to demonstrate their effectiveness and performance. △ Less

Submitted 15 March, 2022; originally announced March 2022.

Comments: To appear in ACC2022

Feasibility Guaranteed Traffic Merging Control Using Control Barrier Functions

Authors: Kaiyuan Xu, Wei Xiao, Christos G. Cassandras

Abstract: We consider the merging control problem for Connected and Automated Vehicles (CAVs) aiming to jointly minimize travel time and energy consumption while providing speed-dependent safety guarantees and satisfying velocity and acceleration constraints. Applying the joint optimal control and control barrier function (OCBF) method, a controller that optimally tracks the unconstrained optimal control so… ▽ More We consider the merging control problem for Connected and Automated Vehicles (CAVs) aiming to jointly minimize travel time and energy consumption while providing speed-dependent safety guarantees and satisfying velocity and acceleration constraints. Applying the joint optimal control and control barrier function (OCBF) method, a controller that optimally tracks the unconstrained optimal control solution while guaranteeing the satisfaction of all constraints is efficiently obtained by transforming the optimal tracking problem into a sequence of quadratic programs (QPs). However, these QPs can become infeasible, especially under tight control bounds, thus failing to guarantee safety constraints. We solve this problem by deriving a control-dependent feasibility constraint corresponding to each CBF constraint which is added to each QP and we show that each such modified QP is guaranteed to be feasible. Extensive simulations of the merging control problem illustrate the effectiveness of this feasibility guaranteed controller. △ Less

Submitted 8 March, 2022; originally announced March 2022.

Minimax Multi-Agent Persistent Monitoring of a Network System

Authors: Samuel C. Pinto, Shirantha Welikala, Sean B. Andersson, Julien M. Hendrickx, Christos G. Cassandras

Abstract: We investigate the problem of optimally observing a finite set of targets using a mobile agent over an infinite time horizon. The agent is tasked to move in a network-constrained structure to gather information so as to minimize the worst-case uncertainty about the internal states of the targets. To do this, the agent has to decide its sequence of target-visits and the corresponding dwell-times at… ▽ More We investigate the problem of optimally observing a finite set of targets using a mobile agent over an infinite time horizon. The agent is tasked to move in a network-constrained structure to gather information so as to minimize the worst-case uncertainty about the internal states of the targets. To do this, the agent has to decide its sequence of target-visits and the corresponding dwell-times at each visited target. For a given visiting sequence, we prove that in an optimal dwelling time allocation the peak uncertainty is the same among all the targets. This allows us to formulate the optimization of dwelling times as a resource allocation problem and to solve it using a novel efficient algorithm. Next, we optimize the visiting sequence using a greedy exploration process, using heuristics inspired by others developed in the context of the traveling salesman problem. Numerical results are included to illustrate the contributions. △ Less

Submitted 17 January, 2022; originally announced January 2022.

Generalized Proximal Policy Optimization with Sample Reuse

Authors: James Queeney, Ioannis Ch. Paschalidis, Christos G. Cassandras

Abstract: In real-world decision making tasks, it is critical for data-driven reinforcement learning methods to be both stable and sample efficient. On-policy methods typically generate reliable policy improvement throughout training, while off-policy methods make more efficient use of data through sample reuse. In this work, we combine the theoretically supported stability benefits of on-policy algorithms… ▽ More In real-world decision making tasks, it is critical for data-driven reinforcement learning methods to be both stable and sample efficient. On-policy methods typically generate reliable policy improvement throughout training, while off-policy methods make more efficient use of data through sample reuse. In this work, we combine the theoretically supported stability benefits of on-policy algorithms with the sample efficiency of off-policy algorithms. We develop policy improvement guarantees that are suitable for the off-policy setting, and connect these bounds to the clip** mechanism used in Proximal Policy Optimization. This motivates an off-policy version of the popular algorithm that we call Generalized Proximal Policy Optimization with Sample Reuse. We demonstrate both theoretically and empirically that our algorithm delivers improved performance by effectively balancing the competing goals of stability and sample efficiency. △ Less

Submitted 29 October, 2021; originally announced November 2021.

Comments: To appear in 35th Conference on Neural Information Processing Systems (NeurIPS 2021)

Planning Strategies for Lane Reversals in Transportation Networks

Authors: Salomon Wollenstein-Betech, Ioannis Ch. Paschalidis, Christos G. Cassandras

Abstract: This paper studies strategies to optimize the lane configuration of a transportation network for a given set of Origin-Destination demands using a planning macroscopic network flow model. The lane reversal problem is, in general, NP-hard since the optimization is made over integer variables. To overcome this burden, we reformulate the problem using a piecewise affine approximation of the travel la… ▽ More This paper studies strategies to optimize the lane configuration of a transportation network for a given set of Origin-Destination demands using a planning macroscopic network flow model. The lane reversal problem is, in general, NP-hard since the optimization is made over integer variables. To overcome this burden, we reformulate the problem using a piecewise affine approximation of the travel latency function which allows us to exploit the total unimodularity property of Integer Linear Programming (ILP). Consequently, we transform the ILP problem to a linear program by relaxing the integer variables. In addition, our method is capable of solving the problem for a desired number of lane reversals which serves to perform cost-benefit analysis. We perform a case study using the transportation network of Eastern Massachusetts (EMA) and we test our method against the original lane configuration and a projected lower bound solution. Our empirical results quantify the travel time savings for different levels of demand intensity. We observe reduction in travel times up to 40% for certain links in the network. △ Less

Submitted 14 July, 2021; originally announced July 2021.

Comments: Proc. IEEE Int. Conf. on Intelligent Transportation Systems, Indianapolis, USA, 2021. (In Press)

Decentralized Time and Energy-Optimal Control of Connected and Automated Vehicles in a Roundabout

Authors: Kaiyuan Xu, Christos G. Cassandras, Wei Xiao

Abstract: The paper considers the problem of controlling Connected and Automated Vehicles (CAVs) traveling through a three-entry roundabout so as to jointly minimize both the travel time and the energy consumption while providing speed-dependent safety guarantees, as well as satisfying velocity and acceleration constraints. We first design a systematic approach to dynamically determine the safety constraint… ▽ More The paper considers the problem of controlling Connected and Automated Vehicles (CAVs) traveling through a three-entry roundabout so as to jointly minimize both the travel time and the energy consumption while providing speed-dependent safety guarantees, as well as satisfying velocity and acceleration constraints. We first design a systematic approach to dynamically determine the safety constraints and derive the unconstrained optimal control solution. A joint optimal control and barrier function (OCBF) method is then applied to efficiently obtain a controller that optimally track the unconstrained optimal solution while guaranteeing all the constraints. Simulation experiments are performed to compare the optimal controller to a baseline of human-driven vehicles showing effectiveness under symmetric and asymmetric roundabout configurations, balanced and imbalanced traffic rates and different sequencing rules for CAVs. △ Less

Submitted 13 April, 2021; originally announced April 2021.

A Semidefinite Programming Approach to Discrete-time Infinite Horizon Persistent Monitoring

Authors: Samuel C. Pinto, Sean B. Andersson, Julien M. Hendrickx, Christos G. Cassandras

Abstract: We investigate the problem of persistent monitoring, where a mobile agent has to survey multiple targets in an environment in order to estimate their internal states. These internal states evolve with linear stochastic dynamics and the agent can observe them with a linear observation model. However, the signal to noise ratio is a monotonically decreasing function of the distance between the agent… ▽ More We investigate the problem of persistent monitoring, where a mobile agent has to survey multiple targets in an environment in order to estimate their internal states. These internal states evolve with linear stochastic dynamics and the agent can observe them with a linear observation model. However, the signal to noise ratio is a monotonically decreasing function of the distance between the agent and the target. The goal is to minimize the uncertainty in the state estimates over the infinite horizon. We show that, for a periodic trajectory with fixed cycle length, the problem can be formulated as a set of semidefinite programs. We design a scheme that leverages the spatial configuration of the targets to guide the search over this set of optimization problems in order to provide efficient trajectories. Results are compared to a state of the art approach and we obtain improvements of up to 91% in terms of cost in a simple scenario, with much lower computational time. △ Less

Submitted 31 March, 2021; originally announced April 2021.

Event-Triggered Safety-Critical Control for Systems with Unknown Dynamics

Authors: Wei Xiao, Calin Belta, Christos G. Cassandras

Abstract: This paper addresses the problem of safety-critical control for systems with unknown dynamics. It has been shown that stabilizing affine control systems to desired (sets of) states while optimizing quadratic costs subject to state and control constraints can be reduced to a sequence of quadratic programs (QPs) by using Control Barrier Functions (CBFs) and Control Lyapunov Functions (CLFs). Our rec… ▽ More This paper addresses the problem of safety-critical control for systems with unknown dynamics. It has been shown that stabilizing affine control systems to desired (sets of) states while optimizing quadratic costs subject to state and control constraints can be reduced to a sequence of quadratic programs (QPs) by using Control Barrier Functions (CBFs) and Control Lyapunov Functions (CLFs). Our recently proposed High Order CBFs (HOCBFs) can accommodate constraints of arbitrary relative degree. One of the main challenges in this approach is obtaining accurate system dynamics, which is especially difficult for systems that require online model identification given limited computational resources and system data. In order to approximate the real unmodelled system dynamics, we define adaptive affine control dynamics which are updated based on the error states obtained by real-time sensor measurements. We define a HOCBF for a safety requirement on the unmodelled system based on the adaptive dynamics and error states, and reformulate the safety-critical control problem as the above mentioned QP. Then, we determine the events required to solve the QP in order to guarantee safety. We also derive a condition that guarantees the satisfaction of the HOCBF constraint between events. We illustrate the effectiveness of the proposed framework on an adaptive cruise control problem and compare it with the classical time-driven approach. △ Less

Submitted 29 March, 2021; originally announced March 2021.

Comments: 8 pages, submitted to CDC2021. arXiv admin note: text overlap with arXiv:2011.08248

Event-Driven Receding Horizon Control of Energy-Aware Dynamic Agents For Distributed Persistent Monitoring

Authors: Shirantha Welikala, Christos G. Cassandras

Abstract: This paper addresses the persistent monitoring problem defined on a network where a set of nodes (targets) needs to be monitored by a team of dynamic energy-aware agents. The objective is to control the agents' motion to jointly optimize the overall agent energy consumption and a measure of overall node state uncertainty, evaluated over a finite period of interest. To achieve these objectives, we… ▽ More This paper addresses the persistent monitoring problem defined on a network where a set of nodes (targets) needs to be monitored by a team of dynamic energy-aware agents. The objective is to control the agents' motion to jointly optimize the overall agent energy consumption and a measure of overall node state uncertainty, evaluated over a finite period of interest. To achieve these objectives, we extend an established event-driven Receding Horizon Control (RHC) solution by adding an optimal controller to account for agent motion dynamics and associated energy consumption. The resulting RHC solution is computationally efficient, distributed and on-line. Finally, numerical results are provided highlighting improvements compared to an existing RHC solution that uses energy-agnostic first-order agents. △ Less

Submitted 25 February, 2021; originally announced February 2021.

Comments: Submitted to the 60th IEEE Conference on Decision and Control

High Order Control Lyapunov-Barrier Functions for Temporal Logic Specifications

Authors: Wei Xiao, Calin A. Belta, Christos G. Cassandras

Abstract: Recent work has shown that stabilizing an affine control system to a desired state while optimizing a quadratic cost subject to state and control constraints can be reduced to a sequence of Quadratic Programs (QPs) by using Control Barrier Functions (CBFs) and Control Lyapunov Functions (CLFs). In our own recent work, we defined High Order CBFs (HOCBFs) for systems and constraints with arbitrary r… ▽ More Recent work has shown that stabilizing an affine control system to a desired state while optimizing a quadratic cost subject to state and control constraints can be reduced to a sequence of Quadratic Programs (QPs) by using Control Barrier Functions (CBFs) and Control Lyapunov Functions (CLFs). In our own recent work, we defined High Order CBFs (HOCBFs) for systems and constraints with arbitrary relative degrees. In this paper, in order to accommodate initial states that do not satisfy the state constraints and constraints with arbitrary relative degree, we generalize HOCBFs to High Order Control Lyapunov-Barrier Functions (HOCLBFs). We also show that the proposed HOCLBFs can be used to guarantee the Boolean satisfaction of Signal Temporal Logic (STL) formulae over the state of the system. We illustrate our approach on a safety-critical optimal control problem (OCP) for a unicycle. △ Less

Submitted 12 February, 2021; originally announced February 2021.

Comments: 9 pages, accepted in ACC 2021

Uncertainty-Aware Policy Optimization: A Robust, Adaptive Trust Region Approach

Authors: James Queeney, Ioannis Ch. Paschalidis, Christos G. Cassandras

Abstract: In order for reinforcement learning techniques to be useful in real-world decision making processes, they must be able to produce robust performance from limited data. Deep policy optimization methods have achieved impressive results on complex tasks, but their real-world adoption remains limited because they often require significant amounts of data to succeed. When combined with small sample siz… ▽ More In order for reinforcement learning techniques to be useful in real-world decision making processes, they must be able to produce robust performance from limited data. Deep policy optimization methods have achieved impressive results on complex tasks, but their real-world adoption remains limited because they often require significant amounts of data to succeed. When combined with small sample sizes, these methods can result in unstable learning due to their reliance on high-dimensional sample-based estimates. In this work, we develop techniques to control the uncertainty introduced by these estimates. We leverage these techniques to propose a deep policy optimization approach designed to produce stable performance even when data is scarce. The resulting algorithm, Uncertainty-Aware Trust Region Policy Optimization, generates robust policy updates that adapt to the level of uncertainty present throughout the learning process. △ Less

Submitted 19 December, 2020; originally announced December 2020.

Comments: To appear in Proceedings of the Thirty-Fifth AAAI Conference on Artificial Intelligence (AAAI-21)

Sufficient Conditions for Feasibility of Optimal Control Problems Using Control Barrier Functions

Authors: Wei Xiao, Calin Belta, Christos G. Cassandras

Abstract: It has been shown that satisfying state and control constraints while optimizing quadratic costs subject to desired (sets of) state convergence for affine control systems can be reduced to a sequence of quadratic programs (QPs) by using Control Barrier Functions (CBFs) and Control Lyapunov Functions (CLFs). One of the main challenges in this approach is ensuring the feasibility of these QPs, espec… ▽ More It has been shown that satisfying state and control constraints while optimizing quadratic costs subject to desired (sets of) state convergence for affine control systems can be reduced to a sequence of quadratic programs (QPs) by using Control Barrier Functions (CBFs) and Control Lyapunov Functions (CLFs). One of the main challenges in this approach is ensuring the feasibility of these QPs, especially under tight control bounds and safety constraints of high relative degree. In this paper, we provide sufficient conditions for guranteed feasibility. The sufficient conditions are captured by a single constraint that is enforced by a CBF, which is added to the QPs such that their feasibility is always guaranteed. The additional constraint is designed to be always compatible with the existing constraints, therefore, it cannot make a feasible set of constraints infeasible - it can only increase the overall feasibility. We illustrate the effectiveness of the proposed approach on an adaptive cruise control problem. △ Less

Submitted 16 November, 2020; originally announced November 2020.

Comments: 10 pages, submitted to Automatica

Comparison of Cooperative Driving Strategies for CAVs at Signal-Free Intersections

Authors: Huile Xu, Christos G. Cassandras, Li Li, Yi Zhang

Abstract: The properties of cooperative driving strategies for planning and controlling Connected and Automated Vehicles (CAVs) at intersections range from some that achieve highly efficient coordination performance to others whose implementation is computationally fast. This paper comprehensively compares the performance of four representative strategies in terms of travel time, energy consumption, computa… ▽ More The properties of cooperative driving strategies for planning and controlling Connected and Automated Vehicles (CAVs) at intersections range from some that achieve highly efficient coordination performance to others whose implementation is computationally fast. This paper comprehensively compares the performance of four representative strategies in terms of travel time, energy consumption, computation time, and fairness under different conditions, including the geometric configuration of intersections, asymmetry in traffic arrival rates, and the relative magnitude of these rates. Our simulation-based study has led to the following conclusions: 1) the Monte Carlo Tree Search (MCTS)-based strategy achieves the best traffic efficiency, whereas the Dynamic Resequencing (DR)-based strategy is energy-optimal; both strategies perform well in all metrics of interest. If the computation budget is adequate, the MCTS strategy is recommended; otherwise, the DR strategy is preferable; 2) An asymmetric intersection has a noticeable impact on the strategies, whereas the influence of the arrival rates can be neglected. When the geometric shape is asymmetrical, the modified First-In-First-Out (FIFO) strategy significantly outperforms the FIFO strategy and works well when the traffic demand is moderate, but their performances are similar in other situations; and 3) Improving traffic efficiency sometimes comes at the cost of fairness, but the DR and MCTS strategies can be adjusted to realize a better trade-off between various performance metrics by appropriately designing their objective functions. △ Less

Submitted 7 October, 2020; originally announced October 2020.

A General Framework for Decentralized Safe Optimal Control of Connected and Automated Vehicles in Multi-Lane Intersections

Authors: Huile Xu, Wei Xiao, Christos G. Cassandras, Yi Zhang, Li Li

Abstract: We address the problem of optimally controlling Connected and Automated Vehicles (CAVs) arriving from four multi-lane roads at an intersection where they conflict in terms of safely crossing (including turns) with no collision. The objective is to jointly minimize the travel time and energy consumption of each CAV while ensuring safety. This problem was solved in prior work for single-lane roads.… ▽ More We address the problem of optimally controlling Connected and Automated Vehicles (CAVs) arriving from four multi-lane roads at an intersection where they conflict in terms of safely crossing (including turns) with no collision. The objective is to jointly minimize the travel time and energy consumption of each CAV while ensuring safety. This problem was solved in prior work for single-lane roads. A direct extension to multiple lanes on each road is limited by the computational complexity required to obtain an explicit optimal control solution. Instead, we propose a general framework that first converts a multi-lane intersection problem into a decentralized optimal control problem for each CAV with less conservative safety constraints than prior work. We then employ a method combining optimal control and control barrier functions, which has been shown to efficiently track tractable unconstrained optimal CAV trajectories while also guaranteeing the satisfaction of all constraints. Simulation examples are included to show the effectiveness of the proposed framework under symmetric and asymmetric intersection geometries and different schemes for CAV sequencing. △ Less

Submitted 6 October, 2020; originally announced October 2020.

Event-Driven Receding Horizon Control for Distributed Estimation in Network Systems

Authors: Shirantha Welikala, Christos G. Cassandras

Abstract: We consider the problem of estimating the states of a distributed network of nodes (targets) through a team of cooperating agents (sensors) persistently visiting the nodes so that an overall measure of estimation error covariance evaluated over a finite period is minimized. We formulate this as a multi-agent persistent monitoring problem where the goal is to control each agent's trajectory defined… ▽ More We consider the problem of estimating the states of a distributed network of nodes (targets) through a team of cooperating agents (sensors) persistently visiting the nodes so that an overall measure of estimation error covariance evaluated over a finite period is minimized. We formulate this as a multi-agent persistent monitoring problem where the goal is to control each agent's trajectory defined as a sequence of target visits and the corresponding dwell times spent making observations at each visited target. A distributed on-line agent controller is developed where each agent solves a sequence of receding horizon control problems (RHCPs) in an event-driven manner. A novel objective function is proposed for these RHCPs so as to optimize the effectiveness of this distributed estimation process and its unimodality property is established under some assumptions. Moreover, a machine learning solution is proposed to improve the computational efficiency of this distributed estimation process by exploiting the history of each agent's trajectory. Finally, extensive numerical results are provided indicating significant improvements compared to other state-of-the-art agent controllers. △ Less

Submitted 12 October, 2021; v1 submitted 24 September, 2020; originally announced September 2020.

Comments: Submitted to IEEE Transactions on Automatic Control

Optimal Minimax Mobile Sensor Scheduling Over a Network

Authors: Samuel C. Pinto, Sean B. Andersson, Julien M. Hendrickx, Christos G. Cassandras

Abstract: We investigate the problem of monitoring multiple targets using a single mobile sensor, with the goal of minimizing the maximum estimation error among all the targets over long time horizons. The sensor can move in a network-constrained structure, where it has to plan which targets to visit and for how long to dwell at each node. We prove that in an optimal observation time allocation, the peak un… ▽ More We investigate the problem of monitoring multiple targets using a single mobile sensor, with the goal of minimizing the maximum estimation error among all the targets over long time horizons. The sensor can move in a network-constrained structure, where it has to plan which targets to visit and for how long to dwell at each node. We prove that in an optimal observation time allocation, the peak uncertainty is the same among all the targets. By further restricting the agent policy to only visit each target once every cycle, we develop a scheme to optimize the agent's behavior that is significantly simpler computationally when compared to previous approaches for similar problems. △ Less

Submitted 23 September, 2020; originally announced September 2020.

Comparison of Centralized and Decentralized Approaches in Cooperative Coverage Problems with Energy-Constrained Agents

Authors: Xiangyu Meng, Xinmiao Sun, Christos G. Cassandras, Kaiyuan Xu

Abstract: A multi-agent coverage problem is considered with energy-constrained agents. The objective of this paper is to compare the coverage performance between centralized and decentralized approaches. To this end, a near-optimal centralized coverage control method is developed under energy depletion and repletion constraints. The optimal coverage formation corresponds to the locations of agents where the… ▽ More A multi-agent coverage problem is considered with energy-constrained agents. The objective of this paper is to compare the coverage performance between centralized and decentralized approaches. To this end, a near-optimal centralized coverage control method is developed under energy depletion and repletion constraints. The optimal coverage formation corresponds to the locations of agents where the coverage performance is maximized. The optimal charging formation corresponds to the locations of agents with one agent fixed at the charging station and the remaining agents maximizing the coverage performance. We control the behavior of this cooperative multi-agent system by switching between the optimal coverage formation and the optimal charging formation. Finally, the optimal dwell times at coverage locations, charging time, and agent trajectories are determined so as to maximize coverage over a given time interval. In particular, our controller guarantees that at any time there is at most one agent leaving the team for energy repletion. △ Less

Submitted 25 August, 2020; originally announced August 2020.

Bridging the Gap between Optimal Trajectory Planning and Safety-Critical Control with Applications to Autonomous Vehicles

Authors: Wei Xiao, Christos G. Cassandras, Calin A. Belta

Abstract: We address the problem of optimizing the performance of a dynamic system while satisfying hard safety constraints at all times. Implementing an optimal control solution is limited by the computational cost required to derive it in real time, especially when constraints become active, as well as the need to rely on simple linear dynamics, simple objective functions, and ignoring noise. The recently… ▽ More We address the problem of optimizing the performance of a dynamic system while satisfying hard safety constraints at all times. Implementing an optimal control solution is limited by the computational cost required to derive it in real time, especially when constraints become active, as well as the need to rely on simple linear dynamics, simple objective functions, and ignoring noise. The recently proposed Control Barrier Function (CBF) method may be used for safety-critical control at the expense of sub-optimal performance. In this paper, we develop a real-time control framework that combines optimal trajectories generated through optimal control with the computationally efficient CBF method providing safety guarantees. We use Hamiltonian analysis to obtain a tractable optimal solution for a linear or linearized system, then employ High Order CBFs (HOCBFs) and Control Lyapunov Functions (CLFs) to account for constraints with arbitrary relative degrees and to track the optimal state, respectively. We further show how to deal with noise in arbitrary relative degree systems. The proposed framework is then applied to the optimal traffic merging problem for Connected and Automated Vehicles (CAVs) where the objective is to jointly minimize the travel time and energy consumption of each CAV subject to speed, acceleration, and speed-dependent safety constraints. In addition, when considering more complex objective functions, nonlinear dynamics and passenger comfort requirements for which analytical optimal control solutions are unavailable, we adapt the HOCBF method to such problems. Simulation examples are included to compare the performance of the proposed framework to optimal solutions (when available) and to a baseline provided by human-driven vehicles with results showing significant improvements in all metrics. △ Less

Submitted 17 August, 2020; originally announced August 2020.

Comments: 15 pages, Provisionally accepted in Automatica