Showing 1–45 of 45 results for author: Tsiotras, P

Reachability and Controllability Analysis of the State Covariance for Linear Stochastic Systems

Authors: Fengjiao Liu, Panagiotis Tsiotras

Abstract: This paper studies the set of terminal state covariances that are reachable over a finite time horizon from a given initial state covariance for a linear stochastic system with additive noise. For discrete-time systems, a complete characterization of the set of reachable state covariances is given. For continuous-time systems, we present an upper bound on the set of reachable state covariances. Mo… ▽ More This paper studies the set of terminal state covariances that are reachable over a finite time horizon from a given initial state covariance for a linear stochastic system with additive noise. For discrete-time systems, a complete characterization of the set of reachable state covariances is given. For continuous-time systems, we present an upper bound on the set of reachable state covariances. Moreover, for both linear discrete-time and continuous-time systems, necessary and sufficient conditions are provided for the controllability of the state covariance over a finite horizon. △ Less

Submitted 20 June, 2024; originally announced June 2024.

Comments: 15 pages, 1 figure

Distribution Steering for Discrete-Time Uncertain Ensemble Systems

Authors: Guangyu Wu, Panagiotis Tsiotras, Anders Lindquist

Abstract: Ensemble systems appear frequently in many engineering applications and, as a result, they have become an important research topic in control theory. These systems are best characterized by the evolution of their underlying state distribution. Despite the work to date, few results exist dealing with the problem of directly modifying (i.e., "steering") the distribution of an ensemble system. In add… ▽ More Ensemble systems appear frequently in many engineering applications and, as a result, they have become an important research topic in control theory. These systems are best characterized by the evolution of their underlying state distribution. Despite the work to date, few results exist dealing with the problem of directly modifying (i.e., "steering") the distribution of an ensemble system. In addition, in most of the existing results, the distribution of the states of an ensemble of discrete-time systems is assumed to be Gaussian. However, in case the system parameters are uncertain, it is not always realistic to assume that the distribution of the system follows a Gaussian distribution, thus complicating the solution of the overall problem. In this paper, we address the general distribution steering problem for first-order discrete-time ensemble systems, where the distributions of the system parameters and the states are arbitrary with finite first few moments. Both linear and nonlinear system dynamics are considered using the method of power moments to transform the original infinite-dimensional problem into a finite-dimensional one. We also propose a control law for the ensuing moment system, which allows us to obtain the power moments of the desired control inputs. Finally, we solve the inverse problem to obtain the feasible control inputs from their corresponding power moments. We provide numerical results to validate our theoretical developments. These include cases where the parameter distribution is uniform, Gaussian, non-Gaussian, and multi-modal, respectively. △ Less

Submitted 20 May, 2024; originally announced May 2024.

Comments: 16 pages, 18 figures

Adaptive Dual Covariance Steering with Active Parameter Estimation

Authors: Jacob W. Knaup, Panagiotis Tsiotras

Abstract: This work examines the optimal covariance steering problem for systems subject to unknown parameters that enter multiplicatively with the state and control, in addition to additive disturbances. In contrast to existing works, the unknown parameters are modeled as random variables and are estimated online. This work proposes the utilization of recursive least squares estimation for efficient parame… ▽ More This work examines the optimal covariance steering problem for systems subject to unknown parameters that enter multiplicatively with the state and control, in addition to additive disturbances. In contrast to existing works, the unknown parameters are modeled as random variables and are estimated online. This work proposes the utilization of recursive least squares estimation for efficient parameter identification. A dual control problem is formulated in which the effect of the planned control policy on the parameter estimates is modeled and optimized for. The parameter estimates are then used to modify the pre-computed control policy online in an adaptive control fashion. Finally, the proposed approach is demonstrated in a vehicle control example with closed-loop parameter identification. △ Less

Submitted 22 March, 2024; originally announced March 2024.

Distributionally Robust Density Control with Wasserstein Ambiguity Sets

Authors: Joshua Pilipovsky, Panagiotis Tsiotras

Abstract: Precise control under uncertainty requires a good understanding and characterization of the noise affecting the system. This paper studies the problem of steering state distributions of dynamical systems subject to partially known uncertainties. We model the distributional uncertainty of the noise process in terms of Wasserstein ambiguity sets, which, based on recent results, have been shown to be… ▽ More Precise control under uncertainty requires a good understanding and characterization of the noise affecting the system. This paper studies the problem of steering state distributions of dynamical systems subject to partially known uncertainties. We model the distributional uncertainty of the noise process in terms of Wasserstein ambiguity sets, which, based on recent results, have been shown to be an effective means of capturing and propagating uncertainty through stochastic LTI systems. To this end, we propagate the distributional uncertainty of the state through the dynamical system, and, using an affine feedback control law, we steer the ambiguity set of the state to a prescribed, terminal ambiguity set. We also enforce distributionally robust CVaR constraints for the transient motion of the state so as to reside within a prescribed constraint space. The resulting optimization problem is formulated as a semi-definite program, which can be solved efficiently using standard off-the-shelf solvers. We illustrate the proposed distributionally-robust framework on a quadrotor landing problem subject to wind turbulence. △ Less

Submitted 18 March, 2024; originally announced March 2024.

Comments: CDC 2024

Data-Driven Robust Covariance Control for Uncertain Linear Systems

Authors: Joshua Pilipovsky, Panagiotis Tsiotras

Abstract: The theory of covariance control and covariance steering (CS) deals with controlling the dispersion of trajectories of a dynamical system, under the implicit assumption that accurate prior knowledge of the system being controlled is available. In this work, we consider the problem of steering the distribution of a discrete-time, linear system subject to exogenous disturbances under an unknown dyna… ▽ More The theory of covariance control and covariance steering (CS) deals with controlling the dispersion of trajectories of a dynamical system, under the implicit assumption that accurate prior knowledge of the system being controlled is available. In this work, we consider the problem of steering the distribution of a discrete-time, linear system subject to exogenous disturbances under an unknown dynamics model. Leveraging concepts from behavioral systems theory, the trajectories of this unknown, noisy system may be (approximately) represented using system data collected through experimentation. Using this fact, we formulate a direct data-driven covariance control problem using input-state data. We then propose a maximum likelihood uncertainty quantification method to estimate and bound the noise realizations in the data collection process. Lastly, we utilize robust convex optimization techniques to solve the resulting norm-bounded uncertain convex program. We illustrate the proposed end-to-end data-driven CS algorithm on a double integrator example and showcase the efficacy and accuracy of the proposed method compared to that of model-based methods △ Less

Submitted 18 May, 2024; v1 submitted 10 December, 2023; originally announced December 2023.

Comments: L4DC 2024

Computationally Efficient Chance Constrained Covariance Control with Output Feedback

Authors: Joshua Pilipovsky, Panagiotis Tsiotras

Abstract: This paper studies the problem of develo** computationally efficient solutions for steering the distribution of the state of a stochastic, linear dynamical system between two boundary Gaussian distributions in the presence of chance-constraints on the state and control input. It is assumed that the state is only partially available through a measurement model corrupted with noise. The filtered s… ▽ More This paper studies the problem of develo** computationally efficient solutions for steering the distribution of the state of a stochastic, linear dynamical system between two boundary Gaussian distributions in the presence of chance-constraints on the state and control input. It is assumed that the state is only partially available through a measurement model corrupted with noise. The filtered state is reconstructed with a Kalman filter, the chance constraints are reformulated as difference of convex (DC) constraints, and the resulting covariance control problem is reformulated as a DC program, which is solved using successive convexification. The efficiency of the proposed method is illustrated on a double integrator example with varying time horizons, and is compared to other state-of-the-art chance constrained covariance control methods. △ Less

Submitted 21 March, 2024; v1 submitted 3 October, 2023; originally announced October 2023.

Comments: v2, submitted to CDC '24

Desensitization and Deception in Differential Games with Asymmetric Information

Authors: Vinodhini Comandur, Tulasi Ram Vechalapu, Venkata Ramana Makkapati, Panagiotis Tsiotras, Seth Hutchinson

Abstract: Desensitization addresses safe optimal planning under parametric uncertainties by providing sensitivity function-based risk estimates. This paper expands upon the existing work on desensitization in optimal control to address safe planning for a class of two-player differential games. In the proposed game, parametric uncertainties correspond to variations of the model parameters for each player ab… ▽ More Desensitization addresses safe optimal planning under parametric uncertainties by providing sensitivity function-based risk estimates. This paper expands upon the existing work on desensitization in optimal control to address safe planning for a class of two-player differential games. In the proposed game, parametric uncertainties correspond to variations of the model parameters for each player about their nominal values. The two players in the proposed formulation are assumed to have perfect information about these nominal parameter values. However, it is assumed that only one of the players has complete knowledge of the actual parameter value, resulting in information asymmetry in the proposed game. This lack of knowledge regarding the parameter variations is expected to result in state constraint violations for the player with an information disadvantage. In this regard, a desensitized feedback strategy that provides safe trajectories is proposed for the player with incomplete information. The proposed feedback strategy is evaluated for instances involving a single pursuer and a single evader with an uncertain moving obstacle, where the pursuer is assumed to only know the nominal value of the obstacle's speed. At the same time, the evader knows the obstacle's true speed, and also the fact that the pursuer knows only the nominal value of the obstacle's speed. Subsequently, deceptive strategies are proposed for the evader, who has an information advantage, and these strategies are assessed against the pursuer's desensitized strategy. △ Less

Submitted 6 February, 2024; v1 submitted 17 September, 2023; originally announced September 2023.

Data-Driven Covariance Steering Control Design

Authors: Joshua Pilipovsky, Panagiotis Tsiotras

Abstract: This paper studies the problem of steering the distribution of a linear time-invariant system from an initial normal distribution to a terminal normal distribution under no knowledge of the system dynamics. This data-driven control framework uses data collected from the input and the state and utilizes the seminal work by Willems et al. to construct a data-based parametrization of the mean and the… ▽ More This paper studies the problem of steering the distribution of a linear time-invariant system from an initial normal distribution to a terminal normal distribution under no knowledge of the system dynamics. This data-driven control framework uses data collected from the input and the state and utilizes the seminal work by Willems et al. to construct a data-based parametrization of the mean and the covariance control problems. These problems are then solved to optimality as convex programs using standard techniques from the covariance control literature. We also discuss the equivalence of indirect and direct data-driven covariance steering designs, as well as a regularized version of the problem that provides a balance between the two. We illustrate the proposed framework through a set of randomized trials on a double integrator system and show that the results match up almost exactly with the corresponding model-based method in the noiseless case. We then analyze the robustness properties of the data-free and data-driven covariance steering methods and demonstrate the trade-offs between performance and optimality among these methods in the presence of data corrupted with exogenous noise. △ Less

Submitted 30 March, 2023; originally announced March 2023.

Comments: Submitted to CDC 2023

Zero-Sum Games between Large-Population Teams: Reachability-based Analysis under Mean-Field Sharing

Authors: Yue Guan, Mohammad Afshari, Panagiotis Tsiotras

Abstract: This work studies the behaviors of two large-population teams competing in a discrete environment. The team-level interactions are modeled as a zero-sum game while the agent dynamics within each team is formulated as a collaborative mean-field team problem. Drawing inspiration from the mean-field literature, we first approximate the large-population team game with its infinite-population limit. Su… ▽ More This work studies the behaviors of two large-population teams competing in a discrete environment. The team-level interactions are modeled as a zero-sum game while the agent dynamics within each team is formulated as a collaborative mean-field team problem. Drawing inspiration from the mean-field literature, we first approximate the large-population team game with its infinite-population limit. Subsequently, we construct a fictitious centralized system and transform the infinite-population game to an equivalent zero-sum game between two coordinators. We study the optimal coordination strategies for each team via a novel reachability analysis and later translate them back to decentralized strategies that the original agents deploy. We prove that the strategies are $ε$-optimal for the original finite-population team game, and we further show that the suboptimality diminishes when team size approaches infinity. The theoretical guarantees are verified by numerical examples. △ Less

Submitted 22 February, 2024; v1 submitted 21 March, 2023; originally announced March 2023.

Comments: Published in AAAI 2024

Journal ref: The 38th Annual AAAI Conference on Artificial Intelligence 2024

Covariance Steering for Systems Subject to Unknown Parameters

Authors: Jacob Knaup, Panagiotis Tsiotras

Abstract: This work considers the optimal covariance steering problem for systems subject to both additive noise and uncertain parameters which may enter multiplicatively with the state and the control. The unknown parameters are modeled as a constant random variable sampled from a distribution with known moments. The optimal covariance steering problem is formulated using a moment-based representation of t… ▽ More This work considers the optimal covariance steering problem for systems subject to both additive noise and uncertain parameters which may enter multiplicatively with the state and the control. The unknown parameters are modeled as a constant random variable sampled from a distribution with known moments. The optimal covariance steering problem is formulated using a moment-based representation of the system dynamics, which includes dependence between the unknown parameters and future states, and is solved using sequential convex programming. The proposed approach is demonstrated numerically using a holonomic spacecraft system and an autonomous vehicle control application. △ Less

Submitted 17 March, 2023; originally announced March 2023.

Comments: 19 pages, 5 figures

Discrete-time Optimal Covariance Steering via Semidefinite Programming

Authors: George Rapakoulias, Panagiotis Tsiotras

Abstract: This paper addresses the optimal covariance steering problem for stochastic discrete-time linear systems subject to probabilistic state and control constraints. A method is presented for efficiently attaining the exact solution of the problem based on a lossless convex relaxation of the original non-linear program using semidefinite programming. Both the constrained and the unconstrained versions… ▽ More This paper addresses the optimal covariance steering problem for stochastic discrete-time linear systems subject to probabilistic state and control constraints. A method is presented for efficiently attaining the exact solution of the problem based on a lossless convex relaxation of the original non-linear program using semidefinite programming. Both the constrained and the unconstrained versions of the problem with either equality or inequality terminal covariance boundary conditions are addressed. We first prove that the proposed relaxation is lossless for all of the above cases. A numerical example is then provided to illustrate the method. Finally, a comparative study is performed in systems of various sizes and steering horizons to illustrate the advantages of the proposed method in terms of computational resources compared to the state of the art. △ Less

Submitted 4 October, 2023; v1 submitted 27 February, 2023; originally announced February 2023.

Comments: This paper has been accepted for publication in CDC 2023

Shield Model Predictive Path Integral: A Computationally Efficient Robust MPC Approach Using Control Barrier Functions

Authors: Ji Yin, Charles Dawson, Chuchu Fan, Panagiotis Tsiotras

Abstract: Model Predictive Path Integral (MPPI) control is a type of sampling-based model predictive control that simulates thousands of trajectories and uses these trajectories to synthesize optimal controls on-the-fly. In practice, however, MPPI encounters problems limiting its application. For instance, it has been observed that MPPI tends to make poor decisions if unmodeled dynamics or environmental dis… ▽ More Model Predictive Path Integral (MPPI) control is a type of sampling-based model predictive control that simulates thousands of trajectories and uses these trajectories to synthesize optimal controls on-the-fly. In practice, however, MPPI encounters problems limiting its application. For instance, it has been observed that MPPI tends to make poor decisions if unmodeled dynamics or environmental disturbances exist, preventing its use in safety-critical applications. Moreover, the multi-threaded simulations used by MPPI require significant onboard computational resources, making the algorithm inaccessible to robots without modern GPUs. To alleviate these issues, we propose a novel (Shield-MPPI) algorithm that provides robustness against unpredicted disturbances and achieves real-time planning using a much smaller number of parallel simulations on regular CPUs. The novel Shield-MPPI algorithm is tested on an aggressive autonomous racing platform both in simulation and using experiments. The results show that the proposed controller greatly reduces the number of constraint violations compared to state-of-the-art robust MPPI variants and stochastic MPC methods. △ Less

Submitted 22 February, 2023; originally announced February 2023.

Comments: 8 pages, 7 figures. Submitted to RA-L for review

Deep Monocular Hazard Detection for Safe Small Body Landing

Authors: Travis Driver, Kento Tomita, Koki Ho, Panagiotis Tsiotras

Abstract: Hazard detection and avoidance is a key technology for future robotic small body sample return and lander missions. Current state-of-the-practice methods rely on high-fidelity, a priori terrain maps, which require extensive human-in-the-loop verification and expensive reconnaissance campaigns to resolve map** uncertainties. We propose a novel safety map** paradigm that leverages deep semantic… ▽ More Hazard detection and avoidance is a key technology for future robotic small body sample return and lander missions. Current state-of-the-practice methods rely on high-fidelity, a priori terrain maps, which require extensive human-in-the-loop verification and expensive reconnaissance campaigns to resolve map** uncertainties. We propose a novel safety map** paradigm that leverages deep semantic segmentation techniques to predict landing safety directly from a single monocular image, thus reducing reliance on high-fidelity, a priori data products. We demonstrate precise and accurate safety map** performance on real in-situ imagery of prospective sample sites from the OSIRIS-REx mission. △ Less

Submitted 30 January, 2023; originally announced January 2023.

Comments: Presented at the AAS/AIAA Space Flight Mechanics Meeting, January 14-19, 2023, Austin, TX, USA

Report number: 23-294

Computationally Efficient Covariance Steering for Systems Subject to Parametric Disturbances and Chance Constraints

Authors: Jacob Knaup, Panagiotis Tsiotras

Abstract: This work investigates the finite-horizon optimal covariance steering problem for discrete-time linear systems subject to both additive and multiplicative uncertainties as well as state and input chance constraints. In particular, a tractable convex approximation of the optimal covariance steering problem is developed by tightening the chance constraints and by introducing a suitable change of var… ▽ More This work investigates the finite-horizon optimal covariance steering problem for discrete-time linear systems subject to both additive and multiplicative uncertainties as well as state and input chance constraints. In particular, a tractable convex approximation of the optimal covariance steering problem is developed by tightening the chance constraints and by introducing a suitable change of variables. The solution of the convex approximation is shown to be a valid (albeit potentially suboptimal) solution to the original chance-constrained covariance steering problem. △ Less

Submitted 18 January, 2023; originally announced January 2023.

Comments: 8 pages, 1 figure

Probabilistic Verification of ReLU Neural Networks via Characteristic Functions

Authors: Joshua Pilipovsky, Vignesh Sivaramakrishnan, Meeko M. K. Oishi, Panagiotis Tsiotras

Abstract: Verifying the input-output relationships of a neural network so as to achieve some desired performance specification is a difficult, yet important, problem due to the growing ubiquity of neural nets in many engineering applications. We use ideas from probability theory in the frequency domain to provide probabilistic verification guarantees for ReLU neural networks. Specifically, we interpret a (d… ▽ More Verifying the input-output relationships of a neural network so as to achieve some desired performance specification is a difficult, yet important, problem due to the growing ubiquity of neural nets in many engineering applications. We use ideas from probability theory in the frequency domain to provide probabilistic verification guarantees for ReLU neural networks. Specifically, we interpret a (deep) feedforward neural network as a discrete dynamical system over a finite horizon that shapes distributions of initial states, and use characteristic functions to propagate the distribution of the input data through the network. Using the inverse Fourier transform, we obtain the corresponding cumulative distribution function of the output set, which can be used to check if the network is performing as expected given any random point from the input set. The proposed approach does not require distributions to have well-defined moments or moment generating functions. We demonstrate our proposed approach on two examples, and compare its performance to related approaches. △ Less

Submitted 3 December, 2022; originally announced December 2022.

Comments: Submitted to L4DC 2023

AstroSLAM: Autonomous Monocular Navigation in the Vicinity of a Celestial Small Body -- Theory and Experiments

Authors: Mehregan Dor, Travis Driver, Kenneth Getzandanner, Panagiotis Tsiotras

Abstract: We propose AstroSLAM, a standalone vision-based solution for autonomous online navigation around an unknown target small celestial body. AstroSLAM is predicated on the formulation of the SLAM problem as an incrementally growing factor graph, facilitated by the use of the GTSAM library and the iSAM2 engine. By combining sensor fusion with orbital motion priors, we achieve improved performance over… ▽ More We propose AstroSLAM, a standalone vision-based solution for autonomous online navigation around an unknown target small celestial body. AstroSLAM is predicated on the formulation of the SLAM problem as an incrementally growing factor graph, facilitated by the use of the GTSAM library and the iSAM2 engine. By combining sensor fusion with orbital motion priors, we achieve improved performance over a baseline SLAM solution. We incorporate orbital motion constraints into the factor graph by devising a novel relative dynamics factor, which links the relative pose of the spacecraft to the problem of predicting trajectories stemming from the motion of the spacecraft in the vicinity of the small body. We demonstrate the excellent performance of AstroSLAM using both real legacy mission imagery and trajectory data courtesy of NASA's Planetary Data System, as well as real in-lab imagery data generated on a 3 degree-of-freedom spacecraft simulator test-bed. △ Less

Submitted 1 December, 2022; originally announced December 2022.

Comments: 28 pages, 19 figures. View AstroSLAM in action: https://youtu.be/Uf8j-AxfMM0

Optimal Covariance Steering for Discrete-Time Linear Stochastic Systems

Authors: Fengjiao Liu, George Rapakoulias, Panagiotis Tsiotras

Abstract: In this paper, we study the optimal control problem for steering the state covariance of a discrete-time linear stochastic system over a finite time horizon. First, we establish the existence and uniqueness of the optimal control law for a quadratic cost function. Then, we show the separation of the optimal mean and the covariance steering problems. We also develop efficient computational methods… ▽ More In this paper, we study the optimal control problem for steering the state covariance of a discrete-time linear stochastic system over a finite time horizon. First, we establish the existence and uniqueness of the optimal control law for a quadratic cost function. Then, we show the separation of the optimal mean and the covariance steering problems. We also develop efficient computational methods for solving for the optimal control law, which is identified as the solution to a semi-definite program. The effectiveness of the proposed approach is demonstrated through numerical examples. In the process, we also obtain some novel theoretical results for a matrix Riccati difference equation, which may be of independent interest. △ Less

Submitted 26 January, 2023; v1 submitted 1 November, 2022; originally announced November 2022.

Distributionally Robust Covariance Steering with Optimal Risk Allocation

Authors: Venkatraman Renganathan, Joshua Pilipovsky, Panagiotis Tsiotras

Abstract: This article extends the optimal covariance steering (CS) problem for discrete time linear stochastic systems modeled using moment-based ambiguity sets. To hedge against the uncertainty in the state distributions while performing covariance steering, distributionally robust risk constraints are employed during the optimal allocation of the risk. Specifically, a distributionally robust iterative ri… ▽ More This article extends the optimal covariance steering (CS) problem for discrete time linear stochastic systems modeled using moment-based ambiguity sets. To hedge against the uncertainty in the state distributions while performing covariance steering, distributionally robust risk constraints are employed during the optimal allocation of the risk. Specifically, a distributionally robust iterative risk allocation (DR-IRA) formalism is used to solve the optimal risk allocation problem for the CS problem using a two-stage approach. The upper-stage of DR-IRA is a convex problem that optimizes the risk, while the lower-stage optimizes the controller with the new distributionally robust risk constraints. The proposed framework results in solutions that are robust against arbitrary distributions in the considered ambiguity set. Finally, we demonstrate our proposed approach using numerical simulations. Addressing the covariance steering problem through the lens of distributional robustness marks the novel contribution of this article. △ Less

Submitted 7 November, 2022; v1 submitted 30 September, 2022; originally announced October 2022.

On the Adversarial Convex Body Chasing Problem

Authors: Yue Guan, Longxu Pan, Daigo Shishika, Panagiotis Tsiotras

Abstract: In this work, we extend the convex bodies chasing problem (CBC) to an adversarial setting, where an agent (the Player) is tasked with chasing a sequence of convex bodies generated adversarially by another agent (the Opponent). The Player aims to minimize the total cost associated with its own movements, while the Opponent tries to maximize the same cost. The set of feasible convex bodies is finite… ▽ More In this work, we extend the convex bodies chasing problem (CBC) to an adversarial setting, where an agent (the Player) is tasked with chasing a sequence of convex bodies generated adversarially by another agent (the Opponent). The Player aims to minimize the total cost associated with its own movements, while the Opponent tries to maximize the same cost. The set of feasible convex bodies is finite and known to both agents, which allows us to provide performance guarantees with max-min optimality. Under certain assumptions, we show the continuity of the optimal value function, and propose an algorithm to numerically approximate the optimal policies for both the Player and the Opponent within a guaranteed tolerance. Finally, the theoretical results are verified through numerical examples. △ Less

Submitted 17 March, 2023; v1 submitted 27 September, 2022; originally announced September 2022.

Risk-Aware Model Predictive Path Integral Control Using Conditional Value-at-Risk

Authors: Ji Yin, Zhiyuan Zhang, Panagiotis Tsiotras

Abstract: In this paper, we present a novel Model Predictive Control method for autonomous robots subject to arbitrary forms of uncertainty. The proposed Risk-Aware Model Predictive Path Integral (RA-MPPI) control utilizes the Conditional Value-at-Risk (CVaR) measure to generate optimal control actions for safety-critical robotic applications. Different from most existing Stochastic MPCs and CVaR optimizati… ▽ More In this paper, we present a novel Model Predictive Control method for autonomous robots subject to arbitrary forms of uncertainty. The proposed Risk-Aware Model Predictive Path Integral (RA-MPPI) control utilizes the Conditional Value-at-Risk (CVaR) measure to generate optimal control actions for safety-critical robotic applications. Different from most existing Stochastic MPCs and CVaR optimization methods that linearize the original dynamics and formulate control tasks as convex programs, the proposed method directly uses the original dynamics without restricting the form of the cost functions or the noise. We apply the novel RA-MPPI controller to an autonomous vehicle to perform aggressive driving maneuvers in cluttered environments. Our simulations and experiments show that the proposed RA-MPPI controller can achieve about the same lap time with significantly fewer collisions compared to the baseline MPPI controller. The proposed controller performs on-line computation at an update frequency of up to 80Hz, utilizing modern Graphics Processing Units (GPUs) to multi-thread the generation of trajectories as well as the CVaR values. △ Less

Submitted 26 September, 2022; originally announced September 2022.

Comments: 7 pages, 7 figures

Optimal Covariance Steering for Continuous-Time Linear Stochastic Systems With Multiplicative Noise

Authors: Fengjiao Liu, Panagiotis Tsiotras

Abstract: In this paper we study the finite-horizon optimal covariance steering problem for a continuous-time linear stochastic system subject to both additive and multiplicative noise. The noise can be continuous or it may contain jumps. Additive noise does not depend on the state or the control, whereas multiplicative noise has a magnitude proportional to the current state. The cost is assumed to be quadr… ▽ More In this paper we study the finite-horizon optimal covariance steering problem for a continuous-time linear stochastic system subject to both additive and multiplicative noise. The noise can be continuous or it may contain jumps. Additive noise does not depend on the state or the control, whereas multiplicative noise has a magnitude proportional to the current state. The cost is assumed to be quadratic in both the state and the control. First, the controllability of the state covariance is established under mild assumptions. Then, the optimal control for steering the covariance is provided. Lastly, the existence and uniqueness of the optimal control is shown. In the process, we provide a result of independent interest regarding the maximal interval of existence of the solution to a matrix Riccati differential equation. △ Less

Submitted 26 January, 2023; v1 submitted 23 June, 2022; originally announced June 2022.

Optimal Covariance Steering for Continuous-Time Linear Stochastic Systems With Additive Noise

Authors: Fengjiao Liu, Panagiotis Tsiotras

Abstract: In this paper, we study the problem of how to optimally steer the state covariance of a general continuous-time linear stochastic system over a finite time interval subject to additive noise. Optimality here means reaching a target state covariance with minimal control energy. The additive noise may include a combination of white Gaussian noise and abrupt "jump noise" that is discontinuous in time… ▽ More In this paper, we study the problem of how to optimally steer the state covariance of a general continuous-time linear stochastic system over a finite time interval subject to additive noise. Optimality here means reaching a target state covariance with minimal control energy. The additive noise may include a combination of white Gaussian noise and abrupt "jump noise" that is discontinuous in time. We first establish the controllability of the state covariance for linear time-varying stochastic systems. We then turn to the derivation of the optimal control, which entails solving two dynamically coupled matrix ordinary differential equations (ODEs) with split boundary conditions. We show the existence and uniqueness of the solution to these coupled matrix ODEs, and thus those of the optimal control. △ Less

Submitted 15 February, 2023; v1 submitted 22 June, 2022; originally announced June 2022.

Comments: 8 pages, 2 figures

Hierarchical Decompositions of Stochastic Pursuit-Evasion Games

Authors: Yue Guan, Mohammad Afshari, Qifan Zhang, Panagiotis Tsiotras

Abstract: In this work we present a hierarchical framework for solving discrete stochastic pursuit-evasion games (PEGs) in large grid worlds. With a partition of the grid world into superstates (e.g., "rooms"), the proposed approach creates a two-resolution decision-making process, which consists of a set of local PEGs at the original state level and an aggregated PEG at the superstate level. Having much sm… ▽ More In this work we present a hierarchical framework for solving discrete stochastic pursuit-evasion games (PEGs) in large grid worlds. With a partition of the grid world into superstates (e.g., "rooms"), the proposed approach creates a two-resolution decision-making process, which consists of a set of local PEGs at the original state level and an aggregated PEG at the superstate level. Having much smaller cardinality, both the local games and the aggregated game can be easily solved to a Nash equilibrium. To connect the decision-making at the two resolutions, we use the Nash values of the local PEGs as the rewards for the aggregated game. Through numerical simulations, we show that the proposed hierarchical framework significantly reduces the computation overhead, while still maintaining a satisfactory level of performance when competing against the flat Nash policies. △ Less

Submitted 14 September, 2022; v1 submitted 2 May, 2022; originally announced May 2022.

Sha** Large Population Agent Behaviors Through Entropy-Regularized Mean-Field Games

Authors: Yue Guan, Mi Zhou, Ali Pakniyat, Panagiotis Tsiotras

Abstract: Mean-field games (MFG) were introduced to efficiently analyze approximate Nash equilibria in large population settings. In this work, we consider entropy-regularized mean-field games with a finite state-action space in a discrete time setting. We show that entropy regularization provides the necessary regularity conditions, that are lacking in the standard finite mean field games. Such regularity… ▽ More Mean-field games (MFG) were introduced to efficiently analyze approximate Nash equilibria in large population settings. In this work, we consider entropy-regularized mean-field games with a finite state-action space in a discrete time setting. We show that entropy regularization provides the necessary regularity conditions, that are lacking in the standard finite mean field games. Such regularity conditions enable us to design fixed-point iteration algorithms to find the unique mean-field equilibrium (MFE). Furthermore, the reference policy used in the regularization provides an extra parameter, through which one can control the behavior of the population. We first consider a stochastic game with a large population of $N$ homogeneous agents. We establish conditions for the existence of a Nash equilibrium in the limiting case as $N$ tends to infinity, and we demonstrate that the Nash equilibrium for the infinite population case is also an $ε$-Nash equilibrium for the $N$-agent system, where the sub-optimality $ε$ is of order $\mathcal{O}\big(1/\sqrt{N}\big)$. Finally, we verify the theoretical guarantees through a resource allocation example and demonstrate the efficacy of using a reference policy to control the behavior of a large population. △ Less

Submitted 22 July, 2022; v1 submitted 14 October, 2021; originally announced October 2021.

Batch Belief Trees for Motion Planning Under Uncertainty

Authors: Dongliang Zheng, Panagiotis Tsiotras

Abstract: In this work, we develop the Batch Belief Trees (BBT) algorithm for motion planning under motion and sensing uncertainties. The algorithm interleaves between batch sampling, building a graph of nominal trajectories in the state space, and searching over the graph to find belief space motion plans. By searching over the graph, BBT finds sophisticated plans that will visit (and revisit) information-… ▽ More In this work, we develop the Batch Belief Trees (BBT) algorithm for motion planning under motion and sensing uncertainties. The algorithm interleaves between batch sampling, building a graph of nominal trajectories in the state space, and searching over the graph to find belief space motion plans. By searching over the graph, BBT finds sophisticated plans that will visit (and revisit) information-rich regions to reduce uncertainty. One of the key benefits of this algorithm is the modified interplay between exploration and exploitation. Instead of an exhaustive search (exploitation) after one exploration step, the proposed algorithm uses batch samples to explore the state space and, in addition, does not require exhaustive search before the next iteration of batch sampling, which adds flexibility.The algorithm finds motion plans that converge to the optimal one as more samples are added to the graph. We test BBT in different planning environments. Our numerical investigation confirms that BBT finds non-trivial motion plans and is faster compared with previous similar methods. △ Less

Submitted 21 April, 2023; v1 submitted 30 September, 2021; originally announced October 2021.

Accelerating Kinodynamic RRT* Through Dimensionality Reduction

Authors: Dongliang Zheng, Panagiotis Tsiotras

Abstract: Sampling-based motion planning algorithms such as RRT* are well-known for their ability to quickly find an initial solution and then converge to the optimal solution asymptotically. However, the convergence rate can be slow for highdimensional planning problems, particularly for dynamical systems where the sampling space is not just the configuration space but the full state space. In this paper,… ▽ More Sampling-based motion planning algorithms such as RRT* are well-known for their ability to quickly find an initial solution and then converge to the optimal solution asymptotically. However, the convergence rate can be slow for highdimensional planning problems, particularly for dynamical systems where the sampling space is not just the configuration space but the full state space. In this paper, we introduce the idea of using a partial-final-state-free (PFF) optimal controller in kinodynamic RRT* [1] to reduce the dimensionality of the sampling space. Instead of sampling the full state space, the proposed accelerated kinodynamic RRT*, called Kino-RRT*, only samples part of the state space, while the rest of the states are selected by the PFF optimal controller. We also propose a delayed and intermittent update of the optimal arrival time of all the edges in the RRT* tree to decrease the computation complexity of the algorithm. We tested the proposed algorithm using 4-D and 10-D state-space linear systems and showed that Kino-RRT* converges much faster than the kinodynamic RRT* algorithm. △ Less

Submitted 2 July, 2021; originally announced July 2021.

Belief Space Planning: A Covariance Steering Approach

Authors: Dongliang Zheng, Jack Ridderhof, Panagiotis Tsiotras, Ali-akbar Agha-mohammadi

Abstract: A new belief space planning algorithm, called covariance steering Belief RoadMap (CS-BRM), is introduced, which is a multi-query algorithm for motion planning of dynamical systems under simultaneous motion and observation uncertainties. CS-BRM extends the probabilistic roadmap (PRM) approach to belief spaces and is based on the recently developed theory of covariance steering (CS) that enables gua… ▽ More A new belief space planning algorithm, called covariance steering Belief RoadMap (CS-BRM), is introduced, which is a multi-query algorithm for motion planning of dynamical systems under simultaneous motion and observation uncertainties. CS-BRM extends the probabilistic roadmap (PRM) approach to belief spaces and is based on the recently developed theory of covariance steering (CS) that enables guaranteed satisfaction of terminal belief constraints in finite-time. The nodes in the CS-BRM are sampled in belief space and represent distributions of the system states. A covariance steering controller steers the system from one BRM node to another, thus acting as an edge controller of the corresponding belief graph that ensures belief constraint satisfaction. After the edge controller is computed, a specific edge cost is assigned to that edge. The CS-BRM algorithm allows the sampling of non-stationary belief nodes, and thus is able to explore the velocity space and find efficient motion plans. The performance of CS-BRM is evaluated and compared to a previous belief space planning method, demonstrating the benefits of the proposed approach. △ Less

Submitted 24 May, 2021; originally announced May 2021.

Value Function Estimators for Feynman-Kac Forward-Backward SDEs in Stochastic Optimal Control

Authors: Kelsey P. Hawkins, Ali Pakniyat, Panagiotis Tsiotras

Abstract: Two novel numerical estimators are proposed for solving forward-backward stochastic differential equations (FBSDEs) appearing in the Feynman-Kac representation of the value function in stochastic optimal control problems. In contrast to the current numerical approaches which are based on the discretization of the continuous-time FBSDE, we propose a converse approach, namely, we obtain a discrete-t… ▽ More Two novel numerical estimators are proposed for solving forward-backward stochastic differential equations (FBSDEs) appearing in the Feynman-Kac representation of the value function in stochastic optimal control problems. In contrast to the current numerical approaches which are based on the discretization of the continuous-time FBSDE, we propose a converse approach, namely, we obtain a discrete-time approximation of the on-policy value function, and then we derive a discrete-time estimator that resembles the continuous-time counterpart. The proposed approach allows for the construction of higher accuracy estimators along with error analysis. The approach is applied to the policy improvement step in reinforcement learning. Numerical results and error analysis are demonstrated using (i) a scalar nonlinear stochastic optimal control problem and (ii) a four-dimensional linear quadratic regulator (LQR) problem. The proposed estimators show significant improvement in terms of accuracy in both cases over Euler-Maruyama-based estimators used in competing approaches. In the case of LQR problems, we demonstrate that our estimators result in near machine-precision level accuracy, in contrast to previously proposed methods that can potentially diverge on the same problems. △ Less

Submitted 30 September, 2021; v1 submitted 25 March, 2021; originally announced March 2021.

Comments: arXiv admin note: text overlap with arXiv:2006.12444

Stochastic Entry Guidance

Authors: Jack Ridderhof, Panagiotis Tsiotras, Breanna J. Johnson

Abstract: In this paper, closed-loop entry guidance in a randomly perturbed atmosphere, using bank angle control, is posed as a stochastic optimal control problem. The entry trajectory, as well as the closed-loop controls, are both modeled as random processes with statistics determined by the entry dynamics, the entry guidance, and the probabilistic structure of altitude-dependent atmospheric density variat… ▽ More In this paper, closed-loop entry guidance in a randomly perturbed atmosphere, using bank angle control, is posed as a stochastic optimal control problem. The entry trajectory, as well as the closed-loop controls, are both modeled as random processes with statistics determined by the entry dynamics, the entry guidance, and the probabilistic structure of altitude-dependent atmospheric density variations. The entry guidance, which is parameterized as a sequence of linear feedback gains, is designed to steer the probability distribution of the entry trajectories while satisfying bounds on the allowable control inputs and on the maximum allowable state errors. Numerical simulations of a Mars entry scenario demonstrate improved range targeting performance with approximately 50% lower 1st and 99th percentile final range errors when using the developed stochastic guidance scheme as compared to the existing Apollo final phase algorithm. △ Less

Submitted 17 January, 2022; v1 submitted 8 March, 2021; originally announced March 2021.

Multi-Agent Consensus Subject to Communication and Privacy Constraints

Authors: Dipankar Maity, Panagiotis Tsiotras

Abstract: We consider a multi-agent consensus problem in the presence of adversarial agents. The adversaries are able to listen to the inter-agent communications and try to estimate the state of the agents. The agents have a limited bit-rate for communication and are required to quantize the transmitted signal in order to meet the bit-rate constraint of the communication channel. We propose a consensus prot… ▽ More We consider a multi-agent consensus problem in the presence of adversarial agents. The adversaries are able to listen to the inter-agent communications and try to estimate the state of the agents. The agents have a limited bit-rate for communication and are required to quantize the transmitted signal in order to meet the bit-rate constraint of the communication channel. We propose a consensus protocol that is protected against the adversaries, i.e., the expected mean-square error of the adversary state estimate is lower bounded. In order to deal with the bit-rate constraint, we propose a dynamic quantization scheme that guarantees protected consensus. △ Less

Submitted 21 February, 2021; originally announced February 2021.

Chance-Constrained Covariance Steering in a Gaussian Random Field via Successive Convex Programming

Authors: Jack Ridderhof, Panagiotis Tsiotras

Abstract: The problem of optimizing affine feedback laws that explicitly steer the mean and covariance of an uncertain system state in the presence of a Gaussian random field is considered. Spatially-dependent disturbances are successively approximated with respect to a nominal trajectory by a sequence of jointly Gaussian random vectors. Sequential updates to the nominal control inputs are computed via conv… ▽ More The problem of optimizing affine feedback laws that explicitly steer the mean and covariance of an uncertain system state in the presence of a Gaussian random field is considered. Spatially-dependent disturbances are successively approximated with respect to a nominal trajectory by a sequence of jointly Gaussian random vectors. Sequential updates to the nominal control inputs are computed via convex optimization that includes the effect of affine state feedback, the perturbing effects of spatial disturbances, and chance constraints on the closed-loop state and control. The developed method is applied to solve for an affine feedback law to minimize the 99th percentile of $Δv$ required to complete an aerocapture mission around a planet with a randomly disturbed atmosphere. △ Less

Submitted 17 January, 2022; v1 submitted 23 January, 2021; originally announced January 2021.

A Generalized A* Algorithm for Finding Globally Optimal Paths in Weighted Colored Graphs

Authors: Jaein Lim, Panagiotis Tsiotras

Abstract: Both geometric and semantic information of the search space is imperative for a good plan. We encode those properties in a weighted colored graph (geometric information in terms of edge weight and semantic information in terms of edge and vertex color), and propose a generalized A* to find the shortest path among the set of paths with minimal inclusion of low-ranked color edges. We prove the compl… ▽ More Both geometric and semantic information of the search space is imperative for a good plan. We encode those properties in a weighted colored graph (geometric information in terms of edge weight and semantic information in terms of edge and vertex color), and propose a generalized A* to find the shortest path among the set of paths with minimal inclusion of low-ranked color edges. We prove the completeness and optimality of this Class-Ordered A* (COA*) algorithm with respect to the hereto defined notion of optimality. The utility of COA* is numerically validated in a ternary graph with feasible, infeasible, and unknown vertices and edges for the cases of a 2D mobile robot, a 3D robotic arm, and a 5D robotic arm with limited sensing capabilities. We compare the results of COA* to that of the regular A* algorithm, the latter of which finds the shortest path regardless of uncertainty, and we show that the COA* dominates the A* solution in terms of finding less uncertain paths. △ Less

Submitted 23 December, 2020; originally announced December 2020.

Learning Nash Equilibria in Zero-Sum Stochastic Games via Entropy-Regularized Policy Approximation

Authors: Yue Guan, Qifan Zhang, Panagiotis Tsiotras

Abstract: We explore the use of policy approximations to reduce the computational cost of learning Nash equilibria in zero-sum stochastic games. We propose a new Q-learning type algorithm that uses a sequence of entropy-regularized soft policies to approximate the Nash policy during the Q-function updates. We prove that under certain conditions, by updating the regularized Q-function, the algorithm converge… ▽ More We explore the use of policy approximations to reduce the computational cost of learning Nash equilibria in zero-sum stochastic games. We propose a new Q-learning type algorithm that uses a sequence of entropy-regularized soft policies to approximate the Nash policy during the Q-function updates. We prove that under certain conditions, by updating the regularized Q-function, the algorithm converges to a Nash equilibrium. We also demonstrate the proposed algorithm's ability to transfer previous training experiences, enabling the agents to adapt quickly to new environments. We provide a dynamic hyper-parameter scheduling scheme to further expedite convergence. Empirical results applied to a number of stochastic games verify that the proposed algorithm converges to the Nash equilibrium, while exhibiting a major speed-up over existing algorithms. △ Less

Submitted 27 June, 2021; v1 submitted 31 August, 2020; originally announced September 2020.

Comments: Accepted at IJCAI-21

Forward-Backward Rapidly-Exploring Random Trees for Stochastic Optimal Control

Authors: Kelsey P. Hawkins, Ali Pakniyat, Evangelos Theodorou, Panagiotis Tsiotras

Abstract: We propose a numerical method for the computation of the forward-backward stochastic differential equations (FBSDE) appearing in the Feynman-Kac representation of the value function in stochastic optimal control problems. By the use of the Girsanov change of probability measures, it is demonstrated how a rapidly-exploring random tree (RRT) method can be utilized for the forward integration pass, a… ▽ More We propose a numerical method for the computation of the forward-backward stochastic differential equations (FBSDE) appearing in the Feynman-Kac representation of the value function in stochastic optimal control problems. By the use of the Girsanov change of probability measures, it is demonstrated how a rapidly-exploring random tree (RRT) method can be utilized for the forward integration pass, as long as the controlled drift terms are appropriately compensated in the backward integration pass. Subsequently, a numerical approximation of the value function is proposed by solving a series of function approximation problems backwards in time along the edges of the constructed RRT. Moreover, a local entropy-weighted least squares Monte Carlo (LSMC) method is developed to concentrate function approximation accuracy in regions most likely to be visited by optimally controlled trajectories. The results of the proposed methodology are demonstrated on linear and nonlinear stochastic optimal control problems with non-quadratic running costs, which reveal significant convergence improvements over previous FBSDE-based numerical solution methods. △ Less

Submitted 25 March, 2021; v1 submitted 22 June, 2020; originally announced June 2020.

Apollonius Allocation Algorithm for Heterogeneous Pursuers to Capture Multiple Evaders

Authors: Venkata Ramana Makkapati, Panagiotis Tsiotras

Abstract: In this paper, we address pursuit-evasion problems involving multiple pursuers and multiple evaders. The pursuer and the evader teams are assumed to be heterogeneous, in the sense that each team has agents with different speed capabilities. The pursuers are all assumed to be following a constant bearing strategy. A dynamic divide and conquer approach, where at every time instant each evader is ass… ▽ More In this paper, we address pursuit-evasion problems involving multiple pursuers and multiple evaders. The pursuer and the evader teams are assumed to be heterogeneous, in the sense that each team has agents with different speed capabilities. The pursuers are all assumed to be following a constant bearing strategy. A dynamic divide and conquer approach, where at every time instant each evader is assigned to a set of pursuers based on the instantaneous positions of all the players, is introduced to solve the multi-agent pursuit problem. In this regard, the corresponding multi-pursuer single-evader problem is analyzed first. Assuming that the evader can follow any strategy, a dynamic task allocation algorithm is proposed for the pursuers. The algorithm is based on the well-known Apollonius circle and allows the pursuers to allocate their resources in an intelligent manner while guaranteeing the capture of the evader in minimum time. The proposed algorithm is then extended to assign pursuers in multi-evader settings that is proven to capture all the evaders in finite time. △ Less

Submitted 18 June, 2020; v1 submitted 17 June, 2020; originally announced June 2020.

Bounded-Rational Pursuit-Evasion Games

Authors: Yue Guan, Dipankar Maity, Christopher M. Kroninger, Panagiotis Tsiotras

Abstract: We present a framework that incorporates the idea of bounded rationality into dynamic stochastic pursuit-evasion games. The solution of a stochastic game is characterized, in general, by its (Nash) equilibria in feedback form. However, computing these Nash equilibrium strategies may require extensive computational resources. In this paper, the agents are modeled as bounded rational entities having… ▽ More We present a framework that incorporates the idea of bounded rationality into dynamic stochastic pursuit-evasion games. The solution of a stochastic game is characterized, in general, by its (Nash) equilibria in feedback form. However, computing these Nash equilibrium strategies may require extensive computational resources. In this paper, the agents are modeled as bounded rational entities having limited computational resources. We illustrate the framework by applying it to a pursuit-evasion game between two vehicles in a stochastic wind field, where both the pursuer and the evader are bounded rational. We show how such a game may be analyzed by properly casting it as an iterative sequence of finite-state Markov Decision Processes (MDPs). Leveraging tools and algorithms from cognitive hierarchy theory ("level-$k$ thinking") we compute the solution of the ensuing discrete game, while taking into consideration the rationality level of each agent. We also present an online algorithm for each agent to infer its opponent rationality level. △ Less

Submitted 15 March, 2020; originally announced March 2020.

Comments: 16 pages, 5 figures, submitted to Journal of Guidance, Control, and Dynamics

Covariance Steering for Discrete-Time Linear-Quadratic Stochastic Dynamic Games

Authors: Venkata Ramana Makkapati, Tanmay Rajpurohit, Kazuhide Okamoto, Panagiotis Tsiotras

Abstract: This paper addresses the problem of steering a discrete-time linear dynamical system from an initial Gaussian distribution to a final distribution in a game-theoretic setting. One of the two players strives to minimize a quadratic payoff, while at the same time tries to meet a given mean and covariance constraint at the final time-step. The other player maximizes the same payoff, but it is assumed… ▽ More This paper addresses the problem of steering a discrete-time linear dynamical system from an initial Gaussian distribution to a final distribution in a game-theoretic setting. One of the two players strives to minimize a quadratic payoff, while at the same time tries to meet a given mean and covariance constraint at the final time-step. The other player maximizes the same payoff, but it is assumed to be indifferent to the terminal constraint. At first, the unconstrained version of the game is examined, and the necessary conditions for the existence of a saddle point are obtained. We then show that obtaining a solution for the one-sided constrained dynamic game is not guaranteed, and subsequently the players' best responses are analyzed. Finally, we propose to numerically solve the problem of steering the distribution under adversarial scenarios using the Jacobi iteration method. The problem of guiding a missile during the endgame is chosen to analyze the proposed approach. A numerical simulation corresponding to the case where the terminal distribution is not achieved is also included, and discuss the necessary conditions to meet the terminal constraint. △ Less

Submitted 6 March, 2020; originally announced March 2020.

Safe Optimal Control under Parametric Uncertainties

Authors: Hemanth Sarabu, Venkata Ramana Makkapati, Vinodhini Comandur, Panagiotis Tsiotras, Seth Hutchinson

Abstract: We address the issue of safe optimal path planning under parametric uncertainties using a novel regularizer that allows trading off optimality with safety. The proposed regularizer leverages the notion that collisions may be modeled as constraint violations in an optimal control setting in order to produce open-loop trajectories with reduced risk of collisions. The risk of constraint violation is… ▽ More We address the issue of safe optimal path planning under parametric uncertainties using a novel regularizer that allows trading off optimality with safety. The proposed regularizer leverages the notion that collisions may be modeled as constraint violations in an optimal control setting in order to produce open-loop trajectories with reduced risk of collisions. The risk of constraint violation is evaluated using a state-dependent relevance function and first-order variations in the constraint function with respect to parametric variations. The approach is generic and can be adapted to any optimal control formulation that deals with constraints under parametric uncertainty. Simulations using a holonomic robot avoiding multiple dynamic obstacles with uncertain velocities are used to demonstrate the effectiveness of the proposed approach. Finally, we introduce the car vs. train problem to emphasize the dependence of the resultant risk aversion behavior on the form of the constraint function used to derive the regularizer. △ Less

Submitted 20 July, 2020; v1 submitted 25 February, 2020; originally announced February 2020.

Comments: Accepted to IEEE Robotics and Automation Letters

Optimal Controller Synthesis and Dynamic Quantizer Switching for Linear-Quadratic-Gaussian Systems

Authors: Dipankar Maity, Panagiotis Tsiotras

Abstract: In networked control systems, often the sensory signals are quantized before being transmitted to the controller. Consequently, performance is affected by the coarseness of this quantization process. Modern communication technologies allow users to obtain resolution-varying quantized measurements based on the prices paid. In this paper, we consider optimal controller synthesis of a Quantized-Feedb… ▽ More In networked control systems, often the sensory signals are quantized before being transmitted to the controller. Consequently, performance is affected by the coarseness of this quantization process. Modern communication technologies allow users to obtain resolution-varying quantized measurements based on the prices paid. In this paper, we consider optimal controller synthesis of a Quantized-Feedback Linear-Quadratic-Gaussian (QF-LQG) system where the measurements are to be quantized before being transmitted to the controller. The system is presented with several choices of quantizers, along with the cost of operating each quantizer. The objective is to jointly select the quantizers and the controller that would maintain an optimal balance between control performance and quantization cost. Under certain assumptions, this problem can be decoupled into two optimization problems: one for optimal controller synthesis and the other for optimal quantizer selection. We show that, similarly to the classical LQG problem, the optimal controller synthesis subproblem is characterized by Riccati equations. On the other hand, the optimal quantizer selection policy is found by solving a certain Markov-Decision-Process (MDP). △ Less

Submitted 31 January, 2020; originally announced January 2020.

Journal ref: IEEE Transactions on Automatic Control, 2021

C-DOC: Co-State Desensitized Optimal Control

Authors: Venkata Ramana Makkapati, Dipankar Maity, Mehregan Dor, Panagiotis Tsiotras

Abstract: In this paper, co-states are used to develop a framework that desensitizes the optimal cost. A general formulation for an optimal control problem with fixed final time is considered. The proposed scheme involves elevating the parameters of interest into states, and further augmenting the co-state equations of the optimal control problem to the dynamical model. A running cost that penalizes the co-… ▽ More In this paper, co-states are used to develop a framework that desensitizes the optimal cost. A general formulation for an optimal control problem with fixed final time is considered. The proposed scheme involves elevating the parameters of interest into states, and further augmenting the co-state equations of the optimal control problem to the dynamical model. A running cost that penalizes the co-states of the targeted parameters is then added to the original cost function. The solution obtained by minimizing the augmented cost yields a control which reduces the dispersion of the original cost with respect to parametric variations. The relationship between co-states and the cost-to-go function, for any given control law, is established substantiating the approach. Numerical examples and Monte-Carlo simulations that demonstrate the proposed scheme are discussed. △ Less

Submitted 30 September, 2019; originally announced October 2019.

Optimal Controller and Quantizer Selection for Partially Observable Linear-Quadratic-Gaussian Systems

Authors: Dipankar Maity, Panagiotis Tsiotras

Abstract: In networked control systems, often the sensory signals are quantized before being transmitted to the controller. Consequently, performance is affected by the coarseness of this quantization process. Modern communication technologies allow users to obtain resolution-varying quantized measurements based on the prices paid. In this paper, we consider joint optimal controller synthesis and quantizer… ▽ More In networked control systems, often the sensory signals are quantized before being transmitted to the controller. Consequently, performance is affected by the coarseness of this quantization process. Modern communication technologies allow users to obtain resolution-varying quantized measurements based on the prices paid. In this paper, we consider joint optimal controller synthesis and quantizer scheduling for a partially observed Quantized-Feedback Linear-Quadratic-Gaussian (QF-LQG) system, where the measurements are quantized before being sent to the controller. The system is presented with several choices of quantizers, along with the cost of using each quantizer. The objective is to jointly select the quantizers and synthesize the controller to strike an optimal balance between control performance and quantization cost. When the innovation signal is quantized instead of the measurement, the problem is decoupled into two optimization problems: one for optimal controller synthesis, and the other for optimal quantizer selection. The optimal controller is found by solving a Riccati equation and the optimal quantizer selection policy is found by solving a linear program (LP)- both of which can be solved offline. △ Less

Submitted 7 November, 2021; v1 submitted 30 September, 2019; originally announced September 2019.

AutoRally An open platform for aggressive autonomous driving

Authors: Brian Goldfain, Paul Drews, Changxi You, Matthew Barulic, Orlin Velev, Panagiotis Tsiotras, James M. Rehg

Abstract: This article presents AutoRally, a 1$:$5 scale robotics testbed for autonomous vehicle research. AutoRally is designed for robustness, ease of use, and reproducibility, so that a team of two people with limited knowledge of mechanical engineering, electrical engineering, and computer science can construct and then operate the testbed to collect real world autonomous driving data in whatever domain… ▽ More This article presents AutoRally, a 1$:$5 scale robotics testbed for autonomous vehicle research. AutoRally is designed for robustness, ease of use, and reproducibility, so that a team of two people with limited knowledge of mechanical engineering, electrical engineering, and computer science can construct and then operate the testbed to collect real world autonomous driving data in whatever domain they wish to study. Complete documentation to construct and operate the platform is available online along with tutorials, example controllers, and a driving dataset collected at the Georgia Tech Autonomous Racing Facility. Offline estimation algorithms are used to determine parameters for physics-based dynamics models using an adaptive limited memory joint state unscented Kalman filter. Online vehicle state estimation using a factor graph optimization scheme and a convolutional neural network for semantic segmentation of drivable surface are presented. All algorithms are tested with real world data from the fleet of six AutoRally robots at the Georgia Tech Autonomous Racing Facility tracks, and serve as a demonstration of the robot$'$s capabilities. △ Less

Submitted 2 June, 2018; originally announced June 2018.

Incremental Sampling-based Motion Planners Using Policy Iteration Methods

Authors: Oktay Arslan, Panagiotis Tsiotras

Abstract: Recent progress in randomized motion planners has led to the development of a new class of sampling-based algorithms that provide asymptotic optimality guarantees, notably the RRT* and the PRM* algorithms. Careful analysis reveals that the so-called "rewiring" step in these algorithms can be interpreted as a local policy iteration (PI) step (i.e., a local policy evaluation step followed by a local… ▽ More Recent progress in randomized motion planners has led to the development of a new class of sampling-based algorithms that provide asymptotic optimality guarantees, notably the RRT* and the PRM* algorithms. Careful analysis reveals that the so-called "rewiring" step in these algorithms can be interpreted as a local policy iteration (PI) step (i.e., a local policy evaluation step followed by a local policy improvement step) so that asymptotically, as the number of samples tend to infinity, both algorithms converge to the optimal path almost surely (with probability 1). Policy iteration, along with value iteration (VI) are common methods for solving dynamic programming (DP) problems. Based on this observation, recently, the RRT$^{\#}$ algorithm has been proposed, which performs, during each iteration, Bellman updates (aka "backups") on those vertices of the graph that have the potential of being part of the optimal path (i.e., the "promising" vertices). The RRT$^{\#}$ algorithm thus utilizes dynamic programming ideas and implements them incrementally on randomly generated graphs to obtain high quality solutions. In this work, and based on this key insight, we explore a different class of dynamic programming algorithms for solving shortest-path problems on random graphs generated by iterative sampling methods. These class of algorithms utilize policy iteration instead of value iteration, and thus are better suited for massive parallelization. Contrary to the RRT* algorithm, the policy improvement during the rewiring step is not performed only locally but rather on a set of vertices that are classified as "promising" during the current iteration. This tends to speed-up the whole process. The resulting algorithm, aptly named Policy Iteration-RRT$^{\#}$ (PI-RRT$^{\#}$) is the first of a new class of DP-inspired algorithms for randomized motion planning that utilize PI methods. △ Less

Submitted 19 September, 2016; originally announced September 2016.

Model Based Reinforcement Learning with Final Time Horizon Optimization

Authors: Wei Sun, Evangelos Theodorou, Panagiotis Tsiotras

Abstract: We present one of the first algorithms on model based reinforcement learning and trajectory optimization with free final time horizon. Grounded on the optimal control theory and Dynamic Programming, we derive a set of backward differential equations that propagate the value function and provide the optimal control policy and the optimal time horizon. The resulting policy generalizes previous resul… ▽ More We present one of the first algorithms on model based reinforcement learning and trajectory optimization with free final time horizon. Grounded on the optimal control theory and Dynamic Programming, we derive a set of backward differential equations that propagate the value function and provide the optimal control policy and the optimal time horizon. The resulting policy generalizes previous results in model based trajectory optimization. Our analysis shows that the proposed algorithm recovers the theoretical optimal solution on linear low dimensional problem. Finally we provide application results on nonlinear systems. △ Less

Submitted 3 September, 2015; originally announced September 2015.

Comments: 9 pages, 5 figures, NIPS2015

Information-Theoretic Stochastic Optimal Control via Incremental Sampling-based Algorithms

Authors: Oktay Arslan, Evangelos Theodorou, Panagiotis Tsiotras

Abstract: This paper considers optimal control of dynamical systems which are represented by nonlinear stochastic differential equations. It is well-known that the optimal control policy for this problem can be obtained as a function of a value function that satisfies a nonlinear partial differential equation, namely, the Hamilton-Jacobi-Bellman equation. This nonlinear PDE must be solved backwards in time,… ▽ More This paper considers optimal control of dynamical systems which are represented by nonlinear stochastic differential equations. It is well-known that the optimal control policy for this problem can be obtained as a function of a value function that satisfies a nonlinear partial differential equation, namely, the Hamilton-Jacobi-Bellman equation. This nonlinear PDE must be solved backwards in time, and this computation is intractable for large scale systems. Under certain assumptions, and after applying a logarithmic transformation, an alternative characterization of the optimal policy can be given in terms of a path integral. Path Integral (PI) based control methods have recently been shown to provide elegant solutions to a broad class of stochastic optimal control problems. One of the implementation challenges with this formalism is the computation of the expectation of a cost functional over the trajectories of the unforced dynamics. Computing such expectation over trajectories that are sampled uniformly may induce numerical instabilities due to the exponentiation of the cost. Therefore, sampling of low-cost trajectories is essential for the practical implementation of PI-based methods. In this paper, we use incremental sampling-based algorithms to sample useful trajectories from the unforced system dynamics, and make a novel connection between Rapidly-exploring Random Trees (RRTs) and information-theoretic stochastic optimal control. We show the results from the numerical implementation of the proposed approach to several examples. △ Less

Submitted 28 May, 2014; originally announced May 2014.

Comments: 18 pages