Showing 1–50 of 73 results for author: Açıkmeşe

Six-Degree-of-Freedom Aircraft Landing Trajectory Planning with Runway Alignment

Authors: Taewan Kim, Abhinav G. Kamath, Niyousha Rahimi, Jasper Corleis, Behçet Açıkmeşe, Mehran Mesbahi

Abstract: This paper presents a numerical optimization algorithm for generating approach and landing trajectories for a six-degree-of-freedom (6-DoF) aircraft. We improve on the existing research on aircraft landing trajectory generation by formulating the trajectory optimization problem with additional real-world operational constraints, including 6-DoF aircraft dynamics, runway alignment, constant wind fi… ▽ More This paper presents a numerical optimization algorithm for generating approach and landing trajectories for a six-degree-of-freedom (6-DoF) aircraft. We improve on the existing research on aircraft landing trajectory generation by formulating the trajectory optimization problem with additional real-world operational constraints, including 6-DoF aircraft dynamics, runway alignment, constant wind field, and obstacle avoidance, to obtain a continuous-time nonconvex optimal control problem. Particularly, the runway alignment constraint enforces the trajectory of the aircraft to be aligned with the runway only during the final approach phase. This is a novel feature that is essential for preventing an approach that is either too steep or too shallow. The proposed method models the runway alignment constraint through a multi-phase trajectory planning scheme, imposing alignment conditions exclusively during the final approach phase. We compare this formulation with the existing state-triggered constraint formulation for runway alignment. To solve the formulated problem, we design a novel sequential convex programming algorithm called xPTR that extends the penalized trust-region (PTR) algorithm by incorporating an extrapolation step to expedite convergence. We validate the proposed method through extensive numerical simulations, including a Monte Carlo study, to evaluate the robustness of the algorithm to varying initial conditions. △ Less

Submitted 26 May, 2024; originally announced May 2024.

Comments: This article was submitted to JGCD

Optimization with Temporal and Logical Specifications via Generalized Mean-based Smooth Robustness Measures

Authors: Samet Uzun, Purnanand Elango, Pierre-Loic Garoche, Behcet Acikmese

Abstract: This paper introduces a generalized mean-based C^1-smooth robustness measure over discrete-time signals (D-GMSR) for signal temporal logic (STL) specifications. In conjunction with its C1-smoothness, D-GMSR is proven to be both sound and complete. Furthermore, it demonstrates favorable gradient properties and addresses locality and masking problems, which are critical for numerical optimization. T… ▽ More This paper introduces a generalized mean-based C^1-smooth robustness measure over discrete-time signals (D-GMSR) for signal temporal logic (STL) specifications. In conjunction with its C1-smoothness, D-GMSR is proven to be both sound and complete. Furthermore, it demonstrates favorable gradient properties and addresses locality and masking problems, which are critical for numerical optimization. The C^1-smoothness of the proposed formulations enables the implementation of robust and efficient numerical optimization algorithms to solve problems with STL specifications while preserving their theoretical guarantees. The practical utility of the proposed robustness measure is demonstrated on two real-world trajectory optimization problems: i) quadrotor flight, and ii) autonomous rocket landing. A sequential convex programming (SCP) framework, incorporating a convergence-guaranteed optimization algorithm (the prox-linear method) is used to solve inherently non-convex trajectory optimization problems with STL specifications. The implementation is available at https://github.com/UW-ACL/D-GMSR △ Less

Submitted 16 May, 2024; originally announced May 2024.

Successive Convexification for Nonlinear Model Predictive Control with Continuous-Time Constraint Satisfaction

Authors: Samet Uzun, Purnanand Elango, Abhinav G. Kamath, Taewan Kim, Behcet Acikmese

Abstract: We propose a nonlinear model predictive control (NMPC) framework based on a direct optimal control method that ensures continuous-time constraint satisfaction and accurate evaluation of the running cost, without compromising computational efficiency. We leverage the recently proposed successive convexification framework for trajectory optimization, where: (1) the path constraints and running cost… ▽ More We propose a nonlinear model predictive control (NMPC) framework based on a direct optimal control method that ensures continuous-time constraint satisfaction and accurate evaluation of the running cost, without compromising computational efficiency. We leverage the recently proposed successive convexification framework for trajectory optimization, where: (1) the path constraints and running cost are equivalently reformulated by augmenting the system dynamics, (2) multiple shooting is used for exact discretization, and (3) a convergence-guaranteed sequential convex programming (SCP) algorithm, the prox-linear method, is used to solve the discretized receding-horizon optimal control problems. The resulting NMPC framework is computationally efficient, owing to its support for warm-starting and premature termination of SCP, and its reliance on first-order information only. We demonstrate the effectiveness of the proposed NMPC framework by means of a numerical example with reference-tracking and obstacle avoidance. The implementation is available at https://github.com/UW-ACL/nmpc-ctcs △ Less

Submitted 26 April, 2024; originally announced May 2024.

Fast Monte Carlo Analysis for 6-DoF Powered-Descent Guidance via GPU-Accelerated Sequential Convex Programming

Authors: Govind M. Chari, Abhinav G. Kamath, Purnanand Elango, Behçet Açıkmeşe

Abstract: We introduce a GPU-accelerated Monte Carlo framework for nonconvex, free-final-time trajectory optimization problems. This framework makes use of the prox-linear method, which belongs to the larger family of sequential convex programming (SCP) algorithms, in conjunction with a constraint reformulation that guarantees inter-sample constraint satisfaction. Key features of this framework are: (1) con… ▽ More We introduce a GPU-accelerated Monte Carlo framework for nonconvex, free-final-time trajectory optimization problems. This framework makes use of the prox-linear method, which belongs to the larger family of sequential convex programming (SCP) algorithms, in conjunction with a constraint reformulation that guarantees inter-sample constraint satisfaction. Key features of this framework are: (1) continuous-time constraint satisfaction; (2) a matrix-inverse-free solution method; (3) the use of the proportional-integral projected gradient (PIPG) method, a first-order convex optimization solver, customized to the convex subproblem at hand; and, (4) an end-to-end, library-free implementation of the algorithm. We demonstrate this GPU-based framework on the 6-DoF powered-descent guidance problem, and show that it is faster than an equivalent serial CPU implementation for Monte Carlo simulations with over 1000 runs. To the best of our knowledge, this is the first GPU-based implementation of a general-purpose nonconvex trajectory optimization solver. △ Less

Submitted 27 April, 2024; originally announced April 2024.

Comments: AIAA SciTech 2024 Forum

Successive Convexification for Trajectory Optimization with Continuous-Time Constraint Satisfaction

Authors: Purnanand Elango, Dayou Luo, Abhinav G. Kamath, Samet Uzun, Taewan Kim, Behçet Açıkmeşe

Abstract: We present successive convexification, a real-time-capable solution method for nonconvex trajectory optimization, with continuous-time constraint satisfaction and guaranteed convergence, that only requires first-order information. The proposed framework combines several key methods to solve a large class of nonlinear optimal control problems: (i) exterior penalty-based reformulation of the path co… ▽ More We present successive convexification, a real-time-capable solution method for nonconvex trajectory optimization, with continuous-time constraint satisfaction and guaranteed convergence, that only requires first-order information. The proposed framework combines several key methods to solve a large class of nonlinear optimal control problems: (i) exterior penalty-based reformulation of the path constraints; (ii) generalized time-dilation; (iii) multiple-shooting discretization; (iv) $\ell_1$ exact penalization of the nonconvex constraints; and (v) the prox-linear method, a sequential convex programming (SCP) algorithm for convex-composite minimization. The reformulation of the path constraints enables continuous-time constraint satisfaction even on sparse discretization grids and obviates the need for mesh refinement heuristics. Through the prox-linear method, we guarantee convergence of the solution method to stationary points of the penalized problem and guarantee that the converged solutions that are feasible with respect to the discretized and control-parameterized optimal control problem are also Karush-Kuhn-Tucker (KKT) points. Furthermore, we highlight the specialization of this property to global minimizers of convex optimal control problems, wherein the reformulated path constraints cannot be represented by canonical cones, i.e., in the form required by existing convex optimization solvers. In addition to theoretical analysis, we demonstrate the effectiveness and real-time capability of the proposed framework with numerical examples based on popular optimal control applications: dynamic obstacle avoidance and rocket landing. △ Less

Submitted 25 April, 2024; originally announced April 2024.

Expected Time-Optimal Control: a Particle MPC-based Approach via Sequential Convex Programming

Authors: Kazuya Echigo, Abhishek Cauligi, Behçet Açıkmeşe

Abstract: In this paper, we consider the problem of minimum-time optimal control for a dynamical system with initial state uncertainties and propose a sequential convex programming (SCP) solution framework. We seek to minimize the expected terminal (mission) time, which is an essential capability for planetary exploration missions where ground rovers have to carry out scientific tasks efficiently within the… ▽ More In this paper, we consider the problem of minimum-time optimal control for a dynamical system with initial state uncertainties and propose a sequential convex programming (SCP) solution framework. We seek to minimize the expected terminal (mission) time, which is an essential capability for planetary exploration missions where ground rovers have to carry out scientific tasks efficiently within the mission timelines in uncertain environments. Our main contribution is to convert the underlying stochastic optimal control problem into a deterministic, numerically tractable, optimal control problem. To this end, the proposed solution framework combines two strategies from previous methods: i) a partial model predictive control with consensus horizon approach and ii) a sum-of-norm cost, a temporally strictly increasing weighted-norm, promoting minimum-time trajectories. Our contribution is to adopt these formulations into an SCP solution framework and obtain a numerically tractable stochastic control algorithm. We then demonstrate the resulting control method in multiple applications: i) a closed-loop linear system as a representative result (a spacecraft double integrator model), ii) an open-loop linear system (the same model), and then iii) a nonlinear system (Dubin's car). △ Less

Submitted 24 April, 2024; originally announced April 2024.

Comments: submitted to CDC 2024

Constraint Preconditioning and Parameter Selection for a First-Order Primal-Dual Method applied to Model Predictive Control

Authors: Govind M. Chari, Yue Yu, Behçet Açıkmeşe

Abstract: Many techniques for real-time trajectory optimization and control require the solution of optimization problems at high frequencies. However, ill-conditioning in the optimization problem can significantly reduce the speed of first-order primal-dual optimization algorithms. We introduce a preconditioning technique and step-size heuristic for Proportional-Integral Projected Gradient (PIPG), a first-… ▽ More Many techniques for real-time trajectory optimization and control require the solution of optimization problems at high frequencies. However, ill-conditioning in the optimization problem can significantly reduce the speed of first-order primal-dual optimization algorithms. We introduce a preconditioning technique and step-size heuristic for Proportional-Integral Projected Gradient (PIPG), a first-order primal-dual algorithm. The preconditioning technique, based on the QR factorization, aims to reduce the condition number of the KKT matrix associated with the optimization problem. Our step-size selection heuristic chooses step-sizes to minimize the upper bound on the convergence of the primal-dual gap for the optimization problem. These algorithms are tested on two model predictive control problem examples and show a solve-time reduction of at least 3.6x. △ Less

Submitted 22 March, 2024; originally announced March 2024.

Remarks on "Successive Convexification: A Superlinearly Convergent Algorithm for Non-convex Optimal Control Problems"

Authors: Dayou Luo, Purnanand Elango, Behcet Acikmese

Abstract: The purpose of this note is to highlight and address inaccuracies in the convergence guarantees of SCvx, a nonconvex trajectory optimization algorithm proposed by Mao et al. (arXiv:1804.06539), and make connections to relevant prior work. Specifically, we identify errors in the convergence proof within Mao et al. (arXiv:1804.06539) and reestablish the proof of convergence by employing a new method… ▽ More The purpose of this note is to highlight and address inaccuracies in the convergence guarantees of SCvx, a nonconvex trajectory optimization algorithm proposed by Mao et al. (arXiv:1804.06539), and make connections to relevant prior work. Specifically, we identify errors in the convergence proof within Mao et al. (arXiv:1804.06539) and reestablish the proof of convergence by employing a new method under stricter assumptions. △ Less

Submitted 13 March, 2024; v1 submitted 1 March, 2024; originally announced March 2024.

Sufficient LMI Copositivity Conditions for Funnel Synthesis of Uncertain Nonlinear Systems

Authors: Taewan Kim, Behçet Açıkmeşe

Abstract: Funnel synthesis refers to a procedure for synthesizing a time-varying controlled invariant set and an associated control law around a nominal trajectory. The computation of the funnel involves solving a continuous-time differential equation or inequality, ensuring the invariance of the funnel. Previous approaches often compromise the invariance property of the funnel; for example, they may enforc… ▽ More Funnel synthesis refers to a procedure for synthesizing a time-varying controlled invariant set and an associated control law around a nominal trajectory. The computation of the funnel involves solving a continuous-time differential equation or inequality, ensuring the invariance of the funnel. Previous approaches often compromise the invariance property of the funnel; for example, they may enforce the equation or the inequality only at discrete temporal nodes and not having formal guarantee of invariance at all times. This paper proposes a computational funnel synthesis method that can satisfy the invariance of the funnel without such compromises. We derive a finite number of linear matrix inequalities (LMIs) that imply the satifaction of a continuous-time differential linear matrix inequality guaranteeing the invariance of the funnel at all times from the initial to the final time. To this end, we utilize LMI conditions ensuring matrix copositivity, which then imply continuous-time invariance. The primary contribution of the paper is to prove that the resulting funnel is indeed invariant over a finite time horizon. We validate the proposed method via a three-dimensional trajectory planning and control problem with obstacle avoidance constraints. △ Less

Submitted 23 February, 2024; originally announced February 2024.

Comments: submitted to LCSS

Spacecraft Rendezvous Guidance via Factorization-Free Sequential Convex Programming using a First-Order Method

Authors: Govind M. Chari, Behçet Açıkmeşe

Abstract: We implement a fully factorization-free algorithm for nonconvex, free-final-time trajectory optimization. This algorithm is based on sequential convex programming and utilizes an inverse-free, exact discretization procedure to ensure dynamic feasibility of the converged trajectory and PIPG, a fast, first-order conic optimization algorithm as the subproblem solver. Although PIPG requires the tuning… ▽ More We implement a fully factorization-free algorithm for nonconvex, free-final-time trajectory optimization. This algorithm is based on sequential convex programming and utilizes an inverse-free, exact discretization procedure to ensure dynamic feasibility of the converged trajectory and PIPG, a fast, first-order conic optimization algorithm as the subproblem solver. Although PIPG requires the tuning of a hyperparameter to achieve fastest convergence, we show that PIPG can be tuned to a nominal trajectory optimization problem and it is robust to variations in initial condition. We demonstrate this with a monte carlo simulation of the free-final-time rendezvous problem, using Clohessy-Wiltshire dynamics, an impulsive thrust model, and various state and control constraints including a spherical keepout zone. △ Less

Submitted 6 February, 2024; originally announced February 2024.

Comments: AAS Rocky Mountain Guidance, Navigation and Control Conference, 2024

Blameless and Optimal Control under Prioritized Safety Constraints

Authors: Natalia Pavlasek, Sarah H. Q. Li, Behçet Açıkmeşe, Meeko Oishi, Claus Danielson

Abstract: In many resource-limited optimal control problems, multiple constraints may be enforced that are jointly infeasible due to external factors such as subsystem failures, unexpected disturbances, or fuel limitations. In this manuscript, we introduce the concept of blameless optimality to characterize control actions that a) satisfy the highest prioritized and feasible constraints and b) remain optima… ▽ More In many resource-limited optimal control problems, multiple constraints may be enforced that are jointly infeasible due to external factors such as subsystem failures, unexpected disturbances, or fuel limitations. In this manuscript, we introduce the concept of blameless optimality to characterize control actions that a) satisfy the highest prioritized and feasible constraints and b) remain optimal with respect to a mission objective. For a general optimal control problem with jointly infeasible constraints, we prove that a single optimization problem cannot find a blamelessly optimal control sequence. Instead, finding blamelessly optimal control actions requires sequentially solving at least two optimal control problems: one to determine the highest priority level of constraints that is feasible and another to determine the optimal control action with respect to these constraints. We apply our results to a rocket landing scenario in which violating at least one safety-induced landing constraint is unavoidable. Leveraging the concept of blameless optimality, we formulate blamelessly optimal controllers that can autonomously prioritize the constraints most critical to a mission. △ Less

Submitted 3 November, 2023; v1 submitted 13 April, 2023; originally announced April 2023.

HALO: Hazard-Aware Landing Optimization for Autonomous Systems

Authors: Christopher R. Hayner, Samuel C. Buckner, Daniel Broyles, Evelyn Madewell, Karen Leung, Behcet Acikmese

Abstract: With autonomous aerial vehicles enacting safety-critical missions, such as the Mars Science Laboratory Curiosity rover's landing on Mars, the tasks of automatically identifying and reasoning about potentially hazardous landing sites is paramount. This paper presents a coupled perception-planning solution which addresses the hazard detection, optimal landing trajectory generation, and contingency p… ▽ More With autonomous aerial vehicles enacting safety-critical missions, such as the Mars Science Laboratory Curiosity rover's landing on Mars, the tasks of automatically identifying and reasoning about potentially hazardous landing sites is paramount. This paper presents a coupled perception-planning solution which addresses the hazard detection, optimal landing trajectory generation, and contingency planning challenges encountered when landing in uncertain environments. Specifically, we develop and combine two novel algorithms, Hazard-Aware Landing Site Selection (HALSS) and Adaptive Deferred-Decision Trajectory Optimization (Adaptive-DDTO), to address the perception and planning challenges, respectively. The HALSS framework processes point cloud information to identify feasible safe landing zones, while Adaptive-DDTO is a multi-target contingency planner that adaptively replans as new perception information is received. We demonstrate the efficacy of our approach using a simulated Martian environment and show that our coupled perception-planning method achieves greater landing success whilst being more fuel efficient compared to a nonadaptive DDTO approach. △ Less

Submitted 4 April, 2023; originally announced April 2023.

Comments: The first two authors have contributed equally to this work. This work is to be published in the proceedings of the 2023 IEEE International Conference on Robotics and Automation (ICRA)

Optimization-based Constrained Funnel Synthesis for Systems with Lipschitz Nonlinearities via Numerical Optimal Control

Authors: Taewan Kim, Purnanand Elango, Taylor P. Reynolds, Behçet Açıkmeşe, Mehran Mesbahi

Abstract: This paper presents a funnel synthesis algorithm for computing controlled invariant sets and feedback control gains around a given nominal trajectory for dynamical systems with locally Lipschitz nonlinearities and bounded disturbances. The resulting funnel synthesis problem involves a differential linear matrix inequality (DLMI) whose solution satisfies a Lyapunov condition that implies invariance… ▽ More This paper presents a funnel synthesis algorithm for computing controlled invariant sets and feedback control gains around a given nominal trajectory for dynamical systems with locally Lipschitz nonlinearities and bounded disturbances. The resulting funnel synthesis problem involves a differential linear matrix inequality (DLMI) whose solution satisfies a Lyapunov condition that implies invariance and attractivity properties. Due to these properties, the proposed method can balance maximization of initial invariant funnel size, i.e., size of the funnel entry, and minimization of the size of the attractive funnel for attenuating the effect of disturbance. To solve the resulting funnel synthesis problem with the DLMI as constraints, we employ a numerical optimal control approach that uses a multiple shooting method to convert the problem into a finite dimensional semidefinite programming problem. This framework does not require piecewise linear system matrices and funnel parameters, which is typically assumed in recent related work. We illustrate the proposed funnel synthesis method with a numerical example. △ Less

Submitted 1 July, 2023; v1 submitted 18 March, 2023; originally announced March 2023.

Comments: 6 pages, 3 figures, accepted to LCSS

Reducing Collision Risk in Multi-Agent Path Planning: Application to Air traffic Management

Authors: Sarah H. Q. Li, Avi Mittal, Pierre-Loïc Garoche, Açıkmeşe, Behçet

Abstract: To minimize collision risks in the multi-agent path planning problem with stochastic transition dynamics, we formulate a Markov decision process congestion game with a multi-linear congestion cost. Players within the game complete individual tasks while minimizing their own collision risks. We show that the set of Nash equilibria coincides with the first-order KKT points of a non-convex optimizati… ▽ More To minimize collision risks in the multi-agent path planning problem with stochastic transition dynamics, we formulate a Markov decision process congestion game with a multi-linear congestion cost. Players within the game complete individual tasks while minimizing their own collision risks. We show that the set of Nash equilibria coincides with the first-order KKT points of a non-convex optimization problem. Our game is applied to a historical flight plan over France to reduce collision risks between commercial aircraft. △ Less

Submitted 10 December, 2022; v1 submitted 8 December, 2022; originally announced December 2022.

Comments: 6 pages, 2 figures

Real-Time Sequential Conic Optimization for Multi-Phase Rocket Landing Guidance

Authors: Abhinav G. Kamath, Purnanand Elango, Yue Yu, Skye Mceowen, Govind M. Chari, John M. Carson III, Behçet Açıkmeşe

Abstract: We introduce a multi-phase rocket landing guidance framework that can handle nonlinear dynamics and does not mandate any additional mixed-integer or nonconvex constraints to handle discrete temporal events/switching. To achieve this, we first introduce sequential conic optimization (SeCO), a new paradigm for solving nonconvex optimal control problems that is entirely devoid of matrix factorization… ▽ More We introduce a multi-phase rocket landing guidance framework that can handle nonlinear dynamics and does not mandate any additional mixed-integer or nonconvex constraints to handle discrete temporal events/switching. To achieve this, we first introduce sequential conic optimization (SeCO), a new paradigm for solving nonconvex optimal control problems that is entirely devoid of matrix factorizations and inversions. This framework combines sequential convex programming (SCP) and first-order conic optimization and can solve unified multi-phase trajectory optimization problems in real-time. The novel features of this framework are: (1) time-interval dilation, which enables multi-phase trajectory optimization with free-transition-time; (2) single-crossing compound state-triggered constraints, which are entirely convex if the trigger and constraint conditions are convex; (3) virtual state, which is a new approach to handling artificial infeasibility in SCP methods that preserves the shapes of the constraint sets; and, (4) the use of the proportional-integral projected gradient method (PIPG), a high-performance first-order conic optimization solver, in tandem with the penalized trust region (PTR) SCP algorithm. We demonstrate the efficacy and real-time capability of SeCO by solving a relevant multi-phase rocket landing guidance problem with nonlinear dynamics and convex constraints only, and observe that our solver is 2.7 times faster than a state-of-the-art convex optimization solver. △ Less

Submitted 29 May, 2023; v1 submitted 1 December, 2022; originally announced December 2022.

Joint Synthesis of Trajectory and Controlled Invariant Funnel for Discrete-time Systems with Locally Lipschitz Nonlinearities

Authors: Taewan Kim, Purnanand Elango, Behcet Acikmese

Abstract: This paper presents a joint synthesis algorithm of trajectory and controlled invariant funnel (CIF) for locally Lipschitz nonlinear systems subject to bounded disturbances. The CIF synthesis refers to a procedure of computing controlled invariance sets and corresponding feedback gains. In contrast to existing CIF synthesis methods that compute the CIF with a pre-defined nominal trajectory, our wor… ▽ More This paper presents a joint synthesis algorithm of trajectory and controlled invariant funnel (CIF) for locally Lipschitz nonlinear systems subject to bounded disturbances. The CIF synthesis refers to a procedure of computing controlled invariance sets and corresponding feedback gains. In contrast to existing CIF synthesis methods that compute the CIF with a pre-defined nominal trajectory, our work aims to optimize the nominal trajectory and the CIF jointly to satisfy feasibility conditions without the relaxation of constraints and obtain a more cost-optimal nominal trajectory. The proposed work has a recursive scheme that mainly optimize trajectory update and funnel update. The trajectory update step optimizes the nominal trajectory while ensuring the feasibility of the CIF. Then, the funnel update step computes the funnel around the nominal trajectory so that the CIF guarantees an invariance property. As a result, with the optimized trajectory and CIF, any resulting trajectory propagated from an initial set by the control law with the computed feedback gain remains within the feasible region around the nominal trajectory under the presence of bounded disturbances. We validate the proposed method via two applications from robotics. △ Less

Submitted 12 January, 2024; v1 submitted 7 September, 2022; originally announced September 2022.

Comments: This paper has been accepted to International Journal of Robust and Nonlinear Control

Real-Time Quadrotor Trajectory Optimization with Time-Triggered Corridor Constraints

Authors: Yue Yu, Kartik Nagpal, Skye Mceowen, Behçet Açıkmeşe, Ufuk Topcu

Abstract: One of the keys to flying quadrotors is to optimize their trajectories within the set of collision-free corridors. These corridors impose nonconvex constraints on the trajectories, making real-time trajectory optimization challenging. We introduce a novel numerical method that approximates the nonconvex corridor constraints with time-triggered convex corridor constraints. This method combines bise… ▽ More One of the keys to flying quadrotors is to optimize their trajectories within the set of collision-free corridors. These corridors impose nonconvex constraints on the trajectories, making real-time trajectory optimization challenging. We introduce a novel numerical method that approximates the nonconvex corridor constraints with time-triggered convex corridor constraints. This method combines bisection search and repeated infeasibility detection. We further develop a customized C++ implementation of the proposed method, based on a first-order conic optimization method that detects infeasibility and exploits problem structure. We demonstrate the efficiency and effectiveness of the proposed method using numerical simulation on randomly generated problem instances as well as indoor flight experiments with hoop obstacles. Compared with mixed integer programming, the proposed method is about 50--200 times faster. △ Less

Submitted 15 August, 2022; originally announced August 2022.

Set-based value operators for non-stationary Markovian environments

Authors: Sarah H. Q. Li, Assalé Adjé, Pierre-Loïc Garoche, Behçet Açıkmeşe

Abstract: This paper analyzes finite state Markov Decision Processes (MDPs) with uncertain parameters in compact sets and re-examines results from robust MDP via set-based fixed point theory. To this end, we generalize the Bellman and policy evaluation operators to contracting operators on the value function space and denote them as \emph{value operators}. We lift these value operators to act on \emph{sets}… ▽ More This paper analyzes finite state Markov Decision Processes (MDPs) with uncertain parameters in compact sets and re-examines results from robust MDP via set-based fixed point theory. To this end, we generalize the Bellman and policy evaluation operators to contracting operators on the value function space and denote them as \emph{value operators}. We lift these value operators to act on \emph{sets} of value functions and denote them as \emph{set-based value operators}. We prove that the set-based value operators are \emph{contractions} in the space of compact value function sets. Leveraging insights from set theory, we generalize the rectangularity condition in classic robust MDP literature to a containment condition for all value operators, which is weaker and can be applied to a larger set of parameter-uncertain MDPs and contracting operators in dynamic programming. We prove that both the rectangularity condition and the containment condition sufficiently ensure that the set-based value operator's fixed point set contains its own extrema elements. For convex and compact sets of uncertain MDP parameters, we show equivalence between the classic robust value function and the supremum of the fixed point set of the set-based Bellman operator. Under dynamically changing MDP parameters in compact sets, we prove a set convergence result for value iteration, which otherwise may not converge to a single value function. Finally, we derive novel guarantees for probabilistic path-planning problems in planet exploration and stratospheric station-kee**. △ Less

Submitted 8 August, 2023; v1 submitted 14 July, 2022; originally announced July 2022.

Comments: 17 pages, 11 figures, 1 table

Congestion-aware path coordination game with Markov decision process dynamics

Authors: Sarah H. Q. Li, Dan Calderone, Behcet Acikmese

Abstract: Inspired by the path coordination problem arising from robo-taxis, warehouse management, and mixed-vehicle routing problems, we model a group of heterogeneous players responding to stochastic demands as a congestion game under Markov decision process dynamics. Players share a common state-action space but have unique transition dynamics, and each player's unique cost is a {function} of the joint s… ▽ More Inspired by the path coordination problem arising from robo-taxis, warehouse management, and mixed-vehicle routing problems, we model a group of heterogeneous players responding to stochastic demands as a congestion game under Markov decision process dynamics. Players share a common state-action space but have unique transition dynamics, and each player's unique cost is a {function} of the joint state-action probability distribution. For a class of player cost functions, we formulate the player-specific optimization problem, prove the equivalence between the Nash equilibrium and the solution of a potential minimization problem, and derive dynamic programming approaches to solve the Nash equilibrium. We apply this game to model multi-agent path coordination and introduce congestion-based cost functions that enable players to complete individual tasks while avoiding congestion with their opponents. Finally, we present a learning algorithm for finding the Nash equilibrium that has linear complexity in the number of players. We demonstrate our game model on a multi-robot warehouse \change{path coordination problem}, in which robots autonomously retrieve and deliver packages while avoiding congested paths. △ Less

Submitted 5 July, 2022; v1 submitted 22 March, 2022; originally announced March 2022.

Comments: 6 pages, 4 figures

Extrapolated Proportional-Integral Projected Gradient Method for Conic Optimization

Authors: Yue Yu, Purnanand Elango, Behçet Açıkmeşe, Ufuk Topcu

Abstract: Conic optimization is the minimization of a convex quadratic function subject to conic constraints. We introduce a novel first-order method for conic optimization, named \emph{extrapolated proportional-integral projected gradient method (xPIPG)}, that automatically detects infeasibility. The iterates of xPIPG either asymptotically satisfy a set of primal-dual optimality conditions, or generate a p… ▽ More Conic optimization is the minimization of a convex quadratic function subject to conic constraints. We introduce a novel first-order method for conic optimization, named \emph{extrapolated proportional-integral projected gradient method (xPIPG)}, that automatically detects infeasibility. The iterates of xPIPG either asymptotically satisfy a set of primal-dual optimality conditions, or generate a proof of primal or dual infeasibility. We demonstrate the application of xPIPG using benchmark problems in model predictive control. xPIPG outperforms many state-of-the-art conic optimization solvers, especially when solving large-scale problems. △ Less

Submitted 23 June, 2022; v1 submitted 8 March, 2022; originally announced March 2022.

SCvx-fast: A Superlinearly Convergent Algorithm for A Class of Non-Convex Optimal Control Problems

Authors: Yuanqi Mao, Behcet Acikmese

Abstract: In this paper, we extend our previous results and formally propose the SCvx-fast algorithm, a new addition to the Successive Convexification algorithmic framework. The said algorithm solves non-convex optimal control problems with specific types of state constraints (i.e. union of convex keep-out zones) and is faster to converge than SCvx, its predecessor. In order to preserve more feasibility, th… ▽ More In this paper, we extend our previous results and formally propose the SCvx-fast algorithm, a new addition to the Successive Convexification algorithmic framework. The said algorithm solves non-convex optimal control problems with specific types of state constraints (i.e. union of convex keep-out zones) and is faster to converge than SCvx, its predecessor. In order to preserve more feasibility, the proposed algorithm uses a novel project-and convexify procedure to successively convexify both state constraints and system dynamics, and thus a finite dimensional convex programming subproblem is solved at each succession. It also gets rid of the dependency on trust regions, gaining the ability to take larger steps and thus ultimately attaining faster convergence. The extension is in three folds as follows. i) We can now initialize the algorithm from an infeasible starting point, and regain feasibility in just one step; ii) We get rid of the smoothness conditions on the constraints so that a broader range of "obstacles" can be included. Significant changes are made to adjust the algorithm accordingly; iii) We obtain a proof of superlinear rate of convergence, a new theoretical result for SCvx-fast. Benefiting from its specific problem setup and the project-and convexify procedure, the SCvx-fast algorithm is particularly suitable for solving trajectory planning problems with collision avoidance constraints. Numerical simulations are performed, affirming the fast convergence rate. With powerful convex programming solvers, the algorithm can be implemented onboard for real-time autonomous guidance applications. △ Less

Submitted 30 November, 2021; originally announced December 2021.

Comments: 22 pages, 7 figures. arXiv admin note: substantial text overlap with arXiv:1701.00558

Guided Policy Search using Sequential Convex Programming for Initialization of Trajectory Optimization Algorithms

Authors: Taewan Kim, Purnanand Elango, Danylo Malyuta, Behcet Acikmese

Abstract: Nonlinear trajectory optimization algorithms have been developed to handle optimal control problems with nonlinear dynamics and nonconvex constraints in trajectory planning. The performance and computational efficiency of many trajectory optimization methods are sensitive to the initial guess, i.e., the trajectory guess needed by the recursive trajectory optimization algorithm. Motivated by this o… ▽ More Nonlinear trajectory optimization algorithms have been developed to handle optimal control problems with nonlinear dynamics and nonconvex constraints in trajectory planning. The performance and computational efficiency of many trajectory optimization methods are sensitive to the initial guess, i.e., the trajectory guess needed by the recursive trajectory optimization algorithm. Motivated by this observation, we tackle the initialization problem for trajectory optimization via policy optimization. To optimize a policy, we propose a guided policy search method that has two key components: i) Trajectory update; ii) Policy update. The trajectory update involves offline solutions of a large number of trajectory optimization problems from different initial states via Sequential Convex Programming (SCP). Here we take a single SCP step to generate the trajectory iterate for each problem. In conjunction with these iterates, we also generate additional trajectories around each iterate via a feedback control law. Then all these trajectories are used by a stochastic gradient descent algorithm to update the neural network policy, i.e., the policy update step. As a result, the trained policy makes it possible to generate trajectory candidates that are close to the optimality and feasibility and that provide excellent initial guesses for the trajectory optimization methods. We validate the proposed method via a real-world 6-degree-of-freedom powered descent guidance problem for a reusable rocket. △ Less

Submitted 19 May, 2022; v1 submitted 13 October, 2021; originally announced October 2021.

Comments: Presented in American Control Conference (ACC) 2022

Discrete-Time Linear-Quadratic Regulation via Optimal Transport

Authors: Mathias Hudoba de Badyn, Erik Miehling, Dylan Janak, Behçet Açıkmeşe, Mehran Mesbahi, Tamer Başar, John Lygeros, Roy S. Smith

Abstract: In this paper, we consider a discrete-time stochastic control problem with uncertain initial and target states. We first discuss the connection between optimal transport and stochastic control problems of this form. Next, we formulate a linear-quadratic regulator problem where the initial and terminal states are distributed according to specified probability densities. A closed-form solution for t… ▽ More In this paper, we consider a discrete-time stochastic control problem with uncertain initial and target states. We first discuss the connection between optimal transport and stochastic control problems of this form. Next, we formulate a linear-quadratic regulator problem where the initial and terminal states are distributed according to specified probability densities. A closed-form solution for the optimal transport map in the case of linear-time varying systems is derived, along with an algorithm for computing the optimal map. Two numerical examples pertaining to swarm deployment demonstrate the practical applicability of the model, and performance of the numerical method. △ Less

Submitted 6 September, 2021; originally announced September 2021.

Comments: 8 pages, 6 figures. To be included in the Proceedings of the 60th Conference on Decision and Control. This version includes proofs

Proportional-Integral Projected Gradient Method for Conic Optimization

Authors: Yue Yu, Purnanand Elango, Ufuk Topcu, Behçet Açıkmeşe

Abstract: Conic optimization is the minimization of a differentiable convex objective function subject to conic constraints. We propose a novel primal-dual first-order method for conic optimization, named proportional-integral projected gradient method (PIPG). PIPG ensures that both the primal-dual gap and the constraint violation converge to zero at the rate of \(O(1/k)\), where \(k\) is the number of iter… ▽ More Conic optimization is the minimization of a differentiable convex objective function subject to conic constraints. We propose a novel primal-dual first-order method for conic optimization, named proportional-integral projected gradient method (PIPG). PIPG ensures that both the primal-dual gap and the constraint violation converge to zero at the rate of \(O(1/k)\), where \(k\) is the number of iterations. If the objective function is strongly convex, PIPG improves the convergence rate of the primal-dual gap to \(O(1/k^2)\). Further, unlike any existing first-order methods, PIPG also improves the convergence rate of the constraint violation to \(O(1/k^3)\). We demonstrate the application of PIPG in constrained optimal control problems. △ Less

Submitted 13 December, 2021; v1 submitted 23 August, 2021; originally announced August 2021.

Advances in Trajectory Optimization for Space Vehicle Control

Authors: Danylo Malyuta, Yue Yu, Purnanand Elango, Behcet Acikmese

Abstract: Space mission design places a premium on cost and operational efficiency. The search for new science and life beyond Earth calls for spacecraft that can deliver scientific payloads to geologically rich yet hazardous landing sites. At the same time, the last four decades of optimization research have put a suite of powerful optimization tools at the fingertips of the controls engineer. As we enter… ▽ More Space mission design places a premium on cost and operational efficiency. The search for new science and life beyond Earth calls for spacecraft that can deliver scientific payloads to geologically rich yet hazardous landing sites. At the same time, the last four decades of optimization research have put a suite of powerful optimization tools at the fingertips of the controls engineer. As we enter the new decade, optimization theory, algorithms, and software tooling have reached a critical mass to start seeing serious application in space vehicle guidance and control systems. This survey paper provides a detailed overview of recent advances, successes, and promising directions for optimization-based space vehicle control. The considered applications include planetary landing, rendezvous and proximity operations, small body landing, constrained attitude reorientation, endo-atmospheric flight including ascent and reentry, and orbit transfer and injection. The primary focus is on the last ten years of progress, which have seen a veritable rise in the number of applications using three core technologies: lossless convexification, sequential convex programming, and model predictive control. The reader will come away with a well-rounded understanding of the state-of-the-art in each space vehicle control application, and will be well positioned to tackle important current open problems using convex optimization as a core technology. △ Less

Submitted 23 August, 2021; v1 submitted 4 August, 2021; originally announced August 2021.

Comments: 100 pages, 18 figures, 1 table; accepted in Elsevier Annual Reviews in Control

Fast Homotopy for Spacecraft Rendezvous Trajectory Optimization with Discrete Logic

Authors: Danylo Malyuta, Behcet Acikmese

Abstract: This paper presents a computationally efficient optimization algorithm for solving nonconvex optimal control problems that involve discrete logic constraints. Traditional solution methods for these constraints require binary variables and mixed-integer programming, which is prohibitively slow and computationally expensive. This paper targets a fast solution that is capable of real-time implementat… ▽ More This paper presents a computationally efficient optimization algorithm for solving nonconvex optimal control problems that involve discrete logic constraints. Traditional solution methods for these constraints require binary variables and mixed-integer programming, which is prohibitively slow and computationally expensive. This paper targets a fast solution that is capable of real-time implementation onboard spacecraft. To do so, a novel algorithm is developed that blends sequential convex programming and numerical continuation into a single iterative solution process. Inside the algorithm, discrete logic constraints are approximated by smooth functions, and a homotopy parameter governs the accuracy of this approximation. As the algorithm converges, the homotopy parameter is updated such that the smooth approximations enforce the exact discrete logic. The effectiveness of this approach is numerically demonstrated for a realistic rendezvous scenario inspired by the Apollo Transposition and Docking maneuver. In under 15 seconds of cumulative solver time, the algorithm is able to reliably find difficult fuel-optimal trajectories that obey the following discrete logic constraints: thruster minimum impulse-bit, range-triggered approach cone, and range-triggered plume im**ement. The optimized trajectory uses significantly less fuel than reported NASA design targets. △ Less

Submitted 14 July, 2021; originally announced July 2021.

Comments: 40 pages, 19 figures; submitted to the AIAA Journal of Guidance, Control, and Dynamics

Convex Optimization for Trajectory Generation

Authors: Danylo Malyuta, Taylor P. Reynolds, Michael Szmuk, Thomas Lew, Riccardo Bonalli, Marco Pavone, Behcet Acikmese

Abstract: Reliable and efficient trajectory generation methods are a fundamental need for autonomous dynamical systems of tomorrow. The goal of this article is to provide a comprehensive tutorial of three major convex optimization-based trajectory generation methods: lossless convexification (LCvx), and two sequential convex programming algorithms known as SCvx and GuSTO. In this article, trajectory generat… ▽ More Reliable and efficient trajectory generation methods are a fundamental need for autonomous dynamical systems of tomorrow. The goal of this article is to provide a comprehensive tutorial of three major convex optimization-based trajectory generation methods: lossless convexification (LCvx), and two sequential convex programming algorithms known as SCvx and GuSTO. In this article, trajectory generation is the computation of a dynamically feasible state and control signal that satisfies a set of constraints while optimizing key mission objectives. The trajectory generation problem is almost always nonconvex, which typically means that it is not readily amenable to efficient and reliable solution onboard an autonomous vehicle. The three algorithms that we discuss use problem reformulation and a systematic algorithmic strategy to nonetheless solve nonconvex trajectory generation tasks through the use of a convex optimizer. The theoretical guarantees and computational speed offered by convex optimization have made the algorithms popular in both research and industry circles. To date, the list of applications includes rocket landing, spacecraft hypersonic reentry, spacecraft rendezvous and docking, aerial motion planning for fixed-wing and quadrotor vehicles, robot motion planning, and more. Among these applications are high-profile rocket flights conducted by organizations like NASA, Masten Space Systems, SpaceX, and Blue Origin. This article aims to give the reader the tools and understanding necessary to work with each algorithm, and to know what each method can and cannot do. A publicly available source code repository supports the provided numerical examples. By the end of the article, the reader should be ready to use the methods, to extend them, and to contribute to their many exciting modern applications. △ Less

Submitted 16 June, 2021; originally announced June 2021.

Comments: 68 pages, 42 figures, 5 tables. This work has been submitted to the IEEE for possible publication

On Controllability and Persistency of Excitation in Data-Driven Control: Extensions of Willems' Fundamental Lemma

Authors: Yue Yu, Shahriar Talebi, Henk J. van Waarde, Ufuk Topcu, Mehran Mesbahi, Behçet Açıkmeşe

Abstract: Willems' fundamental lemma asserts that all trajectories of a linear time-invariant system can be obtained from a finite number of measured ones, assuming that controllability and a persistency of excitation condition hold. We show that these two conditions can be relaxed. First, we prove that the controllability condition can be replaced by a condition on the controllable subspace, unobservable s… ▽ More Willems' fundamental lemma asserts that all trajectories of a linear time-invariant system can be obtained from a finite number of measured ones, assuming that controllability and a persistency of excitation condition hold. We show that these two conditions can be relaxed. First, we prove that the controllability condition can be replaced by a condition on the controllable subspace, unobservable subspace, and a certain subspace associated with the measured trajectories. Second, we prove that the persistency of excitation requirement can be relaxed if the degree of a certain minimal polynomial is tightly bounded. Our results show that data-driven predictive control using online data is equivalent to model predictive control, even for uncontrollable systems. Moreover, our results significantly reduce the amount of data needed in identifying homogeneous multi-agent systems. △ Less

Submitted 9 April, 2021; v1 submitted 4 February, 2021; originally announced February 2021.

Decentralized State-Dependent Markov Chain Synthesis with an Application to Swarm Guidance

Authors: Samet Uzun, Nazim Kemal Ure, Behcet Acikmese

Abstract: This paper introduces a decentralized state-dependent Markov chain synthesis (DSMC) algorithm for finite-state Markov chains. We present a state-dependent consensus protocol that achieves exponential convergence under mild technical conditions, without relying on any connectivity assumptions regarding the dynamic network topology. Utilizing the proposed consensus protocol, we develop the DSMC algo… ▽ More This paper introduces a decentralized state-dependent Markov chain synthesis (DSMC) algorithm for finite-state Markov chains. We present a state-dependent consensus protocol that achieves exponential convergence under mild technical conditions, without relying on any connectivity assumptions regarding the dynamic network topology. Utilizing the proposed consensus protocol, we develop the DSMC algorithm, updating the Markov matrix based on the current state while ensuring the convergence conditions of the consensus protocol. This result establishes the desired steady-state distribution for the resulting Markov chain, ensuring exponential convergence from all initial distributions while adhering to transition constraints and minimizing state transitions. The DSMC's performance is demonstrated through a probabilistic swarm guidance example, which interprets the spatial distribution of a swarm comprising a large number of mobile agents as a probability distribution and utilizes the Markov chain to compute transition probabilities between states. Simulation results demonstrate faster convergence for the DSMC based algorithm when compared to the previous Markov chain based swarm guidance algorithms. △ Less

Submitted 26 April, 2024; v1 submitted 4 December, 2020; originally announced December 2020.

Comments: arXiv admin note: text overlap with arXiv:2012.01928

Stability of Gradient Learning Dynamics in Continuous Games: Vector Action Spaces

Authors: Benjamin J. Chasnov, Daniel Calderone, Behçet Açıkmeşe, Samuel A. Burden, Lillian J. Ratliff

Abstract: Towards characterizing the optimization landscape of games, this paper analyzes the stability of gradient-based dynamics near fixed points of two-player continuous games. We introduce the quadratic numerical range as a method to characterize the spectrum of game dynamics and prove the robustness of equilibria to variations in learning rates. By decomposing the game Jacobian into symmetric and skew… ▽ More Towards characterizing the optimization landscape of games, this paper analyzes the stability of gradient-based dynamics near fixed points of two-player continuous games. We introduce the quadratic numerical range as a method to characterize the spectrum of game dynamics and prove the robustness of equilibria to variations in learning rates. By decomposing the game Jacobian into symmetric and skew-symmetric components, we assess the contribution of a vector field's potential and rotational components to the stability of differential Nash equilibria. Our results show that in zero-sum games, all Nash are stable and robust; in potential games, all stable points are Nash. For general-sum games, we provide a sufficient condition for instability. We conclude with a numerical example in which learning with timescale separation results in faster convergence. △ Less

Submitted 13 January, 2021; v1 submitted 6 November, 2020; originally announced November 2020.

Comments: extension of arXiv:2011.03650 to vector action spaces. Submitted to IEEE L-CSS

Synthesis of Interval Observers for Nonlinear Discrete-Time Systems

Authors: Adam M Tahir, Behcet Acikmese

Abstract: A systematic procedure to synthesize interval observers for nonlinear discrete-time systems is proposed. The feedback gains and other matrices are found from the solutions to semidefinite feasibility programs. Two cases are considered: (1) the interval observer is in the same coordinate frame as the given system, and (2) the interval observer uses a coordinate transformation. The conditions where… ▽ More A systematic procedure to synthesize interval observers for nonlinear discrete-time systems is proposed. The feedback gains and other matrices are found from the solutions to semidefinite feasibility programs. Two cases are considered: (1) the interval observer is in the same coordinate frame as the given system, and (2) the interval observer uses a coordinate transformation. The conditions where coordinate transformations are necessary are detailed. Numerical examples are provided to showcase the effectiveness of the interval observers and demonstrate their application to sampled-data systems. △ Less

Submitted 8 November, 2020; originally announced November 2020.

Stability of Gradient Learning Dynamics in Continuous Games: Scalar Action Spaces

Authors: Benjamin J. Chasnov, Daniel Calderone, Behçet Açıkmeşe, Samuel A. Burden, Lillian J. Ratliff

Abstract: Learning processes in games explain how players grapple with one another in seeking an equilibrium. We study a natural model of learning based on individual gradients in two-player continuous games. In such games, the arguably natural notion of a local equilibrium is a differential Nash equilibrium. However, the set of locally exponentially stable equilibria of the learning dynamics do not necessa… ▽ More Learning processes in games explain how players grapple with one another in seeking an equilibrium. We study a natural model of learning based on individual gradients in two-player continuous games. In such games, the arguably natural notion of a local equilibrium is a differential Nash equilibrium. However, the set of locally exponentially stable equilibria of the learning dynamics do not necessarily coincide with the set of differential Nash equilibria of the corresponding game. To characterize this gap, we provide formal guarantees for the stability or instability of such fixed points by leveraging the spectrum of the linearized game dynamics. We provide a comprehensive understanding of scalar games and find that equilibria that are both stable and Nash are robust to variations in learning rates. △ Less

Submitted 6 November, 2020; originally announced November 2020.

Comments: Accepted to 2020 IEEE Conference on Decision and Control

Proportional-Integral Projected Gradient Method for Model Predictive Control

Authors: Yue Yu, Purnanand Elango, Behçet Açikmeşe

Abstract: Recently there has been an increasing interest in primal-dual methods for model predictive control (MPC), which require minimizing the (augmented) Lagrangian at each iteration. We propose a novel first order primal-dual method, termed \emph{proportional-integral projected gradient method}, for MPC where the underlying finite horizon optimal control problem has both state and input constraints. Ins… ▽ More Recently there has been an increasing interest in primal-dual methods for model predictive control (MPC), which require minimizing the (augmented) Lagrangian at each iteration. We propose a novel first order primal-dual method, termed \emph{proportional-integral projected gradient method}, for MPC where the underlying finite horizon optimal control problem has both state and input constraints. Instead of minimizing the (augmented) Lagrangian, each iteration of our method only computes a single projection onto the state and input constraint set. Our method ensures that, along a sequence of averaged iterates, both the distance to optimum and the constraint violation converge to zero at a rate of \(O(1/k)\) if the objective function is convex, where \(k\) is the iteration number. If the objective function is strongly convex, this rate can be improved to \(O(1/k^2)\) for the distance to optimum and \(O(1/k^3)\) for the constraint violation. We compare our method against existing methods via a trajectory-planning example with convexified keep-out-zone constraints. △ Less

Submitted 18 December, 2020; v1 submitted 15 September, 2020; originally announced September 2020.

Comments: Julia code available at: https://github.com/purnanandelango/pi-projgrad-demo

Disturbance Decoupling for Gradient-based Multi-Agent Learning with Quadratic Costs

Authors: Sarah H. Q. Li, Lillian Ratliff, Behçet Açıkmeşe

Abstract: Motivated by applications of multi-agent learning in noisy environments, this paper studies the robustness of gradient-based learning dynamics with respect to disturbances. While disturbances injected along a coordinate corresponding to any individual player's actions can always affect the overall learning dynamics, a subset of players can be disturbance decoupled---i.e., such players' actions are… ▽ More Motivated by applications of multi-agent learning in noisy environments, this paper studies the robustness of gradient-based learning dynamics with respect to disturbances. While disturbances injected along a coordinate corresponding to any individual player's actions can always affect the overall learning dynamics, a subset of players can be disturbance decoupled---i.e., such players' actions are completely unaffected by the injected disturbance. We provide necessary and sufficient conditions to guarantee this property for games with quadratic cost functions, which encompass quadratic one-shot continuous games, finite-horizon linear quadratic (LQ) dynamic games, and bilinear games. Specifically, disturbance decoupling is characterized by both algebraic and graph-theoretic conditions on the learning dynamics, the latter is obtained by constructing a game graph based on gradients of players' costs. For LQ games, we show that disturbance decoupling imposes constraints on the controllable and unobservable subspaces of players. For two player bilinear games, we show that disturbance decoupling within a player's action coordinates imposes constraints on the payoff matrices. Illustrative numerical examples are provided. △ Less

Submitted 10 October, 2020; v1 submitted 14 July, 2020; originally announced July 2020.

Journal ref: IEEE Control Systems Letters, vol. 5, no. 1, pp. 223-228, Jan. 2021

RC Circuits based Distributed Conditional Gradient Method

Authors: Yue Yu, Behçet Açıkmeşe

Abstract: We consider distributed optimization on undirected connected graphs. We propose a novel distributed conditional gradient method with (O(1/\sqrt{k})) convergence. Compared with existing methods, each iteration of our method uses both communication and linear minimization step only once rather than multiple times. We further extend our results to cases with composite local constraints. We demonstrat… ▽ More We consider distributed optimization on undirected connected graphs. We propose a novel distributed conditional gradient method with (O(1/\sqrt{k})) convergence. Compared with existing methods, each iteration of our method uses both communication and linear minimization step only once rather than multiple times. We further extend our results to cases with composite local constraints. We demonstrate our results via examples on distributed matrix completion problem. △ Less

Submitted 15 March, 2020; originally announced March 2020.

Bounding Fixed Points of Set-Based Bellman Operator and Nash Equilibria of Stochastic Games

Authors: Sarah H. Q. Li, Assalé, Adjé, Pierre-Loïc Garoche, Behçet Açıkmeşe

Abstract: Motivated by uncertain parameters encountered in Markov decision processes (MDPs) and stochastic games, we study the effect of parameter uncertainty on Bellman operator-based algorithms under a set-based framework. Specifically, we first consider a family of MDPs where the cost parameters are in a given compact set; we then define a Bellman operator acting on a set of value functions to produce a… ▽ More Motivated by uncertain parameters encountered in Markov decision processes (MDPs) and stochastic games, we study the effect of parameter uncertainty on Bellman operator-based algorithms under a set-based framework. Specifically, we first consider a family of MDPs where the cost parameters are in a given compact set; we then define a Bellman operator acting on a set of value functions to produce a new set of value functions as the output under all possible variations in the cost parameter. We prove the existence of a fixed point of this set-based Bellman operator by showing that it is contractive on a complete metric space, and explore its relationship with the corresponding family of MDPs and stochastic games. Additionally, we show that given interval set bounded cost parameters, we can form exact bounds on the set of optimal value functions. Finally, we utilize our results to bound the value function trajectory of a player in a stochastic game. △ Less

Submitted 10 October, 2020; v1 submitted 22 January, 2020; originally announced January 2020.

Comments: 15 pages, 4 figures

Fixed Points of the Set-Based Bellman Operator

Authors: Sarah H. Q. Li, Assalé Adjé, Pierre-Loïc Garoche, Behçet Açıkmeşe

Abstract: Motivated by uncertain parameters encountered in Markov decision processes (MDPs), we study the effect of parameter uncertainty on Bellman operator-based methods. Specifically, we consider a family of MDPs where the cost parameters are from a given compact set. We then define a Bellman operator acting on an input set of value functions to produce a new set of value functions as the output under al… ▽ More Motivated by uncertain parameters encountered in Markov decision processes (MDPs), we study the effect of parameter uncertainty on Bellman operator-based methods. Specifically, we consider a family of MDPs where the cost parameters are from a given compact set. We then define a Bellman operator acting on an input set of value functions to produce a new set of value functions as the output under all possible variations in the cost parameters. Finally we prove the existence of a fixed point of this set-based Bellman operator by showing that it is a contractive operator on a complete metric space. △ Less

Submitted 29 February, 2020; v1 submitted 13 January, 2020; originally announced January 2020.

Comments: 4 pages

Lossless Convexification of Optimal Control Problems with Semi-continuous Inputs

Authors: Danylo Malyuta, Behcet Acikmese

Abstract: This paper presents a novel convex optimization-based method for finding the globally optimal solutions of a class of mixed-integer non-convex optimal control problems. We consider problems with non-convex constraints that restrict the input norms to be either zero or lower- and upper-bounded. The non-convex problem is relaxed to a convex one whose optimal solution is proved to be optimal almost e… ▽ More This paper presents a novel convex optimization-based method for finding the globally optimal solutions of a class of mixed-integer non-convex optimal control problems. We consider problems with non-convex constraints that restrict the input norms to be either zero or lower- and upper-bounded. The non-convex problem is relaxed to a convex one whose optimal solution is proved to be optimal almost everywhere for the original problem, a procedure known as lossless convexification. This paper is the first to allow individual input sets to overlap and to have different norm bounds, integral input and state costs, and convex state constraints that can be activated at discrete time instances. The solution relies on second-order cone programming and demonstrates that a meaningful class of optimal control problems with binary variables can be solved reliably and in polynomial time. A rocket landing example with a coupled thrust-gimbal constraint corroborates the effectiveness of the approach. △ Less

Submitted 18 November, 2019; originally announced November 2019.

Comments: 8 pages, 4 figures; submitted to 21st IFAC World Congress, 2020. arXiv admin note: substantial text overlap with arXiv:1902.02726

RLC Circuits based Distributed Mirror Descent Method

Authors: Yue Yu, Behçet Açıkmeşe

Abstract: We consider distributed optimization with smooth convex objective functions defined on an undirected connected graph. Inspired by mirror descent mehod and RLC circuits, we propose a novel distributed mirror descent method. Compared with mirror-prox method, our algorithm achieves the same \(\mathcal{O}(1/k)\) iteration complexity with only half the computation cost per iteration. We further extend… ▽ More We consider distributed optimization with smooth convex objective functions defined on an undirected connected graph. Inspired by mirror descent mehod and RLC circuits, we propose a novel distributed mirror descent method. Compared with mirror-prox method, our algorithm achieves the same \(\mathcal{O}(1/k)\) iteration complexity with only half the computation cost per iteration. We further extend our results to cases where a) gradients are corrupted by stochastic noise, and b) objective function is composed of both smooth and non-smooth terms. We demonstrate our theoretical results via numerical experiments. △ Less

Submitted 22 February, 2020; v1 submitted 14 November, 2019; originally announced November 2019.

Sensitivity Analysis for Markov Decision Process Congestion Games

Authors: Sarah H. Q. Li, Daniel Calderone, Lillian Ratliff, Behcet Acikmese

Abstract: We consider a non-atomic congestion game where each decision maker performs selfish optimization over states of a common MDP. The decision makers optimize for their own expected costs, and influence each other through congestion effects on the state-action costs. We analyze on the sensitivity of MDP congestion game equilibria to uncertainty and perturbations in the state-action costs by applying a… ▽ More We consider a non-atomic congestion game where each decision maker performs selfish optimization over states of a common MDP. The decision makers optimize for their own expected costs, and influence each other through congestion effects on the state-action costs. We analyze on the sensitivity of MDP congestion game equilibria to uncertainty and perturbations in the state-action costs by applying an implicit function type analysis. The occurrence of a stochastic Braess paradox is defined, analyzed based on sensitivity of game equilibria and demonstrated in simulation. We further analyze how the introduction of stochastic dynamics affects the magnitude of Braess paradox in comparison to deterministic dynamics. △ Less

Submitted 12 September, 2019; v1 submitted 9 September, 2019; originally announced September 2019.

Adaptive Constraint Satisfaction for Markov Decision Process Congestion Games: Application to Transportation Networks

Authors: Sarah H. Q. Li, Yue Yu, Nicolas Miguel, Dan Calderone, Lillian J. Ratliff, Behcet Acikmese

Abstract: Under the Markov decision process (MDP) congestion game framework, we study the problem of enforcing population distribution constraints on a population of players with stochastic dynamics and coupled congestion costs. Existing research demonstrates that the constraints on the players' population distribution can be satisfied by enforcing tolls. However, computing the minimum toll value for constr… ▽ More Under the Markov decision process (MDP) congestion game framework, we study the problem of enforcing population distribution constraints on a population of players with stochastic dynamics and coupled congestion costs. Existing research demonstrates that the constraints on the players' population distribution can be satisfied by enforcing tolls. However, computing the minimum toll value for constraint satisfaction requires accurate modeling of the player's congestion costs. Motivated by settings where an accurate congestion cost model is unavailable (e.g. transportation networks), we consider an MDP congestion game with unknown congestion costs. We assume that a constraint-enforcing authority can repeatedly enforce tolls on a population of players who converges to an $ε$-optimal population distribution for any given toll. We then construct a myopic update algorithm to compute the minimum toll value while ensuring that the constraints are satisfied on average. We analyze how the players' sub-optimal responses to tolls impact the rates of convergence towards the minimum toll value and constraint satisfaction. Finally, we construct a congestion game model for Uber drivers in Manhattan, New York City (NYC) using data from the Taxi and Limousine Commission (TLC) to illustrate how to efficiently reduce congestion while minimizing the impact on driver earnings. △ Less

Submitted 14 August, 2022; v1 submitted 21 July, 2019; originally announced July 2019.

Comments: 10 pages, 5 figures

Fast Trajectory Optimization via Successive Convexification for Spacecraft Rendezvous with Integer Constraints

Authors: Danylo Malyuta, Taylor P. Reynolds, Michael Szmuk, Behcet Acikmese, Mehran Mesbahi

Abstract: In this paper we present a fast method based on successive convexification for generating fuel-optimized spacecraft rendezvous trajectories in the presence of mixed-integer constraints. A recently developed paradigm of state-triggered constraints allows to efficiently embed a subset of discrete decision constraints into the continuous optimization framework of successive convexification. As a resu… ▽ More In this paper we present a fast method based on successive convexification for generating fuel-optimized spacecraft rendezvous trajectories in the presence of mixed-integer constraints. A recently developed paradigm of state-triggered constraints allows to efficiently embed a subset of discrete decision constraints into the continuous optimization framework of successive convexification. As a result, we are able to solve difficult trajectory optimization problems at interactive speeds, as opposed to a mixed-integer programming approach that would require significantly more solution time and computing power. Our method is applied to the real problem of transposition and docking of the Apollo command and service module with the lunar module. We demonstrate that, within seconds, we are able to obtain trajectories that are up to 90 percent more fuel efficient (saving up to 45 kg of fuel) than non-optimization based Apollo-era design targets. Our trajectories take explicit account of minimum thrust pulse width and plume im**ement constraints. Both of these constraints are naturally mixed-integer, but we handle them as state-triggered constraints. In its current state, our algorithm will serve as a useful off-line design tool for rapid trajectory trade studies. △ Less

Submitted 11 June, 2019; originally announced June 2019.

Comments: 23 pages, 10 figures, submitted to AIAA SciTech 2020

Maximal Invariant Set Computation and Design for Markov Chains

Authors: Dylan Janak, Behçet Açıkmeşe

Abstract: We describe an algorithm for computing the maximal invariant set for a Markov chain with linear safety constraints on the distribution over states. We then propose a Markov chain synthesis method that guarantees finite determination of the maximal invariant set. Although this problem is bilinear in the general case, we are able to optimize the convergence rate to a desirable steady-state distribut… ▽ More We describe an algorithm for computing the maximal invariant set for a Markov chain with linear safety constraints on the distribution over states. We then propose a Markov chain synthesis method that guarantees finite determination of the maximal invariant set. Although this problem is bilinear in the general case, we are able to optimize the convergence rate to a desirable steady-state distribution over reversible Markov chains by solving a Semidefinite Program (SDP), which promotes efficient computation of the maximal invariant set. We then demonstrate this approach with a decentralized swarm guidance application subject to density upper bounds. △ Less

Submitted 2 May, 2019; originally announced May 2019.

Comments: Accepted to American Control Conference (ACC) 2019

Dual Quaternion Based Powered Descent Guidance with State-Triggered Constraints

Authors: Taylor P. Reynolds, Michael Szmuk, Danylo Malyuta, Mehran Mesbahi, Behcet Acikmese, John M. Carson III

Abstract: This paper presents a numerical algorithm for computing 6-degree-of-freedom free-final-time powered descent guidance trajectories. The trajectory generation problem is formulated using a unit dual quaternion representation of the rigid body dynamics, and several standard path constraints. Our formulation also includes a special line of sight constraints that is enforced only within a specified ban… ▽ More This paper presents a numerical algorithm for computing 6-degree-of-freedom free-final-time powered descent guidance trajectories. The trajectory generation problem is formulated using a unit dual quaternion representation of the rigid body dynamics, and several standard path constraints. Our formulation also includes a special line of sight constraints that is enforced only within a specified band of slant ranges relative to the landing site, a novel feature that is especially relevant to Terrain and Hazard Relative Navigation. We use the newly introduced state-triggered constraints to formulate these range constraints in a manner that is amenable to real-time implementations. The resulting non-convex optimal control problem is solved iteratively as a sequence of convex second-order cone programs that locally approximate the non-convex problem. Each second-order cone program is solved using a customizable interior point method solver. Also introduced are a scaling method and a new heuristic technique that guide the convergence process towards dynamic feasibility. To demonstrate the capabilities of our algorithm, two numerical case studies are presented. The first studies the effect of including a slant-range-triggered line of sight constraint on the resulting trajectories. The second study performs a Monte Carlo analysis to assess the algorithm's robustness to initial conditions and real-time performance. △ Less

Submitted 19 April, 2019; originally announced April 2019.

Comments: Submitted to the AIAA Journal of Guidance, Control, and Dynamics

Tolling for Constraint Satisfaction in Markov Decision Process Congestion Games

Authors: Sarah H. Q. Li, Yue Yu, Daniel Calderone, Lillian Ratliff, Behcet Acikmese

Abstract: Markov decision process (MDP) congestion game is an extension of classic congestion games, where a continuous population of selfish agents solves Markov decision processes with congestion: the payoff of a strategy decreases as more population uses it. We draw parallels between key concepts from capacitated congestion games and MDP. In particular, we show that population mass constraints in MDP con… ▽ More Markov decision process (MDP) congestion game is an extension of classic congestion games, where a continuous population of selfish agents solves Markov decision processes with congestion: the payoff of a strategy decreases as more population uses it. We draw parallels between key concepts from capacitated congestion games and MDP. In particular, we show that population mass constraints in MDP congestion games are equivalent to imposing tolls/incentives on the reward function, which can be utilized by social planners to achieve auxiliary objectives. We demonstrate such methods in a simulated Seattle ride-share model, where tolls and incentives are enforced for two separate objectives: to guarantee minimum driver density in downtown Seattle, and to shift the game equilibrium towards a maximum social output. △ Less

Submitted 2 March, 2019; originally announced March 2019.

Comments: 7 pages, 6 figures, accepted to American Control Conference 2019

Approximate Multiparametric Mixed-integer Convex Programming

Authors: Danylo Malyuta, Behcet Acikmese

Abstract: We propose an algorithm for generating explicit solutions of multiparametric mixed-integer convex programs to within a given suboptimality tolerance. The algorithm is applicable to a very general class of optimization problems, but is most useful for hybrid model predictive control, where on-line implementation is hampered by the worst-case exponential complexity of mixed-integer solvers. The outp… ▽ More We propose an algorithm for generating explicit solutions of multiparametric mixed-integer convex programs to within a given suboptimality tolerance. The algorithm is applicable to a very general class of optimization problems, but is most useful for hybrid model predictive control, where on-line implementation is hampered by the worst-case exponential complexity of mixed-integer solvers. The output is a simplicial partition which defines a static map from the current state to a suboptimal solution. The primary theoretical contribution of this paper is to introduce a non-zero optimal cost overlap metric which is necessary and sufficient for convergence. The overlap size is also linked to partition complexity. The algorithm is massively parallelizable and our implementation, which is publicly available, is run on a cluster of several hundred processors. Not only does our solution have a deterministic runtime, simulations show that our approach is faster than on-line optimization by up to three orders of magnitude. △ Less

Submitted 10 June, 2019; v1 submitted 28 February, 2019; originally announced February 2019.

Comments: 6 pages, 6 figures, accepted to IEEE Control Systems Letters (L-CSS); v4: final submission

Partition-based Feasible Integer Solution Pre-computation for Hybrid Model Predictive Control

Authors: Danylo Malyuta, Behcet Acikmese, Martin Cacan, David S. Bayard

Abstract: For multiparametric mixed-integer convex programming problems such as those encountered in hybrid model predictive control, we propose an algorithm for generating a feasible partition of a subset of the parameter space. The result is a static map from the current parameter to a suboptimal integer solution such that the remaining convex program is feasible. Convergence is proven with a new insight… ▽ More For multiparametric mixed-integer convex programming problems such as those encountered in hybrid model predictive control, we propose an algorithm for generating a feasible partition of a subset of the parameter space. The result is a static map from the current parameter to a suboptimal integer solution such that the remaining convex program is feasible. Convergence is proven with a new insight that the overlap among the feasible parameter sets of each integer solution governs the partition complexity. The partition is stored as a tree which makes querying the feasible solution efficient. The algorithm can be used to warm start a mixed integer solver with a real-time guarantee or to provide a reference integer solution in several suboptimal MPC schemes. The algorithm is tested on randomly generated systems with up to six states, demonstrating the effectiveness of the approach. △ Less

Submitted 28 February, 2019; originally announced February 2019.

Comments: 7 pages, 2 figures, accepted for European Control Conference 2019

Robust Model Predictive Control for Linear Systems with State and Input Dependent Uncertainties

Authors: Danylo Malyuta, Behcet Acikmese, Martin Cacan

Abstract: This paper presents a computationally efficient robust model predictive control law for discrete linear time invariant systems subject to additive disturbances that may depend on the state and/or input norms. Despite the dependency being non-convex, we are able to capture it exactly for input dependency and approximately for state dependency in at most a second order cone programming problem. The… ▽ More This paper presents a computationally efficient robust model predictive control law for discrete linear time invariant systems subject to additive disturbances that may depend on the state and/or input norms. Despite the dependency being non-convex, we are able to capture it exactly for input dependency and approximately for state dependency in at most a second order cone programming problem. The formulation has linear complexity in the planning horizon length. The approach is thus amenable to efficient real-time implementation with a guarantee on recursive feasibility and global optimality. Robust position control of a satellite is considered as an illustrative example. △ Less

Submitted 9 August, 2019; v1 submitted 28 February, 2019; originally announced February 2019.

Comments: 7 pages, 5 figures, accepted for IEEE American Control Conference 2019; added section on semi-feedback MPC

Stochastic Bregman Parallel Direction Method of Multipliers for Distributed Optimization

Authors: Yue Yu, Behçet Açıkmeşe

Abstract: Bregman parallel direction method of multipliers (BPDMM) efficiently solves distributed optimization over a network, which arises in a wide spectrum of collaborative multi-agent learning applications. In this paper, we generalize BPDMM to stochastic BPDMM, where each iteration only solves local optimization on a randomly selected subset of nodes rather than all the nodes in the network. Such gener… ▽ More Bregman parallel direction method of multipliers (BPDMM) efficiently solves distributed optimization over a network, which arises in a wide spectrum of collaborative multi-agent learning applications. In this paper, we generalize BPDMM to stochastic BPDMM, where each iteration only solves local optimization on a randomly selected subset of nodes rather than all the nodes in the network. Such generalization reduce the need for computational resources and allows applications to larger scale networks. We establish both the global convergence and the \(O(1/T)\) iteration complexity of stochastic BPDMM. We demonstrate our results via numerical examples. △ Less

Submitted 1 March, 2019; v1 submitted 25 February, 2019; originally announced February 2019.

Real-Time Quad-Rotor Path Planning Using Convex Optimization and Compound State-Triggered Constraints

Authors: Michael Szmuk, Danylo Malyuta, Taylor P. Reynolds, Margaret Skye Mceowen, Behcet Acikmese

Abstract: The contribution of this paper is the application of compound state-triggered constraints (STCs) to real-time quad-rotor path planning. Originally developed for rocket landing applications, STCs are made up of a trigger condition and a constraint condition that are arranged such that satisfaction of the former implies satisfaction of the latter. Compound STCs go a step further by allowing multiple… ▽ More The contribution of this paper is the application of compound state-triggered constraints (STCs) to real-time quad-rotor path planning. Originally developed for rocket landing applications, STCs are made up of a trigger condition and a constraint condition that are arranged such that satisfaction of the former implies satisfaction of the latter. Compound STCs go a step further by allowing multiple trigger and constraint conditions to be combined via Boolean "and" or "or" operations. The logical implications embodied by STCs can be formulated using continuous variables, and thus enable the incorporation of discrete decision making into a continuous optimization framework. In this paper, compound STCs are used to solve quad-rotor path planning problems that would typically require the use of computationally expensive mixed-integer programming techniques. Two scenarios are considered: (1) a quad-rotor flying through a hoop, and (2) a pair of quad-rotors carrying a beam-like payload through an obstacle course. Successive convexification is used to solve the resulting non-convex optimization problem. Monte-Carlo simulation results show that our approach can reliably generate trajectories at rates upwards of 3 and 1.5 Hz for the first and second scenarios, respectively. △ Less

Submitted 25 February, 2019; originally announced February 2019.

Comments: 8 pages, 4 figures, IEEE RA-L with IROS 2019 option