-
Six-Degree-of-Freedom Aircraft Landing Trajectory Planning with Runway Alignment
Authors:
Taewan Kim,
Abhinav G. Kamath,
Niyousha Rahimi,
Jasper Corleis,
Behçet Açıkmeşe,
Mehran Mesbahi
Abstract:
This paper presents a numerical optimization algorithm for generating approach and landing trajectories for a six-degree-of-freedom (6-DoF) aircraft. We improve on the existing research on aircraft landing trajectory generation by formulating the trajectory optimization problem with additional real-world operational constraints, including 6-DoF aircraft dynamics, runway alignment, constant wind fi…
▽ More
This paper presents a numerical optimization algorithm for generating approach and landing trajectories for a six-degree-of-freedom (6-DoF) aircraft. We improve on the existing research on aircraft landing trajectory generation by formulating the trajectory optimization problem with additional real-world operational constraints, including 6-DoF aircraft dynamics, runway alignment, constant wind field, and obstacle avoidance, to obtain a continuous-time nonconvex optimal control problem. Particularly, the runway alignment constraint enforces the trajectory of the aircraft to be aligned with the runway only during the final approach phase. This is a novel feature that is essential for preventing an approach that is either too steep or too shallow. The proposed method models the runway alignment constraint through a multi-phase trajectory planning scheme, imposing alignment conditions exclusively during the final approach phase. We compare this formulation with the existing state-triggered constraint formulation for runway alignment. To solve the formulated problem, we design a novel sequential convex programming algorithm called xPTR that extends the penalized trust-region (PTR) algorithm by incorporating an extrapolation step to expedite convergence. We validate the proposed method through extensive numerical simulations, including a Monte Carlo study, to evaluate the robustness of the algorithm to varying initial conditions.
△ Less
Submitted 26 May, 2024;
originally announced May 2024.
-
Optimization with Temporal and Logical Specifications via Generalized Mean-based Smooth Robustness Measures
Authors:
Samet Uzun,
Purnanand Elango,
Pierre-Loic Garoche,
Behcet Acikmese
Abstract:
This paper introduces a generalized mean-based C^1-smooth robustness measure over discrete-time signals (D-GMSR) for signal temporal logic (STL) specifications. In conjunction with its C1-smoothness, D-GMSR is proven to be both sound and complete. Furthermore, it demonstrates favorable gradient properties and addresses locality and masking problems, which are critical for numerical optimization. T…
▽ More
This paper introduces a generalized mean-based C^1-smooth robustness measure over discrete-time signals (D-GMSR) for signal temporal logic (STL) specifications. In conjunction with its C1-smoothness, D-GMSR is proven to be both sound and complete. Furthermore, it demonstrates favorable gradient properties and addresses locality and masking problems, which are critical for numerical optimization. The C^1-smoothness of the proposed formulations enables the implementation of robust and efficient numerical optimization algorithms to solve problems with STL specifications while preserving their theoretical guarantees. The practical utility of the proposed robustness measure is demonstrated on two real-world trajectory optimization problems: i) quadrotor flight, and ii) autonomous rocket landing. A sequential convex programming (SCP) framework, incorporating a convergence-guaranteed optimization algorithm (the prox-linear method) is used to solve inherently non-convex trajectory optimization problems with STL specifications. The implementation is available at https://github.com/UW-ACL/D-GMSR
△ Less
Submitted 16 May, 2024;
originally announced May 2024.
-
Successive Convexification for Nonlinear Model Predictive Control with Continuous-Time Constraint Satisfaction
Authors:
Samet Uzun,
Purnanand Elango,
Abhinav G. Kamath,
Taewan Kim,
Behcet Acikmese
Abstract:
We propose a nonlinear model predictive control (NMPC) framework based on a direct optimal control method that ensures continuous-time constraint satisfaction and accurate evaluation of the running cost, without compromising computational efficiency. We leverage the recently proposed successive convexification framework for trajectory optimization, where: (1) the path constraints and running cost…
▽ More
We propose a nonlinear model predictive control (NMPC) framework based on a direct optimal control method that ensures continuous-time constraint satisfaction and accurate evaluation of the running cost, without compromising computational efficiency. We leverage the recently proposed successive convexification framework for trajectory optimization, where: (1) the path constraints and running cost are equivalently reformulated by augmenting the system dynamics, (2) multiple shooting is used for exact discretization, and (3) a convergence-guaranteed sequential convex programming (SCP) algorithm, the prox-linear method, is used to solve the discretized receding-horizon optimal control problems. The resulting NMPC framework is computationally efficient, owing to its support for warm-starting and premature termination of SCP, and its reliance on first-order information only. We demonstrate the effectiveness of the proposed NMPC framework by means of a numerical example with reference-tracking and obstacle avoidance. The implementation is available at https://github.com/UW-ACL/nmpc-ctcs
△ Less
Submitted 26 April, 2024;
originally announced May 2024.
-
Fast Monte Carlo Analysis for 6-DoF Powered-Descent Guidance via GPU-Accelerated Sequential Convex Programming
Authors:
Govind M. Chari,
Abhinav G. Kamath,
Purnanand Elango,
Behçet Açıkmeşe
Abstract:
We introduce a GPU-accelerated Monte Carlo framework for nonconvex, free-final-time trajectory optimization problems. This framework makes use of the prox-linear method, which belongs to the larger family of sequential convex programming (SCP) algorithms, in conjunction with a constraint reformulation that guarantees inter-sample constraint satisfaction. Key features of this framework are: (1) con…
▽ More
We introduce a GPU-accelerated Monte Carlo framework for nonconvex, free-final-time trajectory optimization problems. This framework makes use of the prox-linear method, which belongs to the larger family of sequential convex programming (SCP) algorithms, in conjunction with a constraint reformulation that guarantees inter-sample constraint satisfaction. Key features of this framework are: (1) continuous-time constraint satisfaction; (2) a matrix-inverse-free solution method; (3) the use of the proportional-integral projected gradient (PIPG) method, a first-order convex optimization solver, customized to the convex subproblem at hand; and, (4) an end-to-end, library-free implementation of the algorithm. We demonstrate this GPU-based framework on the 6-DoF powered-descent guidance problem, and show that it is faster than an equivalent serial CPU implementation for Monte Carlo simulations with over 1000 runs. To the best of our knowledge, this is the first GPU-based implementation of a general-purpose nonconvex trajectory optimization solver.
△ Less
Submitted 27 April, 2024;
originally announced April 2024.
-
Successive Convexification for Trajectory Optimization with Continuous-Time Constraint Satisfaction
Authors:
Purnanand Elango,
Dayou Luo,
Abhinav G. Kamath,
Samet Uzun,
Taewan Kim,
Behçet Açıkmeşe
Abstract:
We present successive convexification, a real-time-capable solution method for nonconvex trajectory optimization, with continuous-time constraint satisfaction and guaranteed convergence, that only requires first-order information. The proposed framework combines several key methods to solve a large class of nonlinear optimal control problems: (i) exterior penalty-based reformulation of the path co…
▽ More
We present successive convexification, a real-time-capable solution method for nonconvex trajectory optimization, with continuous-time constraint satisfaction and guaranteed convergence, that only requires first-order information. The proposed framework combines several key methods to solve a large class of nonlinear optimal control problems: (i) exterior penalty-based reformulation of the path constraints; (ii) generalized time-dilation; (iii) multiple-shooting discretization; (iv) $\ell_1$ exact penalization of the nonconvex constraints; and (v) the prox-linear method, a sequential convex programming (SCP) algorithm for convex-composite minimization. The reformulation of the path constraints enables continuous-time constraint satisfaction even on sparse discretization grids and obviates the need for mesh refinement heuristics. Through the prox-linear method, we guarantee convergence of the solution method to stationary points of the penalized problem and guarantee that the converged solutions that are feasible with respect to the discretized and control-parameterized optimal control problem are also Karush-Kuhn-Tucker (KKT) points. Furthermore, we highlight the specialization of this property to global minimizers of convex optimal control problems, wherein the reformulated path constraints cannot be represented by canonical cones, i.e., in the form required by existing convex optimization solvers. In addition to theoretical analysis, we demonstrate the effectiveness and real-time capability of the proposed framework with numerical examples based on popular optimal control applications: dynamic obstacle avoidance and rocket landing.
△ Less
Submitted 25 April, 2024;
originally announced April 2024.
-
Expected Time-Optimal Control: a Particle MPC-based Approach via Sequential Convex Programming
Authors:
Kazuya Echigo,
Abhishek Cauligi,
Behçet Açıkmeşe
Abstract:
In this paper, we consider the problem of minimum-time optimal control for a dynamical system with initial state uncertainties and propose a sequential convex programming (SCP) solution framework. We seek to minimize the expected terminal (mission) time, which is an essential capability for planetary exploration missions where ground rovers have to carry out scientific tasks efficiently within the…
▽ More
In this paper, we consider the problem of minimum-time optimal control for a dynamical system with initial state uncertainties and propose a sequential convex programming (SCP) solution framework. We seek to minimize the expected terminal (mission) time, which is an essential capability for planetary exploration missions where ground rovers have to carry out scientific tasks efficiently within the mission timelines in uncertain environments. Our main contribution is to convert the underlying stochastic optimal control problem into a deterministic, numerically tractable, optimal control problem. To this end, the proposed solution framework combines two strategies from previous methods: i) a partial model predictive control with consensus horizon approach and ii) a sum-of-norm cost, a temporally strictly increasing weighted-norm, promoting minimum-time trajectories. Our contribution is to adopt these formulations into an SCP solution framework and obtain a numerically tractable stochastic control algorithm. We then demonstrate the resulting control method in multiple applications: i) a closed-loop linear system as a representative result (a spacecraft double integrator model), ii) an open-loop linear system (the same model), and then iii) a nonlinear system (Dubin's car).
△ Less
Submitted 24 April, 2024;
originally announced April 2024.
-
Constraint Preconditioning and Parameter Selection for a First-Order Primal-Dual Method applied to Model Predictive Control
Authors:
Govind M. Chari,
Yue Yu,
Behçet Açıkmeşe
Abstract:
Many techniques for real-time trajectory optimization and control require the solution of optimization problems at high frequencies. However, ill-conditioning in the optimization problem can significantly reduce the speed of first-order primal-dual optimization algorithms. We introduce a preconditioning technique and step-size heuristic for Proportional-Integral Projected Gradient (PIPG), a first-…
▽ More
Many techniques for real-time trajectory optimization and control require the solution of optimization problems at high frequencies. However, ill-conditioning in the optimization problem can significantly reduce the speed of first-order primal-dual optimization algorithms. We introduce a preconditioning technique and step-size heuristic for Proportional-Integral Projected Gradient (PIPG), a first-order primal-dual algorithm. The preconditioning technique, based on the QR factorization, aims to reduce the condition number of the KKT matrix associated with the optimization problem. Our step-size selection heuristic chooses step-sizes to minimize the upper bound on the convergence of the primal-dual gap for the optimization problem. These algorithms are tested on two model predictive control problem examples and show a solve-time reduction of at least 3.6x.
△ Less
Submitted 22 March, 2024;
originally announced March 2024.
-
Remarks on "Successive Convexification: A Superlinearly Convergent Algorithm for Non-convex Optimal Control Problems"
Authors:
Dayou Luo,
Purnanand Elango,
Behcet Acikmese
Abstract:
The purpose of this note is to highlight and address inaccuracies in the convergence guarantees of SCvx, a nonconvex trajectory optimization algorithm proposed by Mao et al. (arXiv:1804.06539), and make connections to relevant prior work. Specifically, we identify errors in the convergence proof within Mao et al. (arXiv:1804.06539) and reestablish the proof of convergence by employing a new method…
▽ More
The purpose of this note is to highlight and address inaccuracies in the convergence guarantees of SCvx, a nonconvex trajectory optimization algorithm proposed by Mao et al. (arXiv:1804.06539), and make connections to relevant prior work. Specifically, we identify errors in the convergence proof within Mao et al. (arXiv:1804.06539) and reestablish the proof of convergence by employing a new method under stricter assumptions.
△ Less
Submitted 13 March, 2024; v1 submitted 1 March, 2024;
originally announced March 2024.
-
Sufficient LMI Copositivity Conditions for Funnel Synthesis of Uncertain Nonlinear Systems
Authors:
Taewan Kim,
Behçet Açıkmeşe
Abstract:
Funnel synthesis refers to a procedure for synthesizing a time-varying controlled invariant set and an associated control law around a nominal trajectory. The computation of the funnel involves solving a continuous-time differential equation or inequality, ensuring the invariance of the funnel. Previous approaches often compromise the invariance property of the funnel; for example, they may enforc…
▽ More
Funnel synthesis refers to a procedure for synthesizing a time-varying controlled invariant set and an associated control law around a nominal trajectory. The computation of the funnel involves solving a continuous-time differential equation or inequality, ensuring the invariance of the funnel. Previous approaches often compromise the invariance property of the funnel; for example, they may enforce the equation or the inequality only at discrete temporal nodes and not having formal guarantee of invariance at all times. This paper proposes a computational funnel synthesis method that can satisfy the invariance of the funnel without such compromises. We derive a finite number of linear matrix inequalities (LMIs) that imply the satifaction of a continuous-time differential linear matrix inequality guaranteeing the invariance of the funnel at all times from the initial to the final time. To this end, we utilize LMI conditions ensuring matrix copositivity, which then imply continuous-time invariance. The primary contribution of the paper is to prove that the resulting funnel is indeed invariant over a finite time horizon. We validate the proposed method via a three-dimensional trajectory planning and control problem with obstacle avoidance constraints.
△ Less
Submitted 23 February, 2024;
originally announced February 2024.
-
Spacecraft Rendezvous Guidance via Factorization-Free Sequential Convex Programming using a First-Order Method
Authors:
Govind M. Chari,
Behçet Açıkmeşe
Abstract:
We implement a fully factorization-free algorithm for nonconvex, free-final-time trajectory optimization. This algorithm is based on sequential convex programming and utilizes an inverse-free, exact discretization procedure to ensure dynamic feasibility of the converged trajectory and PIPG, a fast, first-order conic optimization algorithm as the subproblem solver. Although PIPG requires the tuning…
▽ More
We implement a fully factorization-free algorithm for nonconvex, free-final-time trajectory optimization. This algorithm is based on sequential convex programming and utilizes an inverse-free, exact discretization procedure to ensure dynamic feasibility of the converged trajectory and PIPG, a fast, first-order conic optimization algorithm as the subproblem solver. Although PIPG requires the tuning of a hyperparameter to achieve fastest convergence, we show that PIPG can be tuned to a nominal trajectory optimization problem and it is robust to variations in initial condition. We demonstrate this with a monte carlo simulation of the free-final-time rendezvous problem, using Clohessy-Wiltshire dynamics, an impulsive thrust model, and various state and control constraints including a spherical keepout zone.
△ Less
Submitted 6 February, 2024;
originally announced February 2024.
-
Blameless and Optimal Control under Prioritized Safety Constraints
Authors:
Natalia Pavlasek,
Sarah H. Q. Li,
Behçet Açıkmeşe,
Meeko Oishi,
Claus Danielson
Abstract:
In many resource-limited optimal control problems, multiple constraints may be enforced that are jointly infeasible due to external factors such as subsystem failures, unexpected disturbances, or fuel limitations. In this manuscript, we introduce the concept of blameless optimality to characterize control actions that a) satisfy the highest prioritized and feasible constraints and b) remain optima…
▽ More
In many resource-limited optimal control problems, multiple constraints may be enforced that are jointly infeasible due to external factors such as subsystem failures, unexpected disturbances, or fuel limitations. In this manuscript, we introduce the concept of blameless optimality to characterize control actions that a) satisfy the highest prioritized and feasible constraints and b) remain optimal with respect to a mission objective. For a general optimal control problem with jointly infeasible constraints, we prove that a single optimization problem cannot find a blamelessly optimal control sequence. Instead, finding blamelessly optimal control actions requires sequentially solving at least two optimal control problems: one to determine the highest priority level of constraints that is feasible and another to determine the optimal control action with respect to these constraints. We apply our results to a rocket landing scenario in which violating at least one safety-induced landing constraint is unavoidable. Leveraging the concept of blameless optimality, we formulate blamelessly optimal controllers that can autonomously prioritize the constraints most critical to a mission.
△ Less
Submitted 3 November, 2023; v1 submitted 13 April, 2023;
originally announced April 2023.
-
HALO: Hazard-Aware Landing Optimization for Autonomous Systems
Authors:
Christopher R. Hayner,
Samuel C. Buckner,
Daniel Broyles,
Evelyn Madewell,
Karen Leung,
Behcet Acikmese
Abstract:
With autonomous aerial vehicles enacting safety-critical missions, such as the Mars Science Laboratory Curiosity rover's landing on Mars, the tasks of automatically identifying and reasoning about potentially hazardous landing sites is paramount. This paper presents a coupled perception-planning solution which addresses the hazard detection, optimal landing trajectory generation, and contingency p…
▽ More
With autonomous aerial vehicles enacting safety-critical missions, such as the Mars Science Laboratory Curiosity rover's landing on Mars, the tasks of automatically identifying and reasoning about potentially hazardous landing sites is paramount. This paper presents a coupled perception-planning solution which addresses the hazard detection, optimal landing trajectory generation, and contingency planning challenges encountered when landing in uncertain environments. Specifically, we develop and combine two novel algorithms, Hazard-Aware Landing Site Selection (HALSS) and Adaptive Deferred-Decision Trajectory Optimization (Adaptive-DDTO), to address the perception and planning challenges, respectively. The HALSS framework processes point cloud information to identify feasible safe landing zones, while Adaptive-DDTO is a multi-target contingency planner that adaptively replans as new perception information is received. We demonstrate the efficacy of our approach using a simulated Martian environment and show that our coupled perception-planning method achieves greater landing success whilst being more fuel efficient compared to a nonadaptive DDTO approach.
△ Less
Submitted 4 April, 2023;
originally announced April 2023.
-
Optimization-based Constrained Funnel Synthesis for Systems with Lipschitz Nonlinearities via Numerical Optimal Control
Authors:
Taewan Kim,
Purnanand Elango,
Taylor P. Reynolds,
Behçet Açıkmeşe,
Mehran Mesbahi
Abstract:
This paper presents a funnel synthesis algorithm for computing controlled invariant sets and feedback control gains around a given nominal trajectory for dynamical systems with locally Lipschitz nonlinearities and bounded disturbances. The resulting funnel synthesis problem involves a differential linear matrix inequality (DLMI) whose solution satisfies a Lyapunov condition that implies invariance…
▽ More
This paper presents a funnel synthesis algorithm for computing controlled invariant sets and feedback control gains around a given nominal trajectory for dynamical systems with locally Lipschitz nonlinearities and bounded disturbances. The resulting funnel synthesis problem involves a differential linear matrix inequality (DLMI) whose solution satisfies a Lyapunov condition that implies invariance and attractivity properties. Due to these properties, the proposed method can balance maximization of initial invariant funnel size, i.e., size of the funnel entry, and minimization of the size of the attractive funnel for attenuating the effect of disturbance. To solve the resulting funnel synthesis problem with the DLMI as constraints, we employ a numerical optimal control approach that uses a multiple shooting method to convert the problem into a finite dimensional semidefinite programming problem. This framework does not require piecewise linear system matrices and funnel parameters, which is typically assumed in recent related work. We illustrate the proposed funnel synthesis method with a numerical example.
△ Less
Submitted 1 July, 2023; v1 submitted 18 March, 2023;
originally announced March 2023.
-
Real-Time Sequential Conic Optimization for Multi-Phase Rocket Landing Guidance
Authors:
Abhinav G. Kamath,
Purnanand Elango,
Yue Yu,
Skye Mceowen,
Govind M. Chari,
John M. Carson III,
Behçet Açıkmeşe
Abstract:
We introduce a multi-phase rocket landing guidance framework that can handle nonlinear dynamics and does not mandate any additional mixed-integer or nonconvex constraints to handle discrete temporal events/switching. To achieve this, we first introduce sequential conic optimization (SeCO), a new paradigm for solving nonconvex optimal control problems that is entirely devoid of matrix factorization…
▽ More
We introduce a multi-phase rocket landing guidance framework that can handle nonlinear dynamics and does not mandate any additional mixed-integer or nonconvex constraints to handle discrete temporal events/switching. To achieve this, we first introduce sequential conic optimization (SeCO), a new paradigm for solving nonconvex optimal control problems that is entirely devoid of matrix factorizations and inversions. This framework combines sequential convex programming (SCP) and first-order conic optimization and can solve unified multi-phase trajectory optimization problems in real-time. The novel features of this framework are: (1) time-interval dilation, which enables multi-phase trajectory optimization with free-transition-time; (2) single-crossing compound state-triggered constraints, which are entirely convex if the trigger and constraint conditions are convex; (3) virtual state, which is a new approach to handling artificial infeasibility in SCP methods that preserves the shapes of the constraint sets; and, (4) the use of the proportional-integral projected gradient method (PIPG), a high-performance first-order conic optimization solver, in tandem with the penalized trust region (PTR) SCP algorithm. We demonstrate the efficacy and real-time capability of SeCO by solving a relevant multi-phase rocket landing guidance problem with nonlinear dynamics and convex constraints only, and observe that our solver is 2.7 times faster than a state-of-the-art convex optimization solver.
△ Less
Submitted 29 May, 2023; v1 submitted 1 December, 2022;
originally announced December 2022.
-
Joint Synthesis of Trajectory and Controlled Invariant Funnel for Discrete-time Systems with Locally Lipschitz Nonlinearities
Authors:
Taewan Kim,
Purnanand Elango,
Behcet Acikmese
Abstract:
This paper presents a joint synthesis algorithm of trajectory and controlled invariant funnel (CIF) for locally Lipschitz nonlinear systems subject to bounded disturbances. The CIF synthesis refers to a procedure of computing controlled invariance sets and corresponding feedback gains. In contrast to existing CIF synthesis methods that compute the CIF with a pre-defined nominal trajectory, our wor…
▽ More
This paper presents a joint synthesis algorithm of trajectory and controlled invariant funnel (CIF) for locally Lipschitz nonlinear systems subject to bounded disturbances. The CIF synthesis refers to a procedure of computing controlled invariance sets and corresponding feedback gains. In contrast to existing CIF synthesis methods that compute the CIF with a pre-defined nominal trajectory, our work aims to optimize the nominal trajectory and the CIF jointly to satisfy feasibility conditions without the relaxation of constraints and obtain a more cost-optimal nominal trajectory. The proposed work has a recursive scheme that mainly optimize trajectory update and funnel update. The trajectory update step optimizes the nominal trajectory while ensuring the feasibility of the CIF. Then, the funnel update step computes the funnel around the nominal trajectory so that the CIF guarantees an invariance property. As a result, with the optimized trajectory and CIF, any resulting trajectory propagated from an initial set by the control law with the computed feedback gain remains within the feasible region around the nominal trajectory under the presence of bounded disturbances. We validate the proposed method via two applications from robotics.
△ Less
Submitted 12 January, 2024; v1 submitted 7 September, 2022;
originally announced September 2022.
-
Real-Time Quadrotor Trajectory Optimization with Time-Triggered Corridor Constraints
Authors:
Yue Yu,
Kartik Nagpal,
Skye Mceowen,
Behçet Açıkmeşe,
Ufuk Topcu
Abstract:
One of the keys to flying quadrotors is to optimize their trajectories within the set of collision-free corridors. These corridors impose nonconvex constraints on the trajectories, making real-time trajectory optimization challenging. We introduce a novel numerical method that approximates the nonconvex corridor constraints with time-triggered convex corridor constraints. This method combines bise…
▽ More
One of the keys to flying quadrotors is to optimize their trajectories within the set of collision-free corridors. These corridors impose nonconvex constraints on the trajectories, making real-time trajectory optimization challenging. We introduce a novel numerical method that approximates the nonconvex corridor constraints with time-triggered convex corridor constraints. This method combines bisection search and repeated infeasibility detection. We further develop a customized C++ implementation of the proposed method, based on a first-order conic optimization method that detects infeasibility and exploits problem structure. We demonstrate the efficiency and effectiveness of the proposed method using numerical simulation on randomly generated problem instances as well as indoor flight experiments with hoop obstacles. Compared with mixed integer programming, the proposed method is about 50--200 times faster.
△ Less
Submitted 15 August, 2022;
originally announced August 2022.
-
Set-based value operators for non-stationary Markovian environments
Authors:
Sarah H. Q. Li,
Assalé Adjé,
Pierre-Loïc Garoche,
Behçet Açıkmeşe
Abstract:
This paper analyzes finite state Markov Decision Processes (MDPs) with uncertain parameters in compact sets and re-examines results from robust MDP via set-based fixed point theory. To this end, we generalize the Bellman and policy evaluation operators to contracting operators on the value function space and denote them as \emph{value operators}. We lift these value operators to act on \emph{sets}…
▽ More
This paper analyzes finite state Markov Decision Processes (MDPs) with uncertain parameters in compact sets and re-examines results from robust MDP via set-based fixed point theory. To this end, we generalize the Bellman and policy evaluation operators to contracting operators on the value function space and denote them as \emph{value operators}. We lift these value operators to act on \emph{sets} of value functions and denote them as \emph{set-based value operators}. We prove that the set-based value operators are \emph{contractions} in the space of compact value function sets. Leveraging insights from set theory, we generalize the rectangularity condition in classic robust MDP literature to a containment condition for all value operators, which is weaker and can be applied to a larger set of parameter-uncertain MDPs and contracting operators in dynamic programming. We prove that both the rectangularity condition and the containment condition sufficiently ensure that the set-based value operator's fixed point set contains its own extrema elements. For convex and compact sets of uncertain MDP parameters, we show equivalence between the classic robust value function and the supremum of the fixed point set of the set-based Bellman operator. Under dynamically changing MDP parameters in compact sets, we prove a set convergence result for value iteration, which otherwise may not converge to a single value function. Finally, we derive novel guarantees for probabilistic path-planning problems in planet exploration and stratospheric station-kee**.
△ Less
Submitted 8 August, 2023; v1 submitted 14 July, 2022;
originally announced July 2022.
-
Extrapolated Proportional-Integral Projected Gradient Method for Conic Optimization
Authors:
Yue Yu,
Purnanand Elango,
Behçet Açıkmeşe,
Ufuk Topcu
Abstract:
Conic optimization is the minimization of a convex quadratic function subject to conic constraints. We introduce a novel first-order method for conic optimization, named \emph{extrapolated proportional-integral projected gradient method (xPIPG)}, that automatically detects infeasibility. The iterates of xPIPG either asymptotically satisfy a set of primal-dual optimality conditions, or generate a p…
▽ More
Conic optimization is the minimization of a convex quadratic function subject to conic constraints. We introduce a novel first-order method for conic optimization, named \emph{extrapolated proportional-integral projected gradient method (xPIPG)}, that automatically detects infeasibility. The iterates of xPIPG either asymptotically satisfy a set of primal-dual optimality conditions, or generate a proof of primal or dual infeasibility. We demonstrate the application of xPIPG using benchmark problems in model predictive control. xPIPG outperforms many state-of-the-art conic optimization solvers, especially when solving large-scale problems.
△ Less
Submitted 23 June, 2022; v1 submitted 8 March, 2022;
originally announced March 2022.
-
SCvx-fast: A Superlinearly Convergent Algorithm for A Class of Non-Convex Optimal Control Problems
Authors:
Yuanqi Mao,
Behcet Acikmese
Abstract:
In this paper, we extend our previous results and formally propose the SCvx-fast algorithm, a new addition to the Successive Convexification algorithmic framework. The said algorithm solves non-convex optimal control problems with specific types of state constraints (i.e. union of convex keep-out zones) and is faster to converge than SCvx, its predecessor. In order to preserve more feasibility, th…
▽ More
In this paper, we extend our previous results and formally propose the SCvx-fast algorithm, a new addition to the Successive Convexification algorithmic framework. The said algorithm solves non-convex optimal control problems with specific types of state constraints (i.e. union of convex keep-out zones) and is faster to converge than SCvx, its predecessor. In order to preserve more feasibility, the proposed algorithm uses a novel project-and convexify procedure to successively convexify both state constraints and system dynamics, and thus a finite dimensional convex programming subproblem is solved at each succession. It also gets rid of the dependency on trust regions, gaining the ability to take larger steps and thus ultimately attaining faster convergence. The extension is in three folds as follows. i) We can now initialize the algorithm from an infeasible starting point, and regain feasibility in just one step; ii) We get rid of the smoothness conditions on the constraints so that a broader range of "obstacles" can be included. Significant changes are made to adjust the algorithm accordingly; iii) We obtain a proof of superlinear rate of convergence, a new theoretical result for SCvx-fast. Benefiting from its specific problem setup and the project-and convexify procedure, the SCvx-fast algorithm is particularly suitable for solving trajectory planning problems with collision avoidance constraints. Numerical simulations are performed, affirming the fast convergence rate. With powerful convex programming solvers, the algorithm can be implemented onboard for real-time autonomous guidance applications.
△ Less
Submitted 30 November, 2021;
originally announced December 2021.
-
Guided Policy Search using Sequential Convex Programming for Initialization of Trajectory Optimization Algorithms
Authors:
Taewan Kim,
Purnanand Elango,
Danylo Malyuta,
Behcet Acikmese
Abstract:
Nonlinear trajectory optimization algorithms have been developed to handle optimal control problems with nonlinear dynamics and nonconvex constraints in trajectory planning. The performance and computational efficiency of many trajectory optimization methods are sensitive to the initial guess, i.e., the trajectory guess needed by the recursive trajectory optimization algorithm. Motivated by this o…
▽ More
Nonlinear trajectory optimization algorithms have been developed to handle optimal control problems with nonlinear dynamics and nonconvex constraints in trajectory planning. The performance and computational efficiency of many trajectory optimization methods are sensitive to the initial guess, i.e., the trajectory guess needed by the recursive trajectory optimization algorithm. Motivated by this observation, we tackle the initialization problem for trajectory optimization via policy optimization. To optimize a policy, we propose a guided policy search method that has two key components: i) Trajectory update; ii) Policy update. The trajectory update involves offline solutions of a large number of trajectory optimization problems from different initial states via Sequential Convex Programming (SCP). Here we take a single SCP step to generate the trajectory iterate for each problem. In conjunction with these iterates, we also generate additional trajectories around each iterate via a feedback control law. Then all these trajectories are used by a stochastic gradient descent algorithm to update the neural network policy, i.e., the policy update step. As a result, the trained policy makes it possible to generate trajectory candidates that are close to the optimality and feasibility and that provide excellent initial guesses for the trajectory optimization methods. We validate the proposed method via a real-world 6-degree-of-freedom powered descent guidance problem for a reusable rocket.
△ Less
Submitted 19 May, 2022; v1 submitted 13 October, 2021;
originally announced October 2021.
-
Discrete-Time Linear-Quadratic Regulation via Optimal Transport
Authors:
Mathias Hudoba de Badyn,
Erik Miehling,
Dylan Janak,
Behçet Açıkmeşe,
Mehran Mesbahi,
Tamer Başar,
John Lygeros,
Roy S. Smith
Abstract:
In this paper, we consider a discrete-time stochastic control problem with uncertain initial and target states. We first discuss the connection between optimal transport and stochastic control problems of this form. Next, we formulate a linear-quadratic regulator problem where the initial and terminal states are distributed according to specified probability densities. A closed-form solution for t…
▽ More
In this paper, we consider a discrete-time stochastic control problem with uncertain initial and target states. We first discuss the connection between optimal transport and stochastic control problems of this form. Next, we formulate a linear-quadratic regulator problem where the initial and terminal states are distributed according to specified probability densities. A closed-form solution for the optimal transport map in the case of linear-time varying systems is derived, along with an algorithm for computing the optimal map. Two numerical examples pertaining to swarm deployment demonstrate the practical applicability of the model, and performance of the numerical method.
△ Less
Submitted 6 September, 2021;
originally announced September 2021.
-
Proportional-Integral Projected Gradient Method for Conic Optimization
Authors:
Yue Yu,
Purnanand Elango,
Ufuk Topcu,
Behçet Açıkmeşe
Abstract:
Conic optimization is the minimization of a differentiable convex objective function subject to conic constraints. We propose a novel primal-dual first-order method for conic optimization, named proportional-integral projected gradient method (PIPG). PIPG ensures that both the primal-dual gap and the constraint violation converge to zero at the rate of \(O(1/k)\), where \(k\) is the number of iter…
▽ More
Conic optimization is the minimization of a differentiable convex objective function subject to conic constraints. We propose a novel primal-dual first-order method for conic optimization, named proportional-integral projected gradient method (PIPG). PIPG ensures that both the primal-dual gap and the constraint violation converge to zero at the rate of \(O(1/k)\), where \(k\) is the number of iterations. If the objective function is strongly convex, PIPG improves the convergence rate of the primal-dual gap to \(O(1/k^2)\). Further, unlike any existing first-order methods, PIPG also improves the convergence rate of the constraint violation to \(O(1/k^3)\). We demonstrate the application of PIPG in constrained optimal control problems.
△ Less
Submitted 13 December, 2021; v1 submitted 23 August, 2021;
originally announced August 2021.
-
Advances in Trajectory Optimization for Space Vehicle Control
Authors:
Danylo Malyuta,
Yue Yu,
Purnanand Elango,
Behcet Acikmese
Abstract:
Space mission design places a premium on cost and operational efficiency. The search for new science and life beyond Earth calls for spacecraft that can deliver scientific payloads to geologically rich yet hazardous landing sites. At the same time, the last four decades of optimization research have put a suite of powerful optimization tools at the fingertips of the controls engineer. As we enter…
▽ More
Space mission design places a premium on cost and operational efficiency. The search for new science and life beyond Earth calls for spacecraft that can deliver scientific payloads to geologically rich yet hazardous landing sites. At the same time, the last four decades of optimization research have put a suite of powerful optimization tools at the fingertips of the controls engineer. As we enter the new decade, optimization theory, algorithms, and software tooling have reached a critical mass to start seeing serious application in space vehicle guidance and control systems. This survey paper provides a detailed overview of recent advances, successes, and promising directions for optimization-based space vehicle control. The considered applications include planetary landing, rendezvous and proximity operations, small body landing, constrained attitude reorientation, endo-atmospheric flight including ascent and reentry, and orbit transfer and injection. The primary focus is on the last ten years of progress, which have seen a veritable rise in the number of applications using three core technologies: lossless convexification, sequential convex programming, and model predictive control. The reader will come away with a well-rounded understanding of the state-of-the-art in each space vehicle control application, and will be well positioned to tackle important current open problems using convex optimization as a core technology.
△ Less
Submitted 23 August, 2021; v1 submitted 4 August, 2021;
originally announced August 2021.
-
Fast Homotopy for Spacecraft Rendezvous Trajectory Optimization with Discrete Logic
Authors:
Danylo Malyuta,
Behcet Acikmese
Abstract:
This paper presents a computationally efficient optimization algorithm for solving nonconvex optimal control problems that involve discrete logic constraints. Traditional solution methods for these constraints require binary variables and mixed-integer programming, which is prohibitively slow and computationally expensive. This paper targets a fast solution that is capable of real-time implementat…
▽ More
This paper presents a computationally efficient optimization algorithm for solving nonconvex optimal control problems that involve discrete logic constraints. Traditional solution methods for these constraints require binary variables and mixed-integer programming, which is prohibitively slow and computationally expensive. This paper targets a fast solution that is capable of real-time implementation onboard spacecraft. To do so, a novel algorithm is developed that blends sequential convex programming and numerical continuation into a single iterative solution process. Inside the algorithm, discrete logic constraints are approximated by smooth functions, and a homotopy parameter governs the accuracy of this approximation. As the algorithm converges, the homotopy parameter is updated such that the smooth approximations enforce the exact discrete logic. The effectiveness of this approach is numerically demonstrated for a realistic rendezvous scenario inspired by the Apollo Transposition and Docking maneuver. In under 15 seconds of cumulative solver time, the algorithm is able to reliably find difficult fuel-optimal trajectories that obey the following discrete logic constraints: thruster minimum impulse-bit, range-triggered approach cone, and range-triggered plume im**ement. The optimized trajectory uses significantly less fuel than reported NASA design targets.
△ Less
Submitted 14 July, 2021;
originally announced July 2021.
-
Convex Optimization for Trajectory Generation
Authors:
Danylo Malyuta,
Taylor P. Reynolds,
Michael Szmuk,
Thomas Lew,
Riccardo Bonalli,
Marco Pavone,
Behcet Acikmese
Abstract:
Reliable and efficient trajectory generation methods are a fundamental need for autonomous dynamical systems of tomorrow. The goal of this article is to provide a comprehensive tutorial of three major convex optimization-based trajectory generation methods: lossless convexification (LCvx), and two sequential convex programming algorithms known as SCvx and GuSTO. In this article, trajectory generat…
▽ More
Reliable and efficient trajectory generation methods are a fundamental need for autonomous dynamical systems of tomorrow. The goal of this article is to provide a comprehensive tutorial of three major convex optimization-based trajectory generation methods: lossless convexification (LCvx), and two sequential convex programming algorithms known as SCvx and GuSTO. In this article, trajectory generation is the computation of a dynamically feasible state and control signal that satisfies a set of constraints while optimizing key mission objectives. The trajectory generation problem is almost always nonconvex, which typically means that it is not readily amenable to efficient and reliable solution onboard an autonomous vehicle. The three algorithms that we discuss use problem reformulation and a systematic algorithmic strategy to nonetheless solve nonconvex trajectory generation tasks through the use of a convex optimizer. The theoretical guarantees and computational speed offered by convex optimization have made the algorithms popular in both research and industry circles. To date, the list of applications includes rocket landing, spacecraft hypersonic reentry, spacecraft rendezvous and docking, aerial motion planning for fixed-wing and quadrotor vehicles, robot motion planning, and more. Among these applications are high-profile rocket flights conducted by organizations like NASA, Masten Space Systems, SpaceX, and Blue Origin. This article aims to give the reader the tools and understanding necessary to work with each algorithm, and to know what each method can and cannot do. A publicly available source code repository supports the provided numerical examples. By the end of the article, the reader should be ready to use the methods, to extend them, and to contribute to their many exciting modern applications.
△ Less
Submitted 16 June, 2021;
originally announced June 2021.
-
Decentralized State-Dependent Markov Chain Synthesis with an Application to Swarm Guidance
Authors:
Samet Uzun,
Nazim Kemal Ure,
Behcet Acikmese
Abstract:
This paper introduces a decentralized state-dependent Markov chain synthesis (DSMC) algorithm for finite-state Markov chains. We present a state-dependent consensus protocol that achieves exponential convergence under mild technical conditions, without relying on any connectivity assumptions regarding the dynamic network topology. Utilizing the proposed consensus protocol, we develop the DSMC algo…
▽ More
This paper introduces a decentralized state-dependent Markov chain synthesis (DSMC) algorithm for finite-state Markov chains. We present a state-dependent consensus protocol that achieves exponential convergence under mild technical conditions, without relying on any connectivity assumptions regarding the dynamic network topology. Utilizing the proposed consensus protocol, we develop the DSMC algorithm, updating the Markov matrix based on the current state while ensuring the convergence conditions of the consensus protocol. This result establishes the desired steady-state distribution for the resulting Markov chain, ensuring exponential convergence from all initial distributions while adhering to transition constraints and minimizing state transitions. The DSMC's performance is demonstrated through a probabilistic swarm guidance example, which interprets the spatial distribution of a swarm comprising a large number of mobile agents as a probability distribution and utilizes the Markov chain to compute transition probabilities between states. Simulation results demonstrate faster convergence for the DSMC based algorithm when compared to the previous Markov chain based swarm guidance algorithms.
△ Less
Submitted 26 April, 2024; v1 submitted 4 December, 2020;
originally announced December 2020.
-
Synthesis of Interval Observers for Nonlinear Discrete-Time Systems
Authors:
Adam M Tahir,
Behcet Acikmese
Abstract:
A systematic procedure to synthesize interval observers for nonlinear discrete-time systems is proposed. The feedback gains and other matrices are found from the solutions to semidefinite feasibility programs. Two cases are considered: (1) the interval observer is in the same coordinate frame as the given system, and (2) the interval observer uses a coordinate transformation. The conditions where…
▽ More
A systematic procedure to synthesize interval observers for nonlinear discrete-time systems is proposed. The feedback gains and other matrices are found from the solutions to semidefinite feasibility programs. Two cases are considered: (1) the interval observer is in the same coordinate frame as the given system, and (2) the interval observer uses a coordinate transformation. The conditions where coordinate transformations are necessary are detailed. Numerical examples are provided to showcase the effectiveness of the interval observers and demonstrate their application to sampled-data systems.
△ Less
Submitted 8 November, 2020;
originally announced November 2020.
-
Proportional-Integral Projected Gradient Method for Model Predictive Control
Authors:
Yue Yu,
Purnanand Elango,
Behçet Açikmeşe
Abstract:
Recently there has been an increasing interest in primal-dual methods for model predictive control (MPC), which require minimizing the (augmented) Lagrangian at each iteration. We propose a novel first order primal-dual method, termed \emph{proportional-integral projected gradient method}, for MPC where the underlying finite horizon optimal control problem has both state and input constraints. Ins…
▽ More
Recently there has been an increasing interest in primal-dual methods for model predictive control (MPC), which require minimizing the (augmented) Lagrangian at each iteration. We propose a novel first order primal-dual method, termed \emph{proportional-integral projected gradient method}, for MPC where the underlying finite horizon optimal control problem has both state and input constraints. Instead of minimizing the (augmented) Lagrangian, each iteration of our method only computes a single projection onto the state and input constraint set. Our method ensures that, along a sequence of averaged iterates, both the distance to optimum and the constraint violation converge to zero at a rate of \(O(1/k)\) if the objective function is convex, where \(k\) is the iteration number. If the objective function is strongly convex, this rate can be improved to \(O(1/k^2)\) for the distance to optimum and \(O(1/k^3)\) for the constraint violation. We compare our method against existing methods via a trajectory-planning example with convexified keep-out-zone constraints.
△ Less
Submitted 18 December, 2020; v1 submitted 15 September, 2020;
originally announced September 2020.
-
RC Circuits based Distributed Conditional Gradient Method
Authors:
Yue Yu,
Behçet Açıkmeşe
Abstract:
We consider distributed optimization on undirected connected graphs. We propose a novel distributed conditional gradient method with (O(1/\sqrt{k})) convergence. Compared with existing methods, each iteration of our method uses both communication and linear minimization step only once rather than multiple times. We further extend our results to cases with composite local constraints. We demonstrat…
▽ More
We consider distributed optimization on undirected connected graphs. We propose a novel distributed conditional gradient method with (O(1/\sqrt{k})) convergence. Compared with existing methods, each iteration of our method uses both communication and linear minimization step only once rather than multiple times. We further extend our results to cases with composite local constraints. We demonstrate our results via examples on distributed matrix completion problem.
△ Less
Submitted 15 March, 2020;
originally announced March 2020.
-
Bounding Fixed Points of Set-Based Bellman Operator and Nash Equilibria of Stochastic Games
Authors:
Sarah H. Q. Li,
Assalé,
Adjé,
Pierre-Loïc Garoche,
Behçet Açıkmeşe
Abstract:
Motivated by uncertain parameters encountered in Markov decision processes (MDPs) and stochastic games, we study the effect of parameter uncertainty on Bellman operator-based algorithms under a set-based framework. Specifically, we first consider a family of MDPs where the cost parameters are in a given compact set; we then define a Bellman operator acting on a set of value functions to produce a…
▽ More
Motivated by uncertain parameters encountered in Markov decision processes (MDPs) and stochastic games, we study the effect of parameter uncertainty on Bellman operator-based algorithms under a set-based framework. Specifically, we first consider a family of MDPs where the cost parameters are in a given compact set; we then define a Bellman operator acting on a set of value functions to produce a new set of value functions as the output under all possible variations in the cost parameter. We prove the existence of a fixed point of this set-based Bellman operator by showing that it is contractive on a complete metric space, and explore its relationship with the corresponding family of MDPs and stochastic games. Additionally, we show that given interval set bounded cost parameters, we can form exact bounds on the set of optimal value functions. Finally, we utilize our results to bound the value function trajectory of a player in a stochastic game.
△ Less
Submitted 10 October, 2020; v1 submitted 22 January, 2020;
originally announced January 2020.
-
Fixed Points of the Set-Based Bellman Operator
Authors:
Sarah H. Q. Li,
Assalé Adjé,
Pierre-Loïc Garoche,
Behçet Açıkmeşe
Abstract:
Motivated by uncertain parameters encountered in Markov decision processes (MDPs), we study the effect of parameter uncertainty on Bellman operator-based methods. Specifically, we consider a family of MDPs where the cost parameters are from a given compact set. We then define a Bellman operator acting on an input set of value functions to produce a new set of value functions as the output under al…
▽ More
Motivated by uncertain parameters encountered in Markov decision processes (MDPs), we study the effect of parameter uncertainty on Bellman operator-based methods. Specifically, we consider a family of MDPs where the cost parameters are from a given compact set. We then define a Bellman operator acting on an input set of value functions to produce a new set of value functions as the output under all possible variations in the cost parameters. Finally we prove the existence of a fixed point of this set-based Bellman operator by showing that it is a contractive operator on a complete metric space.
△ Less
Submitted 29 February, 2020; v1 submitted 13 January, 2020;
originally announced January 2020.
-
Lossless Convexification of Optimal Control Problems with Semi-continuous Inputs
Authors:
Danylo Malyuta,
Behcet Acikmese
Abstract:
This paper presents a novel convex optimization-based method for finding the globally optimal solutions of a class of mixed-integer non-convex optimal control problems. We consider problems with non-convex constraints that restrict the input norms to be either zero or lower- and upper-bounded. The non-convex problem is relaxed to a convex one whose optimal solution is proved to be optimal almost e…
▽ More
This paper presents a novel convex optimization-based method for finding the globally optimal solutions of a class of mixed-integer non-convex optimal control problems. We consider problems with non-convex constraints that restrict the input norms to be either zero or lower- and upper-bounded. The non-convex problem is relaxed to a convex one whose optimal solution is proved to be optimal almost everywhere for the original problem, a procedure known as lossless convexification. This paper is the first to allow individual input sets to overlap and to have different norm bounds, integral input and state costs, and convex state constraints that can be activated at discrete time instances. The solution relies on second-order cone programming and demonstrates that a meaningful class of optimal control problems with binary variables can be solved reliably and in polynomial time. A rocket landing example with a coupled thrust-gimbal constraint corroborates the effectiveness of the approach.
△ Less
Submitted 18 November, 2019;
originally announced November 2019.
-
RLC Circuits based Distributed Mirror Descent Method
Authors:
Yue Yu,
Behçet Açıkmeşe
Abstract:
We consider distributed optimization with smooth convex objective functions defined on an undirected connected graph. Inspired by mirror descent mehod and RLC circuits, we propose a novel distributed mirror descent method. Compared with mirror-prox method, our algorithm achieves the same \(\mathcal{O}(1/k)\) iteration complexity with only half the computation cost per iteration. We further extend…
▽ More
We consider distributed optimization with smooth convex objective functions defined on an undirected connected graph. Inspired by mirror descent mehod and RLC circuits, we propose a novel distributed mirror descent method. Compared with mirror-prox method, our algorithm achieves the same \(\mathcal{O}(1/k)\) iteration complexity with only half the computation cost per iteration. We further extend our results to cases where a) gradients are corrupted by stochastic noise, and b) objective function is composed of both smooth and non-smooth terms. We demonstrate our theoretical results via numerical experiments.
△ Less
Submitted 22 February, 2020; v1 submitted 14 November, 2019;
originally announced November 2019.
-
Sensitivity Analysis for Markov Decision Process Congestion Games
Authors:
Sarah H. Q. Li,
Daniel Calderone,
Lillian Ratliff,
Behcet Acikmese
Abstract:
We consider a non-atomic congestion game where each decision maker performs selfish optimization over states of a common MDP. The decision makers optimize for their own expected costs, and influence each other through congestion effects on the state-action costs. We analyze on the sensitivity of MDP congestion game equilibria to uncertainty and perturbations in the state-action costs by applying a…
▽ More
We consider a non-atomic congestion game where each decision maker performs selfish optimization over states of a common MDP. The decision makers optimize for their own expected costs, and influence each other through congestion effects on the state-action costs. We analyze on the sensitivity of MDP congestion game equilibria to uncertainty and perturbations in the state-action costs by applying an implicit function type analysis. The occurrence of a stochastic Braess paradox is defined, analyzed based on sensitivity of game equilibria and demonstrated in simulation. We further analyze how the introduction of stochastic dynamics affects the magnitude of Braess paradox in comparison to deterministic dynamics.
△ Less
Submitted 12 September, 2019; v1 submitted 9 September, 2019;
originally announced September 2019.
-
Adaptive Constraint Satisfaction for Markov Decision Process Congestion Games: Application to Transportation Networks
Authors:
Sarah H. Q. Li,
Yue Yu,
Nicolas Miguel,
Dan Calderone,
Lillian J. Ratliff,
Behcet Acikmese
Abstract:
Under the Markov decision process (MDP) congestion game framework, we study the problem of enforcing population distribution constraints on a population of players with stochastic dynamics and coupled congestion costs. Existing research demonstrates that the constraints on the players' population distribution can be satisfied by enforcing tolls. However, computing the minimum toll value for constr…
▽ More
Under the Markov decision process (MDP) congestion game framework, we study the problem of enforcing population distribution constraints on a population of players with stochastic dynamics and coupled congestion costs. Existing research demonstrates that the constraints on the players' population distribution can be satisfied by enforcing tolls. However, computing the minimum toll value for constraint satisfaction requires accurate modeling of the player's congestion costs. Motivated by settings where an accurate congestion cost model is unavailable (e.g. transportation networks), we consider an MDP congestion game with unknown congestion costs. We assume that a constraint-enforcing authority can repeatedly enforce tolls on a population of players who converges to an $ε$-optimal population distribution for any given toll. We then construct a myopic update algorithm to compute the minimum toll value while ensuring that the constraints are satisfied on average. We analyze how the players' sub-optimal responses to tolls impact the rates of convergence towards the minimum toll value and constraint satisfaction. Finally, we construct a congestion game model for Uber drivers in Manhattan, New York City (NYC) using data from the Taxi and Limousine Commission (TLC) to illustrate how to efficiently reduce congestion while minimizing the impact on driver earnings.
△ Less
Submitted 14 August, 2022; v1 submitted 21 July, 2019;
originally announced July 2019.
-
Fast Trajectory Optimization via Successive Convexification for Spacecraft Rendezvous with Integer Constraints
Authors:
Danylo Malyuta,
Taylor P. Reynolds,
Michael Szmuk,
Behcet Acikmese,
Mehran Mesbahi
Abstract:
In this paper we present a fast method based on successive convexification for generating fuel-optimized spacecraft rendezvous trajectories in the presence of mixed-integer constraints. A recently developed paradigm of state-triggered constraints allows to efficiently embed a subset of discrete decision constraints into the continuous optimization framework of successive convexification. As a resu…
▽ More
In this paper we present a fast method based on successive convexification for generating fuel-optimized spacecraft rendezvous trajectories in the presence of mixed-integer constraints. A recently developed paradigm of state-triggered constraints allows to efficiently embed a subset of discrete decision constraints into the continuous optimization framework of successive convexification. As a result, we are able to solve difficult trajectory optimization problems at interactive speeds, as opposed to a mixed-integer programming approach that would require significantly more solution time and computing power. Our method is applied to the real problem of transposition and docking of the Apollo command and service module with the lunar module. We demonstrate that, within seconds, we are able to obtain trajectories that are up to 90 percent more fuel efficient (saving up to 45 kg of fuel) than non-optimization based Apollo-era design targets. Our trajectories take explicit account of minimum thrust pulse width and plume im**ement constraints. Both of these constraints are naturally mixed-integer, but we handle them as state-triggered constraints. In its current state, our algorithm will serve as a useful off-line design tool for rapid trajectory trade studies.
△ Less
Submitted 11 June, 2019;
originally announced June 2019.
-
Maximal Invariant Set Computation and Design for Markov Chains
Authors:
Dylan Janak,
Behçet Açıkmeşe
Abstract:
We describe an algorithm for computing the maximal invariant set for a Markov chain with linear safety constraints on the distribution over states. We then propose a Markov chain synthesis method that guarantees finite determination of the maximal invariant set. Although this problem is bilinear in the general case, we are able to optimize the convergence rate to a desirable steady-state distribut…
▽ More
We describe an algorithm for computing the maximal invariant set for a Markov chain with linear safety constraints on the distribution over states. We then propose a Markov chain synthesis method that guarantees finite determination of the maximal invariant set. Although this problem is bilinear in the general case, we are able to optimize the convergence rate to a desirable steady-state distribution over reversible Markov chains by solving a Semidefinite Program (SDP), which promotes efficient computation of the maximal invariant set. We then demonstrate this approach with a decentralized swarm guidance application subject to density upper bounds.
△ Less
Submitted 2 May, 2019;
originally announced May 2019.
-
Dual Quaternion Based Powered Descent Guidance with State-Triggered Constraints
Authors:
Taylor P. Reynolds,
Michael Szmuk,
Danylo Malyuta,
Mehran Mesbahi,
Behcet Acikmese,
John M. Carson III
Abstract:
This paper presents a numerical algorithm for computing 6-degree-of-freedom free-final-time powered descent guidance trajectories. The trajectory generation problem is formulated using a unit dual quaternion representation of the rigid body dynamics, and several standard path constraints. Our formulation also includes a special line of sight constraints that is enforced only within a specified ban…
▽ More
This paper presents a numerical algorithm for computing 6-degree-of-freedom free-final-time powered descent guidance trajectories. The trajectory generation problem is formulated using a unit dual quaternion representation of the rigid body dynamics, and several standard path constraints. Our formulation also includes a special line of sight constraints that is enforced only within a specified band of slant ranges relative to the landing site, a novel feature that is especially relevant to Terrain and Hazard Relative Navigation. We use the newly introduced state-triggered constraints to formulate these range constraints in a manner that is amenable to real-time implementations. The resulting non-convex optimal control problem is solved iteratively as a sequence of convex second-order cone programs that locally approximate the non-convex problem. Each second-order cone program is solved using a customizable interior point method solver. Also introduced are a scaling method and a new heuristic technique that guide the convergence process towards dynamic feasibility. To demonstrate the capabilities of our algorithm, two numerical case studies are presented. The first studies the effect of including a slant-range-triggered line of sight constraint on the resulting trajectories. The second study performs a Monte Carlo analysis to assess the algorithm's robustness to initial conditions and real-time performance.
△ Less
Submitted 19 April, 2019;
originally announced April 2019.
-
Tolling for Constraint Satisfaction in Markov Decision Process Congestion Games
Authors:
Sarah H. Q. Li,
Yue Yu,
Daniel Calderone,
Lillian Ratliff,
Behcet Acikmese
Abstract:
Markov decision process (MDP) congestion game is an extension of classic congestion games, where a continuous population of selfish agents solves Markov decision processes with congestion: the payoff of a strategy decreases as more population uses it. We draw parallels between key concepts from capacitated congestion games and MDP. In particular, we show that population mass constraints in MDP con…
▽ More
Markov decision process (MDP) congestion game is an extension of classic congestion games, where a continuous population of selfish agents solves Markov decision processes with congestion: the payoff of a strategy decreases as more population uses it. We draw parallels between key concepts from capacitated congestion games and MDP. In particular, we show that population mass constraints in MDP congestion games are equivalent to imposing tolls/incentives on the reward function, which can be utilized by social planners to achieve auxiliary objectives. We demonstrate such methods in a simulated Seattle ride-share model, where tolls and incentives are enforced for two separate objectives: to guarantee minimum driver density in downtown Seattle, and to shift the game equilibrium towards a maximum social output.
△ Less
Submitted 2 March, 2019;
originally announced March 2019.
-
Approximate Multiparametric Mixed-integer Convex Programming
Authors:
Danylo Malyuta,
Behcet Acikmese
Abstract:
We propose an algorithm for generating explicit solutions of multiparametric mixed-integer convex programs to within a given suboptimality tolerance. The algorithm is applicable to a very general class of optimization problems, but is most useful for hybrid model predictive control, where on-line implementation is hampered by the worst-case exponential complexity of mixed-integer solvers. The outp…
▽ More
We propose an algorithm for generating explicit solutions of multiparametric mixed-integer convex programs to within a given suboptimality tolerance. The algorithm is applicable to a very general class of optimization problems, but is most useful for hybrid model predictive control, where on-line implementation is hampered by the worst-case exponential complexity of mixed-integer solvers. The output is a simplicial partition which defines a static map from the current state to a suboptimal solution. The primary theoretical contribution of this paper is to introduce a non-zero optimal cost overlap metric which is necessary and sufficient for convergence. The overlap size is also linked to partition complexity. The algorithm is massively parallelizable and our implementation, which is publicly available, is run on a cluster of several hundred processors. Not only does our solution have a deterministic runtime, simulations show that our approach is faster than on-line optimization by up to three orders of magnitude.
△ Less
Submitted 10 June, 2019; v1 submitted 28 February, 2019;
originally announced February 2019.
-
Partition-based Feasible Integer Solution Pre-computation for Hybrid Model Predictive Control
Authors:
Danylo Malyuta,
Behcet Acikmese,
Martin Cacan,
David S. Bayard
Abstract:
For multiparametric mixed-integer convex programming problems such as those encountered in hybrid model predictive control, we propose an algorithm for generating a feasible partition of a subset of the parameter space. The result is a static map from the current parameter to a suboptimal integer solution such that the remaining convex program is feasible. Convergence is proven with a new insight…
▽ More
For multiparametric mixed-integer convex programming problems such as those encountered in hybrid model predictive control, we propose an algorithm for generating a feasible partition of a subset of the parameter space. The result is a static map from the current parameter to a suboptimal integer solution such that the remaining convex program is feasible. Convergence is proven with a new insight that the overlap among the feasible parameter sets of each integer solution governs the partition complexity. The partition is stored as a tree which makes querying the feasible solution efficient. The algorithm can be used to warm start a mixed integer solver with a real-time guarantee or to provide a reference integer solution in several suboptimal MPC schemes. The algorithm is tested on randomly generated systems with up to six states, demonstrating the effectiveness of the approach.
△ Less
Submitted 28 February, 2019;
originally announced February 2019.
-
Robust Model Predictive Control for Linear Systems with State and Input Dependent Uncertainties
Authors:
Danylo Malyuta,
Behcet Acikmese,
Martin Cacan
Abstract:
This paper presents a computationally efficient robust model predictive control law for discrete linear time invariant systems subject to additive disturbances that may depend on the state and/or input norms. Despite the dependency being non-convex, we are able to capture it exactly for input dependency and approximately for state dependency in at most a second order cone programming problem. The…
▽ More
This paper presents a computationally efficient robust model predictive control law for discrete linear time invariant systems subject to additive disturbances that may depend on the state and/or input norms. Despite the dependency being non-convex, we are able to capture it exactly for input dependency and approximately for state dependency in at most a second order cone programming problem. The formulation has linear complexity in the planning horizon length. The approach is thus amenable to efficient real-time implementation with a guarantee on recursive feasibility and global optimality. Robust position control of a satellite is considered as an illustrative example.
△ Less
Submitted 9 August, 2019; v1 submitted 28 February, 2019;
originally announced February 2019.
-
Stochastic Bregman Parallel Direction Method of Multipliers for Distributed Optimization
Authors:
Yue Yu,
Behçet Açıkmeşe
Abstract:
Bregman parallel direction method of multipliers (BPDMM) efficiently solves distributed optimization over a network, which arises in a wide spectrum of collaborative multi-agent learning applications. In this paper, we generalize BPDMM to stochastic BPDMM, where each iteration only solves local optimization on a randomly selected subset of nodes rather than all the nodes in the network. Such gener…
▽ More
Bregman parallel direction method of multipliers (BPDMM) efficiently solves distributed optimization over a network, which arises in a wide spectrum of collaborative multi-agent learning applications. In this paper, we generalize BPDMM to stochastic BPDMM, where each iteration only solves local optimization on a randomly selected subset of nodes rather than all the nodes in the network. Such generalization reduce the need for computational resources and allows applications to larger scale networks. We establish both the global convergence and the \(O(1/T)\) iteration complexity of stochastic BPDMM. We demonstrate our results via numerical examples.
△ Less
Submitted 1 March, 2019; v1 submitted 25 February, 2019;
originally announced February 2019.
-
Real-Time Quad-Rotor Path Planning Using Convex Optimization and Compound State-Triggered Constraints
Authors:
Michael Szmuk,
Danylo Malyuta,
Taylor P. Reynolds,
Margaret Skye Mceowen,
Behcet Acikmese
Abstract:
The contribution of this paper is the application of compound state-triggered constraints (STCs) to real-time quad-rotor path planning. Originally developed for rocket landing applications, STCs are made up of a trigger condition and a constraint condition that are arranged such that satisfaction of the former implies satisfaction of the latter. Compound STCs go a step further by allowing multiple…
▽ More
The contribution of this paper is the application of compound state-triggered constraints (STCs) to real-time quad-rotor path planning. Originally developed for rocket landing applications, STCs are made up of a trigger condition and a constraint condition that are arranged such that satisfaction of the former implies satisfaction of the latter. Compound STCs go a step further by allowing multiple trigger and constraint conditions to be combined via Boolean "and" or "or" operations. The logical implications embodied by STCs can be formulated using continuous variables, and thus enable the incorporation of discrete decision making into a continuous optimization framework. In this paper, compound STCs are used to solve quad-rotor path planning problems that would typically require the use of computationally expensive mixed-integer programming techniques. Two scenarios are considered: (1) a quad-rotor flying through a hoop, and (2) a pair of quad-rotors carrying a beam-like payload through an obstacle course. Successive convexification is used to solve the resulting non-convex optimization problem. Monte-Carlo simulation results show that our approach can reliably generate trajectories at rates upwards of 3 and 1.5 Hz for the first and second scenarios, respectively.
△ Less
Submitted 25 February, 2019;
originally announced February 2019.
-
Lossless convexification of non-convex optimal control problems with disjoint semi-continuous inputs
Authors:
Danylo Malyuta,
Michael Szmuk,
Behcet Acikmese
Abstract:
This paper presents a convex optimization-based method for finding the globally optimal solutions of a class of mixed-integer non-convex optimal control problems. We consider problems that are non-convex in the input norm, which is a semi-continuous variable that can be zero or lower- and upper-bounded. Using lossless convexification, the non-convex problem is relaxed to a convex problem whose opt…
▽ More
This paper presents a convex optimization-based method for finding the globally optimal solutions of a class of mixed-integer non-convex optimal control problems. We consider problems that are non-convex in the input norm, which is a semi-continuous variable that can be zero or lower- and upper-bounded. Using lossless convexification, the non-convex problem is relaxed to a convex problem whose optimal solution is proved to be optimal almost everywhere for the original problem. The relaxed problem can be solved using second-order cone programming, which is a subclass of convex optimization for which there exist numerically reliable solvers with convergence guarantees and polynomial time complexity. This is the first lossless convexification result for mixed-integer optimization problems. An example of spacecraft docking with a rotating space station corroborates the effectiveness of the approach and features a computation time almost three orders of magnitude shorter than a mixed-integer programming formulation.
△ Less
Submitted 18 November, 2019; v1 submitted 7 February, 2019;
originally announced February 2019.
-
Variable Demand and Multi-commodity Flow in Markovian Network Equilibrium
Authors:
Yue Yu,
Dan Calderone,
Sarah H. Q. Li,
Lillian J. Ratliff,
Behçet Açıkmeşe
Abstract:
Markovian network equilibrium generalizes the classical Wardrop equilibrium in network games. At a Markovian network equilibrium, each player of the game solves a Markov decision process instead of a shortest path problem. We propose two novel extensions of Markovian network equilibrium by considering 1) variable demand, which offers the players a quitting option, and 2) multi-commodity flow, whic…
▽ More
Markovian network equilibrium generalizes the classical Wardrop equilibrium in network games. At a Markovian network equilibrium, each player of the game solves a Markov decision process instead of a shortest path problem. We propose two novel extensions of Markovian network equilibrium by considering 1) variable demand, which offers the players a quitting option, and 2) multi-commodity flow, which allows players to have heterogeneous ending time. We further develop dynamic-programming-based iterative algorithms for the proposed equilibrium problems, together with their arithmetic complexity analysis. Finally, we illustrate our network equilibrium model via a multi-commodity ride-sharing example, and compare the computational efficiency of our algorithms against state-of-the-art optimization software Mosek over extensive numerical experiments.
△ Less
Submitted 18 October, 2021; v1 submitted 24 January, 2019;
originally announced January 2019.
-
Successive Convexification for 6-DoF Powered Descent Guidance with Compound State-Triggered Constraints
Authors:
Michael Szmuk,
Taylor P. Reynolds,
Behcet Acikmese,
Mehran Mesbahi,
John M. Carson III
Abstract:
This paper introduces a continuous formulation for compound state-triggered constraints, which are generalizations of the recently introduced state-triggered constraints. State-triggered constraints are different from ordinary constraints found in optimal control in that they use a state-dependent trigger condition to enable or disable a constraint condition, and can be expressed as continuous fun…
▽ More
This paper introduces a continuous formulation for compound state-triggered constraints, which are generalizations of the recently introduced state-triggered constraints. State-triggered constraints are different from ordinary constraints found in optimal control in that they use a state-dependent trigger condition to enable or disable a constraint condition, and can be expressed as continuous functions that are readily handled by successive convexification. Compound state-triggered constraints go a step further, giving designers the ability to compose trigger and constraint conditions using Boolean and and or operations. Simulations of the 6-degree-of-freedom (DoF) powered descent guidance problem obtained using successive convexification are presented to illustrate the utility of state-triggered and compound state-triggered constraints. The examples employ a velocity-triggered angle of attack constraint to alleviate aerodynamic loads, and a collision avoidance constraint to avoid large geological formations. In particular, the velocity-triggered angle of attack constraint demonstrates the ability of state-triggered constraints to introduce new constraint phases to the solution without resorting to combinatorial techniques.
△ Less
Submitted 8 January, 2019;
originally announced January 2019.
-
Successive Convexification for Real-Time 6-DoF Powered Descent Guidance with State-Triggered Constraints
Authors:
Michael Szmuk,
Taylor P. Reynolds,
Behcet Acikmese
Abstract:
In this paper, we present a real-time successive convexification algorithm for a generalized free-final-time 6-degree-of-freedom powered descent guidance problem. We build on our previous work by introducing the following contributions: (i) a free-ignition-time modification that allows the algorithm to determine the optimal engine ignition time, (ii) a tractable aerodynamics formulation that model…
▽ More
In this paper, we present a real-time successive convexification algorithm for a generalized free-final-time 6-degree-of-freedom powered descent guidance problem. We build on our previous work by introducing the following contributions: (i) a free-ignition-time modification that allows the algorithm to determine the optimal engine ignition time, (ii) a tractable aerodynamics formulation that models both lift and drag, and (iii) a continuous state-triggered constraint formulation that emulates conditionally enforced constraints. In particular, contribution (iii) effectively allows constraints to be enabled or disabled by if-statements conditioned on the solution variables of the parent continuous optimization problem. To the best of our knowledge, this represents a novel formulation in the optimal control literature, and enables a number of interesting applications, including velocity-triggered angle of attack constraints and range-triggered line of sight constraints. Our algorithm converts the resulting generalized powered descent guidance problem from a non-convex free-final-time optimal control problem into a sequence of tractable convex second-order cone programming subproblems. With the aid of virtual control and trust region modifications, these subproblems are solved in succession until convergence is attained. Simulations using a third-party solver demonstrate the real-time capabilities of the proposed algorithm, with a maximum execution time of less than 0.7 seconds over a multitude of problem feature combinations.
△ Less
Submitted 26 November, 2018;
originally announced November 2018.
-
Voronoi Partition-based Scenario Reduction for Fast Sampling-based Stochastic Reachability Computation of LTI Systems
Authors:
Hossein Sartipizadeh,
Abraham P. Vinod,
Behcet Acikmese,
Meeko Oishi
Abstract:
In this paper, we address the stochastic reach-avoid problem for linear systems with additive stochastic uncertainty. We seek to compute the maximum probability that the states remain in a safe set over a finite time horizon and reach a target set at the final time. We employ sampling-based methods and provide a lower bound on the number of scenarios required to guarantee that our estimate provide…
▽ More
In this paper, we address the stochastic reach-avoid problem for linear systems with additive stochastic uncertainty. We seek to compute the maximum probability that the states remain in a safe set over a finite time horizon and reach a target set at the final time. We employ sampling-based methods and provide a lower bound on the number of scenarios required to guarantee that our estimate provides an underapproximation. Due to the probabilistic nature of the sampling-based methods, our underapproximation guarantee is probabilistic, and the proposed lower bound can be used to satisfy a prescribed probabilistic confidence level. To decrease the computational complexity, we propose a Voronoi partition-based to check the reach-avoid constraints at representative partitions (cells), instead of the original scenarios. The state constraints arising from the safe and target sets are tightened appropriately so that the solution provides an underapproximation for the original sampling-based method. We propose a systematic approach for selecting these representative cells and provide the flexibility to trade-off the number of cells needed for accuracy with the computational cost.
△ Less
Submitted 8 November, 2018;
originally announced November 2018.
-
Periodic Event-triggered Control for Incrementally Quadratic Nonlinear Systems
Authors:
Xiangru Xu,
Adam M. Tahir,
Behcet Acikmese
Abstract:
Periodic event-triggered control (PETC) evaluates the triggering rule periodically and is well-suited for implementation on digital platforms. This paper investigates PETC design for nonlinear systems affected by external disturbances under the impulsive system formulation. Sufficient conditions are provided to ensure the input-to-state stability of the resulting closed-loop system for the state f…
▽ More
Periodic event-triggered control (PETC) evaluates the triggering rule periodically and is well-suited for implementation on digital platforms. This paper investigates PETC design for nonlinear systems affected by external disturbances under the impulsive system formulation. Sufficient conditions are provided to ensure the input-to-state stability of the resulting closed-loop system for the state feedback and the observer-based output feedback configurations separately. For each configuration, the sampling period and the triggering functions are provided explicitly. Sufficient conditions in the form of linear matrix inequalities are provided for the PETC design of incrementally quadratic nonlinear systems. Two examples are given to illustrate the effectiveness of the proposed method.
△ Less
Submitted 3 June, 2020; v1 submitted 30 October, 2018;
originally announced October 2018.