-
Equivariant Deep Learning of Mixed-Integer Optimal Control Solutions for Vehicle Decision Making and Motion Planning
Authors:
Rudolf Reiter,
Rien Quirynen,
Moritz Diehl,
Stefano Di Cairano
Abstract:
Mixed-integer quadratic programs (MIQPs) are a versatile way of formulating vehicle decision making and motion planning problems, where the prediction model is a hybrid dynamical system that involves both discrete and continuous decision variables. However, even the most advanced MIQP solvers can hardly account for the challenging requirements of automotive embedded platforms. Thus, we use machine…
▽ More
Mixed-integer quadratic programs (MIQPs) are a versatile way of formulating vehicle decision making and motion planning problems, where the prediction model is a hybrid dynamical system that involves both discrete and continuous decision variables. However, even the most advanced MIQP solvers can hardly account for the challenging requirements of automotive embedded platforms. Thus, we use machine learning to simplify and hence speed up optimization. Our work builds on recent ideas for solving MIQPs in real-time by training a neural network to predict the optimal values of integer variables and solving the remaining problem by online quadratic programming. Specifically, we propose a recurrent permutation equivariant deep set that is particularly suited for imitating MIQPs that involve many obstacles, which is often the major source of computational burden in motion planning problems. Our framework comprises also a feasibility projector that corrects infeasible predictions of integer variables and considerably increases the likelihood of computing a collision-free trajectory. We evaluate the performance, safety and real-time feasibility of decision-making for autonomous driving using the proposed approach on realistic multi-lane traffic scenarios with interactive agents in SUMO simulations.
△ Less
Submitted 13 May, 2024;
originally announced May 2024.
-
Path Planning and Motion Control for Accurate Positioning of Car-like Robots
Authors:
** Dai,
Zejiang Wang,
Yebin Wang,
Rien Quirynen,
Stefano Di Cairano
Abstract:
This paper investigates the planning and control for accurate positioning of car-like robots. We propose a solution that integrates two modules: a motion planner, facilitated by the rapidly-exploring random tree algorithm and continuous-curvature (CC) steering technique, generates a CC trajectory as a reference; and a nonlinear model predictive controller (NMPC) regulates the robot to accurately t…
▽ More
This paper investigates the planning and control for accurate positioning of car-like robots. We propose a solution that integrates two modules: a motion planner, facilitated by the rapidly-exploring random tree algorithm and continuous-curvature (CC) steering technique, generates a CC trajectory as a reference; and a nonlinear model predictive controller (NMPC) regulates the robot to accurately track the reference trajectory. Based on the $μ$-tangency conditions in prior art, we derive explicit existence conditions and develop associated computation methods for a special class of CC paths which not only admit the same driving patterns as Reeds-Shepp paths but also consist of cusp-free clothoid turns. Afterwards, we create an autonomous vehicle parking scenario where the NMPC endeavors to follow the reference trajectory. Feasibility and computational efficiency of the CC steering are validated by numerical simulation. CarSim-Simulink joint simulations statistically verify that with exactly same NMPC, the closed-loop system with CC trajectories as references substantially outperforms the case where Reeds-Shepp trajectories are used as references.
△ Less
Submitted 8 June, 2024; v1 submitted 10 May, 2024;
originally announced May 2024.
-
Progressive Smoothing for Motion Planning in Real-Time NMPC
Authors:
Rudolf Reiter,
Katrin Baumgärtner,
Rien Quirynen,
Moritz Diehl
Abstract:
Nonlinear model predictive control (NMPC) is a popular strategy for solving motion planning problems, including obstacle avoidance constraints, in autonomous driving applications. Non-smooth obstacle shapes, such as rectangles, introduce additional local minima in the underlying optimization problem. Smooth over-approximations, e.g., ellipsoidal shapes, limit the performance due to their conservat…
▽ More
Nonlinear model predictive control (NMPC) is a popular strategy for solving motion planning problems, including obstacle avoidance constraints, in autonomous driving applications. Non-smooth obstacle shapes, such as rectangles, introduce additional local minima in the underlying optimization problem. Smooth over-approximations, e.g., ellipsoidal shapes, limit the performance due to their conservativeness. We propose to vary the smoothness and the related over-approximation by a homotopy. Instead of varying the smoothness in consecutive sequential quadratic programming iterations, we use formulations that decrease the smooth over-approximation from the end towards the beginning of the prediction horizon. Thus, the real-time iterations algorithm is applicable to the proposed NMPC formulation. Different formulations are compared in simulation experiments and shown to successfully improve performance indicators without increasing the computation time.
△ Less
Submitted 4 March, 2024;
originally announced March 2024.
-
Safe multi-agent motion planning under uncertainty for drones using filtered reinforcement learning
Authors:
Sleiman Safaoui,
Abraham P. Vinod,
Ankush Chakrabarty,
Rien Quirynen,
Nobuyuki Yoshikawa,
Stefano Di Cairano
Abstract:
We consider the problem of safe multi-agent motion planning for drones in uncertain, cluttered workspaces. For this problem, we present a tractable motion planner that builds upon the strengths of reinforcement learning and constrained-control-based trajectory planning. First, we use single-agent reinforcement learning to learn motion plans from data that reach the target but may not be collision-…
▽ More
We consider the problem of safe multi-agent motion planning for drones in uncertain, cluttered workspaces. For this problem, we present a tractable motion planner that builds upon the strengths of reinforcement learning and constrained-control-based trajectory planning. First, we use single-agent reinforcement learning to learn motion plans from data that reach the target but may not be collision-free. Next, we use a convex optimization, chance constraints, and set-based methods for constrained control to ensure safety, despite the uncertainty in the workspace, agent motion, and sensing. The proposed approach can handle state and control constraints on the agents, and enforce collision avoidance among themselves and with static obstacles in the workspace with high probability. The proposed approach yields a safe, real-time implementable, multi-agent motion planner that is simpler to train than methods based solely on learning. Numerical simulations and experiments show the efficacy of the approach.
△ Less
Submitted 31 October, 2023;
originally announced November 2023.
-
Real-time Mixed-Integer Quadratic Programming for Vehicle Decision Making and Motion Planning
Authors:
Rien Quirynen,
Sleiman Safaoui,
Stefano Di Cairano
Abstract:
We develop a real-time feasible mixed-integer programming-based decision making (MIP-DM) system for automated driving. Using a linear vehicle model in a road-aligned coordinate frame, the lane change constraints, collision avoidance and traffic rules can be formulated as mixed-integer inequalities, resulting in a mixed-integer quadratic program (MIQP). The proposed MIP-DM simultaneously performs m…
▽ More
We develop a real-time feasible mixed-integer programming-based decision making (MIP-DM) system for automated driving. Using a linear vehicle model in a road-aligned coordinate frame, the lane change constraints, collision avoidance and traffic rules can be formulated as mixed-integer inequalities, resulting in a mixed-integer quadratic program (MIQP). The proposed MIP-DM simultaneously performs maneuver selection and trajectory generation by solving the MIQP at each sampling time instant. While solving MIQPs in real time has been considered intractable in the past, we show that our recently developed solver BB-ASIPM is capable of solving MIP-DM problems on embedded hardware in real time. The performance of this approach is illustrated in simulations in various scenarios including merging points and traffic intersections, and hardware-in-the-loop simulations on dSPACE Scalexio and MicroAutoBox-III. Finally, we present results from hardware experiments on small-scale automated vehicles.
△ Less
Submitted 19 August, 2023;
originally announced August 2023.
-
Friction-Adaptive Stochastic Nonlinear Model Predictive Control for Autonomous Vehicles
Authors:
Sean Vaskov,
Rien Quirynen,
Marcel Menner,
Karl Berntorp
Abstract:
This paper addresses the trajectory-tracking problem under uncertain road-surface conditions for autonomous vehicles. We propose a stochastic nonlinear model predictive controller (SNMPC) that learns a tire--road friction model online using standard automotive-grade sensors. Learning the entire tire--road friction model in real time requires driving in the nonlinear, potentially unstable regime of…
▽ More
This paper addresses the trajectory-tracking problem under uncertain road-surface conditions for autonomous vehicles. We propose a stochastic nonlinear model predictive controller (SNMPC) that learns a tire--road friction model online using standard automotive-grade sensors. Learning the entire tire--road friction model in real time requires driving in the nonlinear, potentially unstable regime of the vehicle dynamics, using a prediction model that may not have fully converged. To handle this, we formulate the tire-friction model learning in a Bayesian framework, and propose two estimators that learn different aspects of the tire--road friction. The estimators output the estimate of the tire-friction model as well as the uncertainty of the estimate, which expresses the confidence in the model for different driving regimes. The SNMPC exploits the uncertainty estimate in its prediction model to take proper action when the uncertainty is large. We validate the approach in an extensive Monte-Carlo study using real vehicle parameters and in CarSim. The results when comparing to various MPC approaches indicate a substantial reduction in constraint violations, as well as a reduction in closed-loop cost. We also demonstrate the real-time feasibility in automotive-grade processors using a dSPACE MicroAutoBox-II rapid prototy** unit, showing a worst-case computation time of roughly 40ms.
△ Less
Submitted 5 May, 2023;
originally announced May 2023.
-
Tailored Presolve Techniques in Branch-and-Bound Method for Fast Mixed-Integer Optimal Control Applications
Authors:
Rien Quirynen,
Stefano Di Cairano
Abstract:
Mixed-integer model predictive control (MI-MPC) can be a powerful tool for modeling hybrid control systems. In case of a linear-quadratic objective in combination with linear or piecewise-linear system dynamics and inequality constraints, MI-MPC needs to solve a mixed-integer quadratic program (MIQP) at each sampling time step. This paper presents a collection of block-sparse presolve techniques t…
▽ More
Mixed-integer model predictive control (MI-MPC) can be a powerful tool for modeling hybrid control systems. In case of a linear-quadratic objective in combination with linear or piecewise-linear system dynamics and inequality constraints, MI-MPC needs to solve a mixed-integer quadratic program (MIQP) at each sampling time step. This paper presents a collection of block-sparse presolve techniques to efficiently remove decision variables, and to remove or tighten inequality constraints, tailored to mixed-integer optimal control problems (MIOCP). In addition, we describe a novel heuristic approach based on an iterative presolve algorithm to compute a feasible but possibly suboptimal MIQP solution. We present benchmarking results for a C code implementation of the proposed BB-ASIPM solver, including a branch-and-bound (B&B) method with the proposed tailored presolve techniques and an active-set based interior point method (ASIPM), compared against multiple state-of-the-art MIQP solvers on a case study of motion planning with obstacle avoidance constraints. Finally, we demonstrate the computational performance of the BB-ASIPM solver on the dSPACE Scalexio real-time embedded hardware using a second case study of stabilization for an underactuated cart-pole with soft contacts.
△ Less
Submitted 10 July, 2023; v1 submitted 22 November, 2022;
originally announced November 2022.
-
PRESAS: Block-Structured Preconditioning of Iterative Solvers within a Primal Active-Set Method for fast MPC
Authors:
Rien Quirynen,
Stefano Di Cairano
Abstract:
Model predictive control (MPC) for linear dynamical systems requires solving an optimal control structured quadratic program (QP) at each sampling instant. This paper proposes a primal active-set strategy (PRESAS) for the efficient solution of such block-sparse QPs, based on a preconditioned iterative solver to compute the search direction in each iteration. Rank-one factorization updates of the p…
▽ More
Model predictive control (MPC) for linear dynamical systems requires solving an optimal control structured quadratic program (QP) at each sampling instant. This paper proposes a primal active-set strategy (PRESAS) for the efficient solution of such block-sparse QPs, based on a preconditioned iterative solver to compute the search direction in each iteration. Rank-one factorization updates of the preconditioner result in a per-iteration computational complexity of $\mathcal{O}(N m^2)$, where $m$ denotes the number of state and control variables and $N$ the number of control intervals. Three different block-structured preconditioning techniques are presented and their numerical properties are studied further. In addition, an augmented Lagrangian based implementation is proposed to avoid a costly initialization procedure to find a primal feasible starting point. Based on a standalone C code implementation, we illustrate the computational performance of PRESAS against current state of the art QP solvers for multiple linear and nonlinear MPC case studies. We also show that the solver is real-time feasible on a dSPACE MicroAutoBox-II rapid prototy** unit for vehicle control applications, and numerical reliability is illustrated based on experimental results from a testbench of small-scale autonomous vehicles.
△ Less
Submitted 12 July, 2020; v1 submitted 4 December, 2019;
originally announced December 2019.
-
Approximate Dynamic Programming For Linear Systems with State and Input Constraints
Authors:
Ankush Chakrabarty,
Rien Quirynen,
Claus Danielson,
Weinan Gao
Abstract:
Enforcing state and input constraints during reinforcement learning (RL) in continuous state spaces is an open but crucial problem which remains a roadblock to using RL in safety-critical applications. This paper leverages invariant sets to update control policies within an approximate dynamic programming (ADP) framework that guarantees constraint satisfaction for all time and converges to the opt…
▽ More
Enforcing state and input constraints during reinforcement learning (RL) in continuous state spaces is an open but crucial problem which remains a roadblock to using RL in safety-critical applications. This paper leverages invariant sets to update control policies within an approximate dynamic programming (ADP) framework that guarantees constraint satisfaction for all time and converges to the optimal policy (in a linear quadratic regulator sense) asymptotically. An algorithm for implementing the proposed constrained ADP approach in a data-driven manner is provided. The potential of this formalism is demonstrated via numerical examples.
△ Less
Submitted 26 June, 2019;
originally announced June 2019.
-
A Structure Exploiting Branch-and-Bound Algorithm for Mixed-Integer Model Predictive Control
Authors:
Pedro Hespanhol,
Rien Quirynen,
Stefano Di Cairano
Abstract:
Mixed-integer model predictive control (MI-MPC) requires the solution of a mixed-integer quadratic program (MIQP) at each sampling instant under strict timing constraints, where part of the state and control variables can only assume a discrete set of values. Several applications in automotive, aerospace and hybrid systems are practical examples of how such discrete-valued variables arise. We util…
▽ More
Mixed-integer model predictive control (MI-MPC) requires the solution of a mixed-integer quadratic program (MIQP) at each sampling instant under strict timing constraints, where part of the state and control variables can only assume a discrete set of values. Several applications in automotive, aerospace and hybrid systems are practical examples of how such discrete-valued variables arise. We utilize the sequential nature and the problem structure of MI-MPC in order to provide a branch-and-bound algorithm that can exploit not only the block-sparse optimal control structure of the problem but that can also be warm started by propagating information from branch-and-bound trees and solution paths at previous time steps. We illustrate the computational performance of the proposed algorithm and compare against current state-of-the-art solvers for multiple MPC case studies, based on a preliminary implementation in MATLAB and C code.
△ Less
Submitted 21 March, 2019;
originally announced March 2019.
-
Adjoint-based SQP Method with Block-wise quasi-Newton Jacobian Updates for Nonlinear Optimal Control
Authors:
Pedro Hespanhol,
Rien Quirynen
Abstract:
Nonlinear model predictive control~(NMPC) generally requires the solution of a non-convex optimization problem at each sampling instant under strict timing constraints, based on a set of differential equations that can often be stiff and/or that may include implicit algebraic equations. This paper provides a local convergence analysis for the recently proposed adjoint-based sequential quadratic pr…
▽ More
Nonlinear model predictive control~(NMPC) generally requires the solution of a non-convex optimization problem at each sampling instant under strict timing constraints, based on a set of differential equations that can often be stiff and/or that may include implicit algebraic equations. This paper provides a local convergence analysis for the recently proposed adjoint-based sequential quadratic programming~(SQP) algorithm that is based on a block-structured variant of the two-sided rank-one~(TR1) quasi-Newton update formula to efficiently compute Jacobian matrix approximations in a sparsity preserving fashion. A particularly efficient algorithm implementation is proposed in case an implicit integration scheme is used for discretization of the optimal control problem, in which matrix factorization and matrix-matrix operations can be avoided entirely. The convergence analysis results as well as the computational performance of the proposed optimization algorithm are illustrated for two simulation case studies of nonlinear MPC.
△ Less
Submitted 20 March, 2019;
originally announced March 2019.
