Stable MPC with maximal terminal sets and quadratic terminal costs

Abstract

This paper develops a technique for computing a quadratic terminal cost for linear model predictive controllers that is valid for all states in the maximal control invariant set. This maximizes the set of recursively feasible states for the controller, ensures asymptotic stability using standard proofs, and allows for easy tuning of the controller in linear operation.

I Introduction

With its ability to optimize the closed-loop system under constraints on states and control signals, model predictive control (MPC) has emerged as the preferred control technique for many advanced applications. Although the underlying ideas can be traced back to the 1960’s [1] and industrial applications appeared already in the 70’s [2], a more complete theoretical understanding of MPC emerged first around the millenium [3]. Advances in optimization algorithms [4, 5, 6] now allow model-predictive controllers to run on cheap embedded hardware, and a wealth of applications have demonstrated the practical value of MPC [7]. With a number of excellent textbooks on MPC [8, 9, 10], it is fair to say that the field of model predictive control for linear constrained systems is rather complete. However, there are still a few important questions that have not been fully resolved. This paper aims to address one of them.

Specifically, we are concerned with the problem of achieving the maximal operating region of a linear MPC with quadratic stage and terminal costs, while guaranteeing asymptotic stability of the closed loop.

The standard stability theorems for linear MPC require that the terminal set is matched with an appropriate terminal cost. Textbooks on MPC, such as [8, 9, 10], only present partial and suboptimal solutions to this problem. The easiest approach is to require that the terminal state reaches the origin, in which case we do not need a terminal penalty. A less restrictive solution is to use the control invariant set induced by some linear state feedback controller as terminal set. A matching terminal cost can then be computed by solving a Lyapunov equation. However, both these approaches result in an unnecessarily small feasible region for the associated MPC controller. To increase their operating regions, one needs to use long prediction horizons. A more economical alternative is to use the maximal control invariant set as a terminal set. Although techniques for computing maximal control invariant sets are well-developed [11], it is not obvious how to match these with an appropriate terminal cost. This paper demonstrates how one can compute a quadratic terminal cost that is valid on the maximal control invariant set, hence allowing the largest possible operating region for the associated MPC controller while guaranteeing asymptotic stability of the closed-loop.

The challenge of determining terminal sets and terminal costs that enlarge the operating region for MPC controllers has been approached by several authors. For example, [12] used the invariant set of a saturated linear controller (instead of linear) as terminal set, [13] computed a piecewise affine terminal cost for hybrid MPC controllers, and [14] and [15] suggested to use terminal costs derived from the Minkowski (control) Lyapunov function induced by the terminal set. Although these latter two approaches enable the MPC controller to operate from all states in the interior of the maximal control invariant set [15], they have a few disadvantages. First, polyhedral Lyapunov functions can have many segments, and therefore be complex to represent and costly to use in the MPC computations. This issue is partially overcome in [14], where the same inequalities are used to define the terminal set and the terminal cost. The second, and more important, disadvantage of the approaches in [15, 14] is that the computed terminal cost only depends on the system dynamics and constraints, but is independent of the stage cost. This necessitates a different convergence proof and makes it hard to affect the properties of the controller when it regulates the system without activating any constraints.

This paper is organized as follows. Section II reviews the linear MPC set-up and the standard stability theorems. Our approach to terminal cost computation is developed in Section III, and evaluated numerically in Section IV. Finally, Section V concludes the paper.

Notation.

Our notation is largely standard. For a set $\mathcal{C}\subseteq\mathbb{R}^{n}$, we let $\partial\mathcal{C}$ denote its boundary, and define the scaled set $\lambda\mathcal{C}=\{x=\lambda y\;|\;y\in\mathcal{C}\}$. We say that $\mathcal{C}$ is a C-set if it is convex, compact, and includes the origin in its interior.

A simplex ${\mathcal{S}}\subset\mathbb{R}^{n}$ is the convex hull of $n+1$ affinely independent vectors $v_{i}\in{\mathbb{R}}^{n}$ which we refer to as vertices. A triangulation of a set ${\mathcal{C}}\in{\mathbb{R}}^{n}$ is a subdivision of the set into a finite number of $n$-dimensional simplices such that any two simplices intersect in a common face (a simplex of any lower dimension) or not at all. In a boundary triangulation, each simplex in the triangulation has one vertex at the origin.

II Linear Model Predictive Control

Consider the discrete-time linear system

$$x_{t+1} =Ax_{t}+Bu_{t},\quad t\geq 0$$ (1)

with linear constraints on the states and controls

$$x_{t}\in\mathcal{X},\qquad u_{t}\in\mathcal{U}.$$ (2)

Both $\mathcal{X}$ and $\mathcal{U}$ are assumed to be polyhedral C-sets.

It is useful to view linear MPC as an approximate solution to the infinite-horizon control problem

$$\begin{array}[c]{ll}\mbox{minimize}&\sum_{t=0}^{\infty}x_{t}^{\top}Qx_{t}+u_{t}^{\top}Ru_{t}\\ \mbox{subject to}&x_{t+1}=Ax_{t}+Bu_{t}\\ &x_{t}\in\mathcal{X},\;u_{t}\in\mathcal{U}\end{array}$$ (6)

At every sampling instant $t$, the MPC control law measures the state $x_{t}$, solves the planning problem

$$\begin{array}[c]{ll}\underset{\{\hat{x}_{k}\},\{\hat{u}_{k}\}}{\mbox{minimize}}&\sum_{k=0}^{T-1}\hat{x}_{k}^{\top}Q\hat{x}_{k}+\hat{u}_{k}^{\top}R\hat{u}_{k}+\hat{v}(\hat{x}_{T})\\ \mbox{subject to}&\hat{x}_{k+1}=A\hat{x}_{k}+B\hat{u}_{k},\;\;k=0,\dots,T-1\\ &\hat{x}_{k}\in\mathcal{X},\;\hat{u}_{k}\in\mathcal{U},\;\;\;\;\;\;\;k=0,\dots,T-1\\ &\hat{x}_{T}\in\mathcal{X}_{T}\\ &\hat{x}_{0}=x_{t}\end{array}$$ (12)

for the predicted optimal controls $\{\hat{u}_{k}^{\star}\}$ and predicted state trajectory $\{\hat{x}_{k}^{\star}\}$, and applies the control

$$u_{t} =\hat{u}_{0}^{\star}.$$ (13)

In the planning problem, $\hat{v}(\hat{x}_{T})=\hat{x}_{T}^{\top}Q_{T}\hat{x}_{T}$ serves as an approximation of the infinite-horizon cost-to-go of (6) from state $\hat{x}_{T}$ at the end of the planning horizon, while the terminal set $\mathcal{X}_{T}$ ensures that the cost-to-go approximation is valid. The standard stability proof for linear MPC imposes the following requirements [8].

Theorem 1

Consider the system (1) with constraints (2) under the the RHC control law (12)-(13). Assume that $(A,B)$ is a reachable pair and let $\mathcal{X}_{0}$ be the set of states $x_{t}$ for which the planning problem (12) admits a feasible solution. If

• $Q\succeq 0$ with $(Q^{1/2},A)$ detectable, $R\succ 0$, $Q_{T}\succ 0$

• The sets $\mathcal{X}$, $\mathcal{U}$ and $\mathcal{X}_{T}\subseteq X$ are C-sets

• For every $x\in\mathcal{X}_{T}$, there exists a $u\in\mathcal{U}$ such that

  $$Ax+Bu\in\mathcal{X}_{T},\,\mbox{ and }$$

  $$\hat{v}(Ax+Bu)-\hat{v}(x)+x^{\top}Qx+u^{\top}Ru\leq 0$$

Then, every trajectory $\{x_{t}\}$ of the closed-loop system remains in $\mathcal{X}_{0}$ and $\lim_{t\rightarrow\infty}x_{t}=0$.

While the first two conditions of the theorem are straightforward to verify, the last one is more involved. In essence, it requires that ${\mathcal{X}}_{T}$ is control invariant and that $\hat{v}(\cdot)$ is an upper bound on the true cost-to-go of (6) for all $x\in\mathcal{X}_{T}$. MPC textbooks, such as [10, 9, 8], typically suggest two approaches to this problem. The first one is to set $\mathcal{X}_{T}=\{0\}$ (forcing the state at the end of the planning horizon to zero), for which we can set $\hat{v}_{T}(x)\equiv 0$. The second one is to choose a terminal set that is invariant under some linear state feedback $u_{t}=-Lx_{t}$. Specifically, one uses the largest invariant set of $x_{t+1}=(A-BL)x_{t}$ contained in the set $\{x\;|\;x\in\mathcal{X}\mbox{ and }-Lx\in\mathcal{U}\}$. A valid quadratic upper bound of the cost-to-go is then $\hat{v}(x)=x^{\top}Q_{T}x$ where $Q_{T}$ solves the Lyapunov equation

$$Q_{T} =Q+L^{\top}RL+(A-BL)^{\top}Q_{T}(A-BL)$$

A particularly convenient choice is to use the infinite-horizon optimal LQR controller $u_{t}=-L_{\infty}x_{t}$ for the cost defined by $Q$ and $R$. In this way, the Lyapunov equation above is satisfied for the solution $P_{\infty}$ to the corresponding discrete-time algebraic Riccati equation

$$P_{\infty} =Q+A^{\top}P_{\infty}A-L_{\infty}^{\top}(B^{\top}P_{\infty}B+R)L_{\infty}$$

where $L_{\infty}=(B^{\top}P_{\infty}B+R)^{-1}B^{\top}P_{\infty}A$. However, the terminal set computed in this way is not necessarily large.

Since $x_{T}$ must belong to $\mathcal{X}_{T}$, a smaller terminal set leads to a smaller set of (recursively) feasible states, and hence to a smaller operating region of the MPC controller. The operating region increases with the planning horizon $T$, but with a small terminal set one typically need a long horizon to be able to operate from all states in the maximal control invariant set [16]. If one is able to use the maximal control invariant set as terminal set, on the other hand, the MPC controller will have the largest possible operating region already with a horizon of one. To illustrate the relationship between terminal set, prediction horizon, and operating regime of linear MPC, we consider the following example from [9].

Example 1

Consider the second-order system

$$x_{t+1} =\begin{pmatrix}1.1&2\\ 0&0.95\end{pmatrix}x_{t}+\begin{pmatrix}0\\ 0.0787\end{pmatrix}u_{t}$$

$$y_{t} =\begin{pmatrix}-1&1\end{pmatrix}x_{t}$$

under the MPC control (12)-(13) with

$$Q =C^{\top}C,\;R=1$$

$$\mathcal{X} =\{x\;|\;\|x\|_{\infty}\leq 8\}$$

$$\mathcal{U} =\{u\;|\;|u|\leq 1\}$$

Figure 1 shows $\mathcal{X}_{0}$ for different horizon lengths when the terminal set is taken to be the maximal invariant set of the infinite-horizon LQR-optimal control law. Note how the operating region of the MPC controller increases with increasing horizon length, and that even with a horizon of 24 samples, the MPC controller can only operate in a subset of the maximal control invariant set.

Figure 1: Operating region $\mathcal{X}_{0}$ for MPC controller depending on the horizon length $T$. A small terminal set may force us to use an unnecessarily long horizon.

In the next section, we will develop a technique for computing a quadratic terminal cost so that one can use the maximal control invariant set as terminal set and still guarantee stability using Theorem 1.

III Computing a quadratic terminal cost valid on the maximal invariant set

Our procedure contains three key steps: we first determine the maximal invariant set, then recover an explicit feedback policy that renders the set invariant, and finally compute an upper bound on the infinite-horizon cost for this control law.

III-A Maximal control invariant and $\lambda$-contractive sets

Definition 1

A set $\mathcal{C}\subseteq\mathbb{R}^{n}$ is control invariant for the system (1) under the constraints (2) if $\mathcal{C}\subseteq\mathcal{X}$ and

$$x \in{\mathcal{C}}\Rightarrow\exists u\in\mathcal{U}\mbox{ such that }Ax+Bu\in\mathcal{C}.$$

Furthermore, we say that $\mathcal{C}_{\infty}$ is maximal control invariant if it is control invariant and contains all control invariant sets for the system (1) under the constraints (2).

Control invariance by itself does not imply that states in ${\mathcal{C}}_{\infty}$ can be driven to zero. For example, if $A=I$ and $B=0$, then ${\mathcal{C}}_{\infty}={\mathcal{X}}$ but $x_{k}=x_{0}$ for all $k\geq 0$. Such situations can be detected and avoided by requiring that the terminal set is contractive in the following sense.

Definition 2

Let $\lambda\in(0,1]$. A set ${\mathcal{C}}\subseteq{\mathcal{X}}$ is called $\lambda$-contractive (1) under the constraints (2), if for every $x\in{\mathcal{C}}$ there exists $u\in{\mathcal{U}}$ such that $Ax+Bu\in\lambda{\mathcal{C}}$. For a given $\lambda$, the maximal $\lambda$-contractive set for (1) under the constraints (2), denoted ${\mathcal{C}}_{\infty}^{\lambda}$, is the union of all $\lambda$-contractive sets.

Note that the maximal $1$-contractive set is identical to the maximal control invariant set. There are several techniques for computing the maximal control invariant set (e.g., [8, Algorithm 10.2]), but $\lambda$-contractive sets are typically computed using the following approach: define the predecessor set

$$\mathcal{P}(\Omega) =\left\{x\in\mathbb{R}^{n}\;|\;Ax+Bu\in\Omega\mbox{ for some }u\in\mathcal{U}\right\}$$

and then execute the following algorithm [8, Algorithm 10.2]

1. Set $\Omega_{0}=\mathcal{X}$, $k=0$.

2. Let $k=k+1$ and $\Omega_{k}=\mathcal{P}(\lambda\Omega_{k-1})\cap{\mathcal{X}}$.

3. If $\Omega_{k}=\Omega_{k-1}$, return $C^{\lambda}_{\infty}=\Omega_{k}$ else goto 2.

For a linear system with C-set constraints on the states and controls, the maximal control invariant set is also a C-set. However, even if $\mathcal{X}$ and $\mathcal{U}$ are polyhedral, the maximal invariant set may not be polyhedral and the algorithm above will not terminate. Conditions for when the algorithm is guaranteed to terminate in a finite number of steps, along with bounds for the number of iterations required, can be found in [17]. When we consider $\lambda$-contractive sets with $\lambda\in[0,1)$, the situation becomes slightly more complicated since the constraints may limit our ability to contract; see [18] and  [19] for a careful convergence analysis of the algorithm.

III-B Recovering the implicit feedback policy

The algorithm described above allows us to compute the maximal $\lambda$-contractive set $\mathcal{C}_{\infty}^{\lambda}$ of a linear system with C-set constraints on states and controls. From Definition 2, we therefore know that there exists an admissible control function $u(x):\mathcal{C}_{\infty}^{\lambda}\mapsto\mathcal{U}$ that steers states in $\mathcal{C}_{\infty}^{\lambda}$ into $\lambda\mathcal{C}_{\infty}^{\lambda}$. However, this control function is only implicit in the calculations. We will now demonstrate how we can recover $u(x)$ as a continuous and piecewise linear control law.

The next result, which is a slight generalization of the conditions proposed by Gutman and Cwikel [20] (see also [11, §4]) demonstrates how we can determine control signals to apply at each vertex of a C-set ${\mathcal{C}}$ to steer the state into $\lambda{\mathcal{C}}$.

Theorem 2

The C-set $\mathcal{C}\subseteq\mathcal{X}$ with vertices $v_{i}$, $i=1,2,\dots,s$ is $\lambda$-contractive for the discrete-time linear system (1) under the constraints (2) if and only if there exists $\lambda_{i}\in[0,\lambda]$, $p_{ij}\in[0,1]$ and $u_{i}\in\mathbb{R}^{m}$ that satisfy

$$\left\{\begin{array}[c]{rcl}Av_{i}+Bu_{i}&=&\sum_{j=1}^{s}p_{ij}v_{j}\\ \sum_{j=1}^{s}p_{ij}&\leq&\lambda_{i}\\ u_{i}&\in&\mathcal{U}\end{array}\right.$$ (17)

for every $i=1,2,\dots,s$.

Together, the equality constraint and the conditions that $p_{ij}\geq 0$ and $\sum_{j=1}^{s}p_{ij}\leq\lambda_{i}\leq\lambda$ imply that for each vertex $v_{i}$ of $\mathcal{C}$, there is an admissible control action $u_{i}$ so that the next state belongs to $\lambda_{i}{\mathcal{C}}$ and hence to $\lambda\mathcal{C}$. Since $\mathcal{U}$ is a C-set, $u\in\mathcal{U}$ can be expressed as a system of linear inequalities or equalities. This means that the conditions in (17) constitute a linear programming feasibility problem.

Remark 1

Although we could fix $\lambda_{i}=\lambda$ for all $i$, it can be useful to minimize $\sum_{i}\lambda_{i}$ to encourage the control to drive the state closer to the origin or to minimize the difference between the $u_{i}$ and a linear control law, e.g. $-L_{\infty}v_{i}$.

As suggested by Gutman and Cwikel [20], it is possible to transform the vertex controls computed in Theorem 2 into a continuous feedback policy that is valid for all $x$. To this end, consider a subdivision of ${\mathcal{C}}$ into simplices $\{\mathcal{S}_{k}\}_{k=1}^{N}$, such that each simplex $\mathcal{S}_{k}$ has one vertex at the origin and the remaining $n$ ones at extreme points of $\mathcal{C}$. Such a boundary triangulation of a C-set is readily determined using standard convex hull algorithms [21, Section 3]. Let $v_{i}$ for $i=1,2,\dots s$ be the extreme points of $\mathcal{C}$, $v_{0}=0$ and $\mathcal{I}_{k}=\{i\;|\;v_{i}\in\mathcal{S}_{k}\}$ be the index set for the vertices of $\mathcal{S}_{k}$.

Since ${\mathcal{S}}_{k}$ is a simplex, any $x\in\mathcal{S}_{k}$ can be written as

$$x =\sum_{i\in{\mathcal{I}}_{k}}p_{i}(x)v_{i}$$ (18)

where $p_{i}(x)\geq 0$ and $\sum_{i\in{\mathcal{I}}_{k}}p_{i}(x)=1$. Moreover, the vertices $v_{i}$ are affinely independent, so if we define $V_{k}\in\mathbb{R}^{n\times n}$ as the matrix whose columns are the non-zero vertices of ${\mathcal{S}}_{k}$ ordered in increasing vertex index $i$, the matrix

$$\begin{pmatrix}0&V_{k}\\ 1&\mathbf{1}^{\top}\end{pmatrix}$$

has full rank. This implies that $V_{k}$ also has full rank.

Next, define $u_{0}=0$ so that $Av_{0}+Bu_{0}=0$. A feasible solution to the conditions in Theorem 2 then implies that

$$Av_{i}+Bu_{i}\in{\mathcal{C}}\qquad\mbox{ for all }i=0,1,\dots,s.$$

Since $\mathcal{C}$ is convex, any convex combination of $Av_{i}+Bu_{i}$ also belongs to ${\mathcal{C}}$. In particular, with $p_{i}(x)$ defined above,

$$\sum_{i\in\mathcal{I}_{k}}p_{i}(x)(Av_{i}+Bu_{i})=Ax+B\sum_{i\in\mathcal{I}_{k}}p_{i}(x)u_{i}\in{\mathcal{C}}$$

Similarly, since each $u_{i}\in{\mathcal{U}}$, and the set ${\mathcal{U}}$ is convex,

$$u(x) =\sum_{i\in\mathcal{I}_{k}}p_{i}(x)u_{i}\,\in{\mathcal{U}}.$$

Thus, $u(x)$ is admissible and renders ${\mathcal{C}}$ $\lambda$-contractive under $x_{k+1}=Ax_{k}+Bu(x_{k})$. To derive an explicit expression for the control policy, let $U_{k}\in\mathbb{R}^{m\times n}$ be the matrix whose columns are $u_{i}$ for $i\in{\mathcal{I}}_{k}\backslash 0$ ordered in increasing vertex index $i$ and $p(x)\in\mathbb{R}^{n}$ as the vector of $p_{i}(x)$ for $i\in{\mathcal{I}}_{k}\backslash 0$ ordered in the same way. Then, for $x\in{\mathcal{S}}_{k}$,

$$x =\sum_{i\in{\mathcal{I}}_{k}}v_{i}p_{i}(x)=V_{k}p(x)\Rightarrow p(x)=V_{k}^{-1}x$$

and

$$u(x) =\sum_{i\in{\mathcal{I}}_{k}}u_{i}p_{i}(x)=U_{k}p(x)=U_{k}V_{k}^{-1}x$$

In other words, the feedback policy

$$u(x) =-L_{k}x=U_{k}V_{k}^{-1}x\qquad x\in{\mathcal{S}}_{k}$$ (19)

renders $\mathcal{C}$ $\lambda$-contractive. It is continuous and piecewise linear, and easy to extract from a solution to (17).

Example 2

Figure 2 shows the triangulation of the maximal control invariant set for the system considered in Example 1. The triangulation of $\mathcal{C}_{\infty}$ is shown in gray while the associated piecewise linear control law is shown in blue.

Figure 2: Triangulation of $\mathcal{C}_{\infty}$ (gray) and the associated piecewise linear control law (blue) for the system in Example 1.

III-C A quadratic upper bound on the cost-to-go

Now that we have developed a procedure to recover an explicit feedback policy that renders the desired terminal set control invariant, we can proceed to compute an upper bound on the associated cost-to-go. In particular, consider $\hat{v}(x)=x^{\top}Px$. Then condition (d) of Theorem 1 reads

$$x^{\top}A_{k}^{\top}PA_{k}x-x^{\top}Px+x^{\top}Q_{k}x \leq 0\quad\forall x\in{\mathcal{S}}_{k}$$

where $A_{k}=(A-BL_{k})$ and $Q_{k}=Q+L_{k}^{\top}RL_{k}$. There are $N$ such inequalities, one for each simplex in the boundary triangulation of the terminal set. As described in [22, 23] these conditions can be verified using semi-definite programming, where the condition that $x\in{\mathcal{S}}_{k}$ is encoded using the S-procedure. This leads to the next result.

Theorem 3

Let ${\mathcal{C}}$ be control invariant for (1) under constraints (2), and let $\{\mathcal{S}_{k}\}_{k=1}^{N}$ be a boundary triangulation of $\mathcal{C}$. Consider the control policy $u_{k}=u(x_{k})$ defined in (19) and let $A_{k}=A-BL_{k}$ and $Q_{k}=Q+L_{k}^{\top}RL_{k}$. If the following semi-definite program

$$\begin{array}[c]{lll}\mbox{minimize}&\mathrm{Tr}(P)&\\ \mbox{subject to}&A_{k}^{\top}PA_{k}-P+Q_{k}+V_{k}^{-\top}W_{k}V_{k}^{-1}\preceq 0&\forall k\\ &W_{k}=W_{k}^{\top}\geq 0&\forall k\\ &\;P\succ 0&\end{array}$$

in variables $P$ and $W_{k}$ is feasible, then condition (d) in Theorem 1 is satisfied with $\mathcal{X}_{T}={\mathcal{C}}$ and $\hat{v}(x)=x^{\top}Px$.

Remark 2

The first matrix inequality uses the S-procedure to reduce conservatism. Since $W_{k}$ is elementwise non-negative, and $V_{k}^{-1}x\geq 0$ when $x\in\mathcal{S}_{k}$, the term $x^{\top}V_{k}^{-\top}W_{k}V_{k}^{-1}x\geq 0$ for $x\in\mathcal{S}_{k}$. However, $V_{k}^{-\top}W_{k}V_{k}^{-1}$ is not necessarily a positive semi-definite matrix and $x^{\top}V_{k}^{-\top}W_{k}V_{k}^{-1}x$ may be negative when $x\not\in\mathcal{S}_{k}$. This makes the linear matrix inequality easier to satisfy than $A_{k}^{\top}PA_{k}-P+Q_{k}\preceq 0$. The S-procedure is, in general, only a sufficient condition for checking positivity of a quadratic form on a polyhedron, but since $\mathcal{S}_{k}$ is a simplex, this use of the S-procedure is loss-less up until dimension $n=4$ [23].

Remark 3

As discussed in § III-B, the matrices $V_{k}$ are invertible per definition. Nevertheless, one can multiply the linear matrix inequalities

$$A_{k}^{\top}PA_{k}-P+Q_{k}+V_{k}^{-\top}W_{k}V_{k}^{-1}\preceq 0$$

by $V_{k}^{\top}$ from the left and $V_{k}$ from the right without affecting the solution set. By accounting for the structure of $A_{k}$, $Q_{k}$ and $L_{k}$, the resulting linear matrix inequality reads

$$(AV_{k}+BU_{k})^{\top}P(AV_{k}+BU_{k})+V_{k}^{\top}(-P+Q)V_{k}+$$

$$U_{k}^{\top}RU_{k}+W_{k}\preceq 0$$

This circumvents the need for recovering the feedback gains $L_{i}$ and avoids any inversions of the $V_{k}$ matrices.

Remark 4

As a final remark, we have used the objective $\mbox{Tr}(P)$, but there are many other options. One such example would be to minimize $\sum_{i}v_{i}^{T}Pv_{i}=\sum_{i}\mbox{Tr}\,P(v_{i}v_{i}^{\top})$, or to minimize $\|P-P_{\infty}\|_{F}^{2}$. Both are easily expressed as SDPs.

III-D The complete algorithm

We now have all the required components for computing a quadratic upper bound to the cost-to-go that is valid in the maximal control invariant set (and, more generally, in the maximal $\lambda$-contractive set). Specifically, we propose to use the following algorithm with $\lambda=1$.

1. Compute $\mathcal{C}^{\lambda}_{\infty}$ and determine its boundary triangulation (including the vertices) using a convex hull algorithm.

2. Recover admissible vertex controls $\{u_{i}\}$ that render ${\mathcal{C}}^{\lambda}_{\infty}$ $\lambda$-contractive by solving the linear program (17).

3. Convert $\{u_{i}\}$ into feedback gains $\{L_{i}\}$ such that

   $$u_{t} =-L_{k}x_{t}\quad\mbox{ for }x_{t}\in\mathcal{S}_{k},\;k=1,\dots,N$$

   using the procedure in Section III-B.

4. Solve the semidefinite program in Theorem 3 for $P$.

If the algorithm returns a feasible solution $P$, then the receding-horizon control law (12)-(13) with $\mathcal{X}_{T}={\mathcal{C}}_{\infty}^{\lambda}$ and $Q_{T}=P$ renders the closed-loop control of the linear system (1) asymptotically stable according to Theorem 1.

Remark 5

Although the proposed procedure is numerical, the new steps that we have introduced rely on convex optimization, are fast and reliable to execute, and do not introduce any conservatism for systems of order $n\leq 4$. Still, since control invariance does not imply asymptotic stabilizability (cf. the discussion in § III-A), we cannot guarantee that the procedure will always find a quadratic upper bound on the infinite-horizon cost-to-go that is valid in the maximal control invariant set. However, with $\lambda<1$, the implicit feedback policy is guaranteed to make the closed loop asymptotically stable and should, in principle, admit a quadratic upper bound on the infinite-horizon cost within the associated maximal $\lambda$-contractive set.

III-E Horizon length and linear performance

With the computed terminal cost, the MPC controller will result in an asymptotically stable closed-loop for all horizon lengths. Close to the origin, it will realize the linear state-feedback that is optimal for a $T$-horizon linear-quadratic control problem with stage cost defined by $Q$ and $R$, and terminal cost $x^{\top}Px$. The fact that the optimal unconstrained control is linear and easy to compute is a distinct advantage over approaches that use more complex terminal costs. It allows us to analyze the frequency domain properties of the controller near the origin, and understand how the horizon length affects the control performance. In particular, if the computed terminal cost matrix $P$ is very different from $P_{\infty}$ and we use a small horizon length, the linear operation of the MPC controller can be far from the infinite-horizon LQR controller. The difference diminishes with increasing horizon length, and is easy to quantify using the Riccati recursion for the associated finite-horizon LQR problem.

IV Numerical examples

In this section, we will illustrate various aspects of our framework by examples.

IV-A Horizon length and performance

Let us first return to the system in Example 1. The maximal control invariant set ${\mathcal{C}}_{\infty}$ has $s=39$ vertices our procedure finds the terminal cost matrix

$$Q_{T} =\begin{pmatrix}38.6&343.1\\ 343.1&4178.5\end{pmatrix}$$

If we compare this with the Riccati solution

$$P_{\infty} =\begin{pmatrix}8.0&26.1\\ 26.1&145.3\end{pmatrix}$$

we note an increased incentive for the MPC controller to bring the terminal state closer to rest when we insist to operate within the maximal operating range. To validate our results further, we let $x_{0}=\begin{pmatrix}7.99&-1.27\end{pmatrix}$, which is just inside the lower right corner of the maximal control invariant set in Figure 1. Recall that the MPC controller that uses the invariant set of the LQR controller and the terminal cost defined by $P_{\infty}$ above requires a planning horizon of at least $26$ samples to operate from this initial value. In contrast, the simulations shown in Figure 3 show how the MPC controller drives the system to rest for all horizon lengths.

Figure 4 shows the partitions of the corresponding explicit MPC control laws. Here, a lighter color indicates that the local behavior in the region is closer the infinite-horizon LQR controller, with white for perfect agreement and black for saturation. We notice how the linear behavior around $x=0$ becomes increasingly close to the infinite-horizon LQR controller as the prediction horizon increases.

Figure 3: The MPC controller with the proposed terminal cost ensures a feasible and stable operation for all horizon lengths.

Figure 4: Partitions of explicit MPC policies for different $T$. Light colors indicate a local behavior close to $u_{t}=-L_{\infty}x_{t}$.

IV-B Stability in absence of terminal set contractivity

Next, we consider the constrained linear system given by

$$A =\begin{pmatrix}0&1\\ -1&0\end{pmatrix},\qquad B=\begin{pmatrix}0\\ 1\end{pmatrix}$$

and $\mathcal{X}=\{x\;|\;\|x\|_{\infty}\leq 5\}$ and $\mathcal{U}=\{u\;|\;|u|\leq 1\}$. It turns out that the maximal control invariant set $\mathcal{C}_{\infty}$ is $\mathcal{X}$ itself.

Since ${\mathcal{C}}_{\infty}$ is not contractive, this system is challenging for approaches such as [14] that use the Minkowski functional of the maximal control invariant set as terminal cost. To see that $\mathcal{C}_{\infty}$ is not contractive, note that $x_{t}=(0\;5)\in\partial\mathcal{C}_{\infty}$. In addition, the first component of $x_{t+1}$ is equal to $5$ for all $u_{t}$, which means that also $x_{t+1}\in\partial\mathcal{C}_{\infty}$. Hence, the set is not contractive, and the corresponding Minkowski functional will not decrease for all $x\in{\mathcal{C}}_{\infty}$.

However, the system admits a quadratic terminal cost that can be found by the approach developed in this paper. With $Q=I$ and $R=1$, the algorithm in Setion III-D returns

$$Q_{T} =\begin{pmatrix}9.65&0.50\\ 0.50&10.67\end{pmatrix}$$

which is the infinite-horizon cost-to-go for the feedback law $u_{t}=(0.1\,-0.1)x_{t}$. In fact, all state feedback laws on the form $u_{t}=-\begin{pmatrix}l_{1}&l_{2}\end{pmatrix}x$ with $|l_{1}+0.1|+|l_{2}|\leq 0.1$ and $l_{1}$ and $l_{2}$ not both equal to zero are admissible in ${\mathcal{X}}$, render the closed-loop system asymptotically stable and ${\mathcal{X}}$ invariant.

IV-C A higher-order system

As another example, we consider the system

$$x_{t+1} =\begin{pmatrix}0.48&0.45&0.38\\ -0.13&0.52&-0.54\\ -0.58&0.32&0.40\end{pmatrix}x_{t}+\begin{pmatrix}0.15\\ 0.00\\ 0.14\end{pmatrix}u_{t}$$

with cost given by $Q=10I$ and $R=1$, and constraints $\mathcal{X}=\{x\;|\;\|x\|_{\infty}\leq 10\}$ and $\mathcal{U}=\{u\;|\;\|u\|_{\infty}\leq 1\}$. In this case, the maximal control invariant set is much larger (a factor 500x) in volume than the invariant set of the LQR controller, see Figure 5.

Figure 5: Maximal invariant set is much larger than the invariant set of the LQR controller.

Yet, the quadratic upper bounds are quite similar: our procedure finds

$$Q_{T} =\begin{pmatrix}23.98&-1.69&1.03\\ -1.69&23.25&0.78\\ 1.03&0.78&24.65\end{pmatrix}$$

while the Riccati solution is

$$P_{\infty} =\begin{pmatrix}21.18&0.59&0.04\\ 0.59&18.21&-1.95\\ 0.04&-1.95&18.97\end{pmatrix}$$

We simulate the closed-loop system from $x_{0}=\begin{pmatrix}-10&10&0\end{pmatrix}^{\top}$. The MPC controller based on the LQR invariant set and terminal cost requires a horizon of $T=10$ for this initial value, and yields the closed-loop response shown in Figure 6. When we use the maximal control invariant set and our upper bound cost, on the other hand, we can use all horizon lengths and get practically indistinguishable responses even for $T=1$.

Figure 6: Initial value responses for MPC controller with terminal penalty and terminal cost defined by LQR solution and horizon $T=10$ (the minimal horizon required).

V Conclusions

We have presented a numerical procedure for computing a quadratic cost-to-go for linear MPC controllers that is valid for the maximal control invariant sets. This results in the largest possible set of recursively feasible states for the MPC controller, while the closed-loop stability follows from the standard stability proof for linear MPC.

We believe that the suggested procedure could be adapted to many scenarios beyond linear MPC and maximal control invariant sets. In essence, the approach only relies on our ability to compute a control invariant set (not necessarily the maximal one), recover admissible vertex controls, and to interpolate these into a piecewise linear feedback law. We leave such extensions as future work.