1 Introduction
For any given initial pair , consider a control system governed by the following linear stochastic evolution equations (SEEs, for short) on a finite time horizon:
|
|
|
(1.1) |
where generates a -semigroup on .
The process denotes the control variable, while denotes the state process.
Associated with the control system (1.1), consider the following quadratic cost functional:
|
|
|
|
(1.2) |
|
|
|
|
The stochastic linear quadratic (LQ, for short) optimal control problems are formulated as follows.
Problem (SLQ). For each initial pair , find a , such that
|
|
|
(1.3) |
For any and , under certain conditions (cf. (H1), by Lemma 3.3), there uniquely exists a mild solution , such that
|
|
|
(1.4) |
Hence with further conditions (H2), (1.2) also becomes well-defined.
We introduce the following common concepts in control theory.
Definition 1.1.
1) The Problem (SLQ) is said to be a standard problem, if for ,
|
|
|
2) The Problem (SLQ) is called finite at , if the right hand side of (1.3) is finite;
3) The Problem (SLQ) is called (uniquely) solvable at , if there (uniquely) exists a control satisfying (1.3). If so, is named an (the) optimal control, while the corresponding are called an (the) optimal state, an (the) optimal pair, respectively;
4) The Problem (SLQ) is called finite (resp. (uniquely) solvable), if it is finite (resp. (uniquely) solvable) for any .
The LQ stochastic optimal control problem is the structurally simplest nontrivial model in optimal stochastic control, which can be used to approximate various complex nonlinear models in biology, finance, and other fields. The study on LQ optimal control problem has its origins at least before Bellman et al. [2]. And numerous works have been presented since then, such as Bensoussan et al. [3] for deterministic control systems.
On the other hand, it is currently well-known that the coefficients of one system are difficult to measure accurately, and it is also challenging to explicitly express all factors with variables in physical laws. A good approach to overcome these problems is to introduce randomness to blur the effects caused by these uncertainties, and to transforme it into a stochastic model for research. From this point of view, it is meaningful to add stochastic factors into the deterministic LQ models.
See Chen et al.[4], Sun-Yong [13, 14] for stochastic control systems in finite dimension, and the rich references therein.
Besides, infinite dimension setting obviously covers more situations than the finite dimension counterpart, but with far more technical difficulties (cf. Kotelenez[7]). As a consequence of its importance, there are a great many of works on the optimal control problems for SEEs. For example, Liu-Tang [8] with newly defined operator valued conditional expectation, they regard the operator valued stochastic integral as error terms and absorb it making use of limit analysis to deal with stochastic integration of operator valued stochastic processes. Besides, the transpositon solutions for BSEEs for both vector valued and operator valued processes serving as the first and second order adjoint processes separately for optimal control problems, can be found in detail in the monograph Lü-Zhang[12]. And especially we refer several works in LQ setting (cf. Guatteri-Tessitore[5], Hafizoglu et al.[6], Lü[9], Lü-Wang[10], Xue et al. [16]).
The expression of open-loop optimal control involves systems in the form of forward and backward SEEs (FBSEEs, for short) (3.9). It is natural to further study its well-posedness.
However, solving the FBSEEs (3.9) is quite tough. Recently, Xu et al. [15] studied the domination-monotonicity conditions associated with this kinds of systems, employing the method of continuation. The study in our paper also provides another background for their work. As for its finite dimensional counterpart, namely forward and backward stochastic differential equations, has attracted extensive attentions for more than three decades with lots of related monographs have appeared (cf. [17], and the references therein). Nevertheless, there are rather few conclusion for FBSEEs due to its difficulties, such as the less of Itô’s formula in infinite dimension. It deserves much more attention in the future. Besides, Addona et al. [1] studied the uniqueness of solution for semilinear stochastic Euler-Bernoulli beam equations which describe elastic systems with structural dam**, and they utilized properties of certain related FBSEEs but not through the classical Itô-Tanaka trick.
Both [1] and [15], as well as others, have to appeal to Yosida approximation and finite dimension approximation, but the convergence analysis is difficult due to the lack of compactness arguments in infinite setting. Interested readers are referred to [12] for more detailed introduction. [16] utilized an “discretization then continuousization” method to cope with the infinite dimensional nature of PDE systems.
It was shown by Chen et al.[4] that the LQ problems may be solvable even with indefinite control weight costs, which is not true in deterministic counterpart. Seemingly, stochastic factors give us more chances to solve the control problems. Nevertheless, Lü et al.[11] showed that solvable stochastic LQ problems may NOT have feedback control. Even though we always prefer seeking feedback operators in the face of LQ problems due to its robustness and elegant explicit form, the facts in [11] tell us that sometimes we have no choice but to identify the optimal control utilizing the open-loop necessary conditions. This justifies the value of the kind of results in this paper to some extent.
Even so, we acknowledge that the feedback operator is indeed one of the charming aspects of LQ problems, but its complexity is beyond the scope of this paper and will be presented elsewhere.
And as shown by [14], the conclusions in open-loop optimal control for stochastic LQ problems are important stepstones for the study of its optimal feedback operators.
The main contributions of this paper are summarized as follows. Utilizing dual analysis methods and by introducing certain adjoint equations, we obtain the Fréchet differentiable of the cost functional with respect to the control variable, and characterize the Fréchet derivatives in detail with the help of the operators derived from the dual analysis. Moreover, the necessary and sufficient conditions for a control to be open-loop optimal are proven to be equivalent to the convexity (or equivalently, nonnegative) of certain cost functional of related homogeneous control problems and a stationary condition (which is exactly the above Fréchet derivatives) together with one coupled forward and backward system. The nonhomogeneous features of the models allow one more step to study stochastic differential games. Besides, we do not make the usual Markovian assumption on the coefficients and weight operators.
The rest of this paper is organized as follows. In Section 2, we present necessary notations and technical conditions for the coefficients and weight operators. Some elementary materials about SEEs and BSEEs are gethered in Section 3. In Section 4, we perform the dual analysis and character the Fréchet differentiable of the cost functional. The necesary and sufficient conditions for a control to be open-loop optimal are studied in Section 5. Section 6 is dedicated to the application of the main results to two-person stochastic differential games. At last, we make some conclusions and discussions in Section 7.
4 Operators derived from state processes
As preparations representing the variational expansion of the cost functional with respect to the control variable, from the well-posedness of the Eq. (1.1), for any initial pair and , define the following operators:
|
|
|
|
|
|
|
|
|
|
|
|
and the processes
|
|
|
where solves (1.1).
Besides, by the linear structure of the control system (1.1), it derives
|
|
|
(4.1) |
and
|
|
|
(4.2) |
To represent the variational expansion of the cost functional, we also should study the adjoint operators of the above operators
|
|
|
Consider the following BSEE
|
|
|
(4.3) |
where and .
There are several definitions of solutions to Eq. (4.3). In this paper, we adopt the concept of transposition solution, which is particularly suited for handling optimal control problems.
See Definition 3.5 for details.
Moreover, by Lemma 3.6, for any and , there exists a unique transposition solution
|
|
|
satisfying
|
|
|
This estimation guarantees the boundedness of the following four linear operators.
Now we present the following proposition about the adjoint operators.
Proposition 4.1.
For any and ,
|
|
|
(4.4) |
where is the transposition solution to (4.3).
Proof.
We merely sketch the proof to illustrate the idea.
Taking , the Eq. (4.3) can be regarded as in the form of (3.5).
Recalling the relation , under the assumptions in (H1), the conditions in Lemma 3.6 are fulfilled.
From the definition of transposition solutions to the Eq. (4.3),
taking the test processes in (3.7) as the controlled state processes in (1.1) with , and in particular, ,
it derives
|
|
|
where is the mild solution to (1.1) for any initial pair and .
Then making use of the operator representations for the processes in the beginning of this section, we obtain
|
|
|
|
(4.5) |
|
|
|
|
Moreover, taking
|
|
|
by turns in the relations (4.5), the results can be proved.
∎
5 Main results for optimal control
One method to study the open-loop solvability of the Problem (SLQ) is to characterize the coefficients of in the variational representation of , . In detail, we have
Theorem 5.1.
Under conditions (H1) and (H2), for any given initial pair , the map** is Fréchet differentiable, and its derivative at is
|
|
|
where the meanings of these operators are interpreted in (5).
Proof.
With the help of the Proposition 4.1, we deduce several representations which are essential to obtain the ultimate optimal necessary conditions.
Taking the linear expressions for the state processes in (4.1) and (4.2) into the cost functional (1.2), it can be verified that
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
To simplify the notations, denote
|
|
|
|
|
|
|
|
|
|
|
|
(5.1) |
|
|
|
|
|
|
|
|
|
|
|
|
From the regularities of the coefficients and estimates for the operators as well as their adjoint operators, it can be derived that is a bounded linear operator from to itself. is a bounded linear operator from to . is a bounded linear operator from to .
Then
|
|
|
|
(5.2) |
|
|
|
|
Moreover, for any and ,
|
|
|
|
|
|
|
|
|
|
|
|
(5.3) |
|
|
|
|
Then by the definition of the Fréchet derivative, the conclusion follows.
∎
Actually, due to the relation (5.2), we have
Corollary 5.2.
1) One of the necessary conditions for the problem (SLQ) associated with the initial pair to be finite is .
2) The problem (SLQ) with the initial pair is solvable if and only if and there exists , such that .
3)
Moreover, if , the cost functional uniquely has an explicit minimizer .
Provided with the above preparation, we present the necessary and sufficient conditions for open-loop optimal controls.
Theorem 5.3.
Under conditions (H1) and (H2), for any given initial pair , is an open-loop optimal control to the Problem (SLQ) associated with if and only if
1) the map** is convex;
2) the process satisfies
|
|
|
where the triple satisfy the following
|
|
|
(5.4) |
Proof.
By the definition of related operators and Proposition 4.1, it can be calculated that
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Sum it up resulting in
|
|
|
|
|
|
|
|
|
|
|
|
Utilizing the linear structure in (4.1), (4.2) and (4.3), it can be deduced that
|
|
|
(5.5) |
The “sufficient” part:
By (5), it indicates that for any and ,
|
|
|
|
(5.6) |
|
|
|
|
Then by (5.5) and (5.6), is an optimal control to the Problem (SLQ) associated with if 1) and 2) hold.
The “necessary” part:
By (5.6), if is an optimal control, for any and , we have
|
|
|
|
|
|
|
|
It indicates that is quadratic and nonnegative in for any fixed . By elementary calculus for quadratic functions, we obtain
|
|
|
Moreover, from the bi-linearity of the cost funtional , and the equivalence between the convexity and the nonnegative for a bi-linearity function, it derives that 1) holds. Besides, by the arbitrariness of , the pointwise condition 2) holds.
∎
The system (5.4) seems to be decoupled at first glance. However, it should be understood in the sense of
|
|
|
(5.7) |
which is a fully-coupled forward and backward stochastic evolution system, and is rather challenging to solve (cf. [15] and the references therein).
The coupling comes from the linear stationary condition, which is also called feedback constrained condition. In other word, if is an open-loop optimal control for the Problem (SLQ) with the initial pair , it must be determined by the system of Eq. (5.7).
Actually, (5.7) is in general called the optimality system of Problem (SLQ) in literature.
6 Linear quadratic stochastic differential games
As an application of the conclusions on stochastic optimal control problems in the previous section, we study the following models on LQ two-person stochastic differential games.
For any given initial pair , consider the following controlled SEEs:
|
|
|
(6.1) |
while the cost functional for player is defined with by
|
|
|
|
(6.2) |
|
|
|
|
|
|
|
|
|
|
|
|
Formally, the cost functionals can be presented in the more explicit form
|
|
|
|
|
|
|
|
The LQ stochastic two-person differential games are formulated as follows.
Problem (SDG).
For any initial pair , what the controls should the players choose to minimize their payoff ?
Definition 6.1.
A pair of strategies are called an open-loop Nash equilibrium of the Problem (SDG) with the initial pair , if
|
|
|
|
|
|
As commonly done in literature, we study the open-loop two-person stochastic differential game by investigating two related stochastic optimal control problems as follows.
Suppose that is an open-loop Nash equilibrium of Problem (SDG) for the initial pair , then consider the following two stochastic optimal control problems:
Problem (SDG1) To minimize
|
|
|
(6.3) |
subject to the state equation
|
|
|
(6.4) |
Problem (SDG2) To minimize
|
|
|
(6.5) |
subject to the state equation
|
|
|
(6.6) |
Since is an open-loop Nash equilibrium of Problem (SDG) associated with the initial pair , it deduces that is an open-loop optimal control of Problem (SDG1), while is an open-loop optimal control of Problem (SDG2). To apply the Theorem 5.3, we impose the following conditions supposed for the coefficients and weighting operators.
Condition 6.2.
Assume that the coefficients in (6.1) and weighting operators in (6.2) satisfy the following conditions
|
|
|
|
(GH1) |
|
|
|
|
|
|
|
|
and
|
|
|
|
(GH2) |
|
|
|
|
|
|
|
|
Now we present the main conclusions of this section.
Theorem 6.3.
Under conditions (GH1) and (GH2), is an open-loop Nash equilibrium of Problem (SDG) for the initial pair if and only if the following conditions for hold:
1) satisfy the feedback constrained conditions
|
|
|
where the triple are transposition solutions to the following FBSEEs
|
|
|
2) the map is convex, where is the cost functional to the homogeneous optimal control problems with controlled state equations
|
|
|
and cost functional
|
|
|
|
|
|
|
|
Proof.
It can be verified by lengthy but straight calculus with the help of Theorem 5.3, and we omit the details.
∎
Note that in the above claim 2), the convexity of the map is equivalent to for all .