Showing 1–37 of 37 results for author: Bensoussan, A

A Control Theoretical Approach to Mean Field Games and Associated Master Equations

Authors: Alain Bensoussan, Ho Man Tai, Tak Kwong Wong, Sheung Chi Phillip Yam

Abstract: We prove the global-in-time well-posedness for a broad class of mean field game problems, which is beyond the special linear-quadratic setting, as long as the mean field sensitivity is not too large. Through the stochastic maximum principle, we adopt the FBSDE approach to investigate the unique existence of the corresponding equilibrium strategies. The corresponding FBSDEs are first solved locally… ▽ More We prove the global-in-time well-posedness for a broad class of mean field game problems, which is beyond the special linear-quadratic setting, as long as the mean field sensitivity is not too large. Through the stochastic maximum principle, we adopt the FBSDE approach to investigate the unique existence of the corresponding equilibrium strategies. The corresponding FBSDEs are first solved locally in time, then by controlling the sensitivity of the backward solutions with respect to the initial condition via some suitable apriori estimates for the corresponding Jacobian flows, the global-in-time solution is warranted. Further analysis on these Jacobian flows will be discussed to establish the regularities, such as linear functional differentiability, of the respective value functions that leads to the ultimate classical well-posedness of the master equation on $\mathbb{R}^d$. To the best of our knowledge, it is the first article to deal with the mean field game problem, as well as its associated master equation, with general cost functionals having quadratic growth under the small mean field effect. In this current approach, we directly impose the structural conditions on the cost functionals, rather than conditions on the Hamiltonian. The advantages of this are threefold: (i) compared with imposing conditions on Hamiltonian, the structural conditions imposed in this work are easily verified, and less demanding on the regularity requirements of the cost functionals while solving the master equation; (ii) the displacement monotonicity is basically just a direct consequence of small mean field effect in the structural conditions; and (iii) when the mean field effect is not that small, we can still provide an accurate lifespan for the local existence. The method in this work can be readily extended to the case with nonlinear drift and non-separable cost functionals. △ Less

Submitted 2 February, 2024; originally announced February 2024.

MSC Class: 49J55; 60H10; 93E20; 60H30

Global Well-Posedness of First-Order Mean Field Games and Master Equations with Nonlinear Dynamics

Authors: Alain Bensoussan, Tak Kwong Wong, Sheung Chi Phillip Yam, Hongwei Yuan

Abstract: This article presents the variant of the approach introduced in the recent work of Bensoussan, Wong, Yam and Yuan [13] to the generic first-order mean field game problem. A major contribution here is the provision of new crucial a priori estimates, whose establishment is fundamentally different from the mentioned work since the associated forward-backward ordinary differential equation (FBODE) sys… ▽ More This article presents the variant of the approach introduced in the recent work of Bensoussan, Wong, Yam and Yuan [13] to the generic first-order mean field game problem. A major contribution here is the provision of new crucial a priori estimates, whose establishment is fundamentally different from the mentioned work since the associated forward-backward ordinary differential equation (FBODE) system is notably different. In addition, we require monotonicity conditions intimately on the coefficient functions but not on the Hamiltonians to handle their non-separable nature and nonlinear dynamics; as tackling Hamiltonians directly, it potentially dissolves much useful information. Compared with the assumptions used in [13], we introduce an additional requirement that the first-order derivative of the drift function in the measure variable cannot be too large relative to the convexity of the running cost function; this requirement only arises when the Hamiltonian is non-separable, and this phenomenon can also be seen in the existing literature. On the other hand, we require less here for the second-order differentiability of the coefficient functions in comparison to that in [13]. Our approach involves first demonstrating the local existence of a solution over small time interval, followed by the provision of new crucial a priori estimates for the sensitivity of the backward equation with respect to the initial condition of forward dynamics; and finally, smoothly gluing the local solutions together to form a global solution. In addition, we establish the local and global existence and uniqueness of classical solutions for the mean field game and its master equation. △ Less

Submitted 14 November, 2023; originally announced November 2023.

Comments: Both this work and arXiv:2305.11848 apply a similar approach to investigate the global-in-time well-posedness of two significant problems in mean field theory, namely mean field games in this work, and the mean field type control problems in arXiv:2305.11848

MSC Class: 60H30; 60H10

Degenerate Mean Field Type Control with Linear and Unbounded Diffusion, and their Associated Equations

Authors: Alain Bensoussan, Ziyu Huang, Shanjian Tang, Sheung Chi Phillip Yam

Abstract: We study the well-posedness of a system of forward-backward stochastic differential equations (FBSDEs) corresponding to a degenerate mean field type control problem, when the diffusion coefficient depends on the state together with its measure and also the control. Degenerate mean field type control problems are rarely studied in the literature. Our method is based on a lifting approach which embe… ▽ More We study the well-posedness of a system of forward-backward stochastic differential equations (FBSDEs) corresponding to a degenerate mean field type control problem, when the diffusion coefficient depends on the state together with its measure and also the control. Degenerate mean field type control problems are rarely studied in the literature. Our method is based on a lifting approach which embeds the control problem and the associated FBSDEs in Wasserstein spaces into certain Hilbert spaces. We use a continuation method to establish the solvability of the FBSDEs and that of the Gâteaux derivatives of this FBSDEs. We then explore the regularity of the value function in time and in measure argument, and we also show that it is the unique classical solution of the associated Bellman equation. We also study the higher regularity of the linear functional derivative of the value function, by then, we obtain the classical solution of the mean field type master equation. △ Less

Submitted 15 November, 2023; originally announced November 2023.

Linear Quadratic Extended Mean Field Games and Control Problems

Authors: Alain Bensoussan, Bohan Li, Sheung Chi Phillip Yam

Abstract: We provide a thorough study of a general class of linear-quadratic extended mean field games and control problems in any dimensions where the mean field terms are allowed to be unbounded and there are also presence of cross terms in the objective functionals. Our investigation focuses on the unique existence of equilibrium strategies for the extended mean field problems by employing the stochastic… ▽ More We provide a thorough study of a general class of linear-quadratic extended mean field games and control problems in any dimensions where the mean field terms are allowed to be unbounded and there are also presence of cross terms in the objective functionals. Our investigation focuses on the unique existence of equilibrium strategies for the extended mean field problems by employing the stochastic maximum principle approach and the appropriate fixed point argument. We provide two distinct proofs, accompanied by two sufficient conditions, that establish the unique existence of the equilibrium strategy over a global time horizon. Both conditions emphasize the importance of sufficiently small coefficients of sensitivity for the cross term, of state and control, and mean field term. To determine the required magnitude of these coefficients, we utilize the singular values of appropriate matrices and Weyl's inequalities. The present proposed theory is consistent with the classical one, namely, our theoretical framework encompasses classical linear-quadratic stochastic control problems as particular cases. Additionally, we establish sufficient conditions for the unique existence of solutions to a particular class of non-symmetric Riccati equations, and we illustrate a counterexample to the existence of equilibrium strategies. Furthermore, we also apply the stochastic maximum principle approach to examine linear-quadratic extended mean field type stochastic control problems. Finally, we conduct a comparative analysis between our method and the alternative master equation approach, specifically addressing the efficacy of the present proposed approach in solving common practical problems, for which the explicit forms of the equilibrium strategies can be obtained directly, even over any global time horizon. △ Less

Submitted 9 November, 2023; originally announced November 2023.

Maximum Principle for Mean Field Type Control Problems with General Volatility Functions

Authors: Alain Bensoussan, Ziyu Huang, Sheung Chi Phillip Yam

Abstract: In this paper, we study the maximum principle of mean field type control problems when the volatility function depends on the state and its measure and also the control, by using our recently developed method. Our method is to embed the mean field type control problem into a Hilbert space to bypass the evolution in the Wasserstein space. We here give a necessary condition and a sufficient conditio… ▽ More In this paper, we study the maximum principle of mean field type control problems when the volatility function depends on the state and its measure and also the control, by using our recently developed method. Our method is to embed the mean field type control problem into a Hilbert space to bypass the evolution in the Wasserstein space. We here give a necessary condition and a sufficient condition for these control problems in Hilbert spaces, and we also derive a system of forward-backward stochastic differential equations. △ Less

Submitted 13 September, 2023; originally announced September 2023.

Reproducing kernel approach to linear quadratic mean field control problems

Authors: Pierre-Cyril Aubin-Frankowski, Alain Bensoussan

Abstract: Mean-field control problems have received continuous interest over the last decade. Despite being more intricate than in classical optimal control, the linear-quadratic setting can still be tackled through Riccati equations. Remarkably, we demonstrate that another significant attribute extends to the mean-field case: the existence of an intrinsic reproducing kernel Hilbert space associated with th… ▽ More Mean-field control problems have received continuous interest over the last decade. Despite being more intricate than in classical optimal control, the linear-quadratic setting can still be tackled through Riccati equations. Remarkably, we demonstrate that another significant attribute extends to the mean-field case: the existence of an intrinsic reproducing kernel Hilbert space associated with the problem. Our findings reveal that this Hilbert space not only encompasses deterministic controlled push-forward map**s but can also represent of stochastic dynamics. Specifically, incorporating Brownian noise affects the deterministic kernel through a conditional expectation, to make the trajectories adapted. Introducing reproducing kernels allows us to rewrite the mean-field control problem as optimizing over a Hilbert space of trajectories rather than controls. This framework even accommodates nonlinear terminal costs, without resorting to adjoint processes or Pontryagin's maximum principle, further highlighting the versatility of the proposed methodology. △ Less

Submitted 22 August, 2023; originally announced August 2023.

MSC Class: 46E22; 49N10; 49N80; 93E20

Alternating minimization for simultaneous estimation of a latent variable and identification of a linear continuous-time dynamic system

Authors: Pierre-Cyril Aubin-Frankowski, Alain Bensoussan, S. Joe Qin

Abstract: We propose an optimization formulation for the simultaneous estimation of a latent variable and the identification of a linear continuous-time dynamic system, given a single input-output pair. We justify this approach based on Bayesian maximum a posteriori estimators. Our scheme takes the form of a convex alternating minimization, over the trajectories and the dynamic model respectively. We prove… ▽ More We propose an optimization formulation for the simultaneous estimation of a latent variable and the identification of a linear continuous-time dynamic system, given a single input-output pair. We justify this approach based on Bayesian maximum a posteriori estimators. Our scheme takes the form of a convex alternating minimization, over the trajectories and the dynamic model respectively. We prove its convergence to a local minimum which verifies a two point-boundary problem for the (latent) state variable and a tensor product expression for the optimal dynamics. △ Less

Submitted 28 June, 2023; originally announced June 2023.

MSC Class: 49N10; 93B30; 62F15

A Theory of First Order Mean Field Type Control Problems and their Equations

Authors: Alain Bensoussan, Tak Kwong Wong, Sheung Chi Phillip Yam, Hongwei Yuan

Abstract: In this article, by using several new crucial {\it a priori} estimates which are still absent in the literature, we provide a comprehensive resolution of the first order generic mean field type control problems and also establish the global-in-time classical solutions of their Bellman and master equations. Rather than develo** the analytical approach via tackling the Bellman and master equation… ▽ More In this article, by using several new crucial {\it a priori} estimates which are still absent in the literature, we provide a comprehensive resolution of the first order generic mean field type control problems and also establish the global-in-time classical solutions of their Bellman and master equations. Rather than develo** the analytical approach via tackling the Bellman and master equation directly, we apply the maximum principle approach by considering the induced forward-backward ordinary differential equation (FBODE) system; indeed, we first show the local-in-time unique existence of the solution of the FBODE system for a variety of terminal data by Banach fixed point argument, and then provide crucial a priori estimates of bounding the sensitivity of the terminal data for the backward equation by utilizing a monotonicity condition that can be deduced from the positive definiteness of the Schur complement of the Hessian matrix of the Lagrangian in the lifted version and manipulating first order condition appropriately; this uniform bound over the whole planning horizon $[0,T]$ allows us to partition $[0,T]$ into a number of sub-intervals with a common small length and then glue the consecutive local-in-time solutions together to form the unique global-in-time solution of the FBODE system. The regularity of the global-in-time solution follows from that of the local ones due to the regularity assumptions on the coefficient functions. Moreover, the regularity of the value function will also be shown with the aid of the regularity of the solution couple of the FBODE system and the regularity assumptions on the coefficient functions, with which we can further deduce that this value function and its linear functional derivative satisfy the Bellman and master equations, respectively. △ Less

Submitted 15 September, 2023; v1 submitted 19 May, 2023; originally announced May 2023.

Comments: This new version corrected some typos with further clarifications after comments received

MSC Class: 35R15; 49L25; 49N70; 91A13; 93E20; 60H30; 60H10; 60H15; 60F99

Mean Field Type Control Problems, Some Hilbert-space-valued FBSDEs, and Related Equations

Authors: Alain Bensoussan, Ho Man Tai, Sheung Chi Phillip Yam

Abstract: In this article, we provide an original systematic global-in-time analysis of mean field type control problems on $\mathbb{R}^n$ with generic cost functionals by the modified approach but not the same, firstly proposed in [7], as the ``lifting'' idea introduced by P. L. Lions. As an alternative to the recent popular analytical method by tackling the master equation, we resolve the control problem… ▽ More In this article, we provide an original systematic global-in-time analysis of mean field type control problems on $\mathbb{R}^n$ with generic cost functionals by the modified approach but not the same, firstly proposed in [7], as the ``lifting'' idea introduced by P. L. Lions. As an alternative to the recent popular analytical method by tackling the master equation, we resolve the control problem in a certain proper Hilbert subspace of the whole space of $L^2$ random variables, it can be regarded as tangent space attached at the initial probability measure. The present work also fills the gap of the global-in-time solvability and extends the previous works of [7,11] which only dealt with quadratic cost functionals in control; the problem is linked to the global solvability of the Hilbert-space-valued forward-backward stochastic differential equation (FBSDE), which is solved by variational techniques here. We also rely on the Jacobian flow of the solution to this FBSDE to establish the regularities of the value function, including its linearly functional differentiability, which leads to the classical well-posedness of the Bellman equation. Together with the linear functional derivatives and the gradient of the linear functional derivatives of the solution to the FBSDE, we also obtain the classical well-posedness of the master equation. △ Less

Submitted 6 May, 2023; originally announced May 2023.

MSC Class: 49N80; 49J55; 49K45; 60H30; 93E20

A Formal Metareasoning Model of Concurrent Planning and Execution

Authors: Amihay Elboher, Ava Bensoussan, Erez Karpas, Wheeler Ruml, Shahaf S. Shperberg, Solomon E. Shimony

Abstract: Agents that plan and act in the real world must deal with the fact that time passes as they are planning. When timing is tight, there may be insufficient time to complete the search for a plan before it is time to act. By commencing execution before search concludes, one gains time to search by making planning and execution concurrent. However, this incurs the risk of making incorrect action choic… ▽ More Agents that plan and act in the real world must deal with the fact that time passes as they are planning. When timing is tight, there may be insufficient time to complete the search for a plan before it is time to act. By commencing execution before search concludes, one gains time to search by making planning and execution concurrent. However, this incurs the risk of making incorrect action choices, especially if actions are irreversible. This tradeoff between opportunity and risk is the problem addressed in this paper. Our main contribution is to formally define this setting as an abstract metareasoning problem. We find that the abstract problem is intractable. However, we identify special cases that are solvable in polynomial time, develop greedy solution algorithms, and, through tests on instances derived from search problems, find several methods that achieve promising practical performance. This work lays the foundation for a principled time-aware executive that concurrently plans and executes. △ Less

Submitted 5 March, 2023; originally announced March 2023.

A Deep Learning Approximation of Non-Stationary Solutions to Wave Kinetic Equations

Authors: Steven Walton, Minh-Binh Tran, Alain Bensoussan

Abstract: We present a deep learning approximation, stochastic optimization based, method for wave kinetic equations. To build confidence in our approach, we apply the method to a Smoluchowski coagulation equation with multiplicative kernel for which an analytic solution exists. Our deep learning approach is then used to approximate the non-stationary solution to a 3-wave kinetic equation corresponding to a… ▽ More We present a deep learning approximation, stochastic optimization based, method for wave kinetic equations. To build confidence in our approach, we apply the method to a Smoluchowski coagulation equation with multiplicative kernel for which an analytic solution exists. Our deep learning approach is then used to approximate the non-stationary solution to a 3-wave kinetic equation corresponding to acoustic wave systems. To validate the neural network approximation, we compare the decay rate of the total energy with previously obtained theoretical results. A finite volume solution is presented and compared with the present method. △ Less

Submitted 25 September, 2022; originally announced September 2022.

The reproducing kernel Hilbert spaces underlying linear SDE Estimation, Kalman filtering and their relation to optimal control

Authors: Pierre-Cyril Aubin-Frankowski, Alain Bensoussan

Abstract: It is often said that control and estimation problems are in duality. Recently, in (Aubin-Frankowski,2021), we found new reproducing kernels in Linear-Quadratic optimal control by focusing on the Hilbert space of controlled trajectories, allowing for a convenient handling of state constraints and meeting points. We now extend this viewpoint to estimation problems where it is known that kernels are… ▽ More It is often said that control and estimation problems are in duality. Recently, in (Aubin-Frankowski,2021), we found new reproducing kernels in Linear-Quadratic optimal control by focusing on the Hilbert space of controlled trajectories, allowing for a convenient handling of state constraints and meeting points. We now extend this viewpoint to estimation problems where it is known that kernels are the covariances of stochastic processes. Here, the Markovian Gaussian processes stem from the linear stochastic differential equations describing the continuous-time dynamics and observations. Taking extensive care to require minimal invertibility requirements on the operators, we give novel explicit formulas for these covariances. We also determine their reproducing kernel Hilbert spaces, stressing the symmetries between a space of forward-time trajectories and a space of backward-time information vectors. The two spaces play an analogue role for filtering to Sobolev spaces in variational analysis, and allow to recover the Kalman estimate through a direct variational argument. For comparison, we then recover the Kalman filter and smoother formulas through more classical arguments based on the innovation process. Extension to discrete-time observations or infinite-dimensional state, tough technical, would be straightforward. △ Less

Submitted 13 October, 2022; v1 submitted 15 August, 2022; originally announced August 2022.

MSC Class: 46E22; 60G35; 62M20

Operator-valued Kernels and Control of Infinite dimensional Dynamic Systems

Authors: Pierre-Cyril Aubin-Frankowski, Alain Bensoussan

Abstract: The Linear Quadratic Regulator (LQR), which is arguably the most classical problem in control theory, was recently related to kernel methods in (Aubin-Frankowski, SICON, 2021) for finite dimensional systems. We show that this result extends to infinite dimensional systems, i.e.\ control of linear partial differential equations. The quadratic objective paired with the linear dynamics encode the rel… ▽ More The Linear Quadratic Regulator (LQR), which is arguably the most classical problem in control theory, was recently related to kernel methods in (Aubin-Frankowski, SICON, 2021) for finite dimensional systems. We show that this result extends to infinite dimensional systems, i.e.\ control of linear partial differential equations. The quadratic objective paired with the linear dynamics encode the relevant kernel, defining a Hilbert space of controlled trajectories, for which we obtain a concise formula based on the solution of the differential Riccati equation. This paves the way to applying representer theorems from kernel methods to solve infinite dimensional optimal control problems. △ Less

Submitted 11 October, 2022; v1 submitted 19 June, 2022; originally announced June 2022.

MSC Class: 46E22; 49N10; 93C20

Control in Hilbert Space and First Order Mean Field Type Problem

Authors: Alain Bensoussan, Henry Hang Cheung, Sheung Chi Phillip Yam

Abstract: We extend the work \cite{bensoussan2019control} by two of the coauthors, which dealt with a deterministic control problem for which the Hilbert space could be generic and investigated a novel form of the `lifting' technique proposed by P. L. Lions. In \cite{bensoussan2019control}, we only showed the local existence and uniqueness of solutions to the FBODEs in the Hilbert space which were associate… ▽ More We extend the work \cite{bensoussan2019control} by two of the coauthors, which dealt with a deterministic control problem for which the Hilbert space could be generic and investigated a novel form of the `lifting' technique proposed by P. L. Lions. In \cite{bensoussan2019control}, we only showed the local existence and uniqueness of solutions to the FBODEs in the Hilbert space which were associated to the control problems with drift function consisting of the control only. In this article, we establish the global existence and uniqueness of the solutions to the FBODEs in Hilbert space corresponding to control problems with separable drift function which is nonlinear in state and linear in control. We shall also prove the sufficiency of the Pontryagin Maximum Principle and derive the corresponding Bellman equation. Besides, we shall show an analogue in the stationary case. Finally, by using the `lifting' idea as in \cite{stochasticv2,stochasticv1}, we shall apply the result to solve the linear quadratic mean field type control problems, and to show the global existence of the corresponding Bellman equations. △ Less

Submitted 25 August, 2021; originally announced August 2021.

Comments: Invited book chapter in Stochastic Analysis, Filtering, and Stochastic Optimization: A Commemorative Volume to Honor Mark H. A. Davies' Contributions

Value-Gradient based Formulation of Optimal Control Problem and Machine Learning Algorithm

Authors: Alain Bensoussan, Jiayue Han, Sheung Chi Phillip Yam, Xiang Zhou

Abstract: Optimal control problem is typically solved by first finding the value function through Hamilton-Jacobi equation (HJE) and then taking the minimizer of the Hamiltonian to obtain the control. In this work, instead of focusing on the value function, we propose a new formulation for the gradient of the value function (value-gradient) as a decoupled system of partial differential equations in the cont… ▽ More Optimal control problem is typically solved by first finding the value function through Hamilton-Jacobi equation (HJE) and then taking the minimizer of the Hamiltonian to obtain the control. In this work, instead of focusing on the value function, we propose a new formulation for the gradient of the value function (value-gradient) as a decoupled system of partial differential equations in the context of continuous-time deterministic discounted optimal control problem. We develop an efficient iterative scheme for this system of equations in parallel by utilizing the properties that they share the same characteristic curves as the HJE for the value function. For the theoretical part, we prove that this iterative scheme converges linearly in $L_α^2$ sense for some suitable exponent $α$ in a weight function. For the numerical method, we combine characteristic line method with machine learning techniques. Specifically, we generate multiple characteristic curves at each policy iteration from an ensemble of initial states, and compute both the value function and its gradient simultaneously on each curve as the labelled data. Then supervised machine learning is applied to minimize the weighted squared loss for both the value function and its gradients. Experimental results demonstrate that this new method not only significantly increases the accuracy but also improves the efficiency and robustness of the numerical estimates, particularly with less amount of characteristics data or fewer training steps. △ Less

Submitted 9 September, 2021; v1 submitted 16 March, 2021; originally announced March 2021.

Machine Learning and Control Theory

Authors: Alain Bensoussan, Yiqun Li, Dinh Phan Cao Nguyen, Minh-Binh Tran, Sheung Chi Phillip Yam, Xiang Zhou

Abstract: We survey in this article the connections between Machine Learning and Control Theory. Control Theory provide useful concepts and tools for Machine Learning. Conversely Machine Learning can be used to solve large control problems. In the first part of the paper, we develop the connections between reinforcement learning and Markov Decision Processes, which are discrete time control problems. In the… ▽ More We survey in this article the connections between Machine Learning and Control Theory. Control Theory provide useful concepts and tools for Machine Learning. Conversely Machine Learning can be used to solve large control problems. In the first part of the paper, we develop the connections between reinforcement learning and Markov Decision Processes, which are discrete time control problems. In the second part, we review the concept of supervised learning and the relation with static optimization. Deep learning which extends supervised learning, can be viewed as a control problem. In the third part, we present the links between stochastic gradient descent and mean-field theory. Conversely, in the fourth and fifth parts, we review machine learning approaches to stochastic control problems, and focus on the deterministic case, to explain, more easily, the numerical algorithms. △ Less

Submitted 9 June, 2020; originally announced June 2020.

Control on Hilbert Spaces and Application to Some Mean Field Type Control Problems

Authors: Alain Bensoussan, P. Jameson Graber, Sheung Chi Phillip Yam

Abstract: We propose a new approach to studying classical solutions of the Bellman equation and Master equation for mean field type control problems, using a novel form of the "lifting" idea introduced by P.-L. Lions. Rather than studying the usual system of Hamilton-Jacobi/Fokker-Planck PDEs using analytic techniques, we instead study a stochastic control problem on a specially constructed Hilbert space, w… ▽ More We propose a new approach to studying classical solutions of the Bellman equation and Master equation for mean field type control problems, using a novel form of the "lifting" idea introduced by P.-L. Lions. Rather than studying the usual system of Hamilton-Jacobi/Fokker-Planck PDEs using analytic techniques, we instead study a stochastic control problem on a specially constructed Hilbert space, which is reminiscent of a tangent space on the Wasserstein space in optimal transport. On this Hilbert space we can use classical control theory techniques, despite the fact that it is infinite dimensional. A consequence of our construction is that the mean field type control problem appears as a special case. Thus we preserve the advantages of the lifiting procedure, while removing some of the difficulties. Our approach extends previous work by two of the coauthors, which dealt with a deterministic control problem for which the Hilbert space could be generic. △ Less

Submitted 9 May, 2023; v1 submitted 21 May, 2020; originally announced May 2020.

MSC Class: 49N80

Identification of linear dynamical systems and machine learning

Authors: Alain Bensoussan, Fatih Gelir, Viswanath Ramakrishna, Minh-Binh Tran

Abstract: The topic of identification of dynamic systems, has been at the core of modern control , following the fundamental works of Kalman. Realization Theory has been one of the major outcomes in this domain, with the possibility of identifying a dynamic system from an input-output relationship. The recent development of machine learning concepts has rejuvanated interest for identification. In this paper… ▽ More The topic of identification of dynamic systems, has been at the core of modern control , following the fundamental works of Kalman. Realization Theory has been one of the major outcomes in this domain, with the possibility of identifying a dynamic system from an input-output relationship. The recent development of machine learning concepts has rejuvanated interest for identification. In this paper, we review briefly the results of realization theory, and develop some methods inspired by Machine Learning concepts. △ Less

Submitted 15 May, 2020; originally announced May 2020.

Mathematical formulation of a dynamical system with dry friction subjected to external forces

Authors: A. Bensoussan, A. Brouste, F. B. Cartiaux, C. Mathey, L. Mertz

Abstract: We consider the response of a one-dimensional system with friction. S.W. Shaw (Journal of Sound and Vibration, 1986) introduced the set up of different coefficients for the static and dynamic phases (also called stick and slip phases). He constructs a step by step solution, corresponding to an harmonic forcing. In this paper, we show that the theory of variational inequalities provides an elegant… ▽ More We consider the response of a one-dimensional system with friction. S.W. Shaw (Journal of Sound and Vibration, 1986) introduced the set up of different coefficients for the static and dynamic phases (also called stick and slip phases). He constructs a step by step solution, corresponding to an harmonic forcing. In this paper, we show that the theory of variational inequalities provides an elegant and synthetic approach to obtain the existence and uniqueness of the solution, avoiding the step by step construction. We then apply the theory to a real structure with real data and show that the model is quite accurate. In our case, the forcing motion comes from dilatation, due to temperature. △ Less

Submitted 4 January, 2020; originally announced January 2020.

Mean Field approach to stochastic control with partial information

Authors: Alain Bensoussan, Sheung Chi Phillip Yam

Abstract: The classical stochastic control problem under partial information can be formulated as a control problem for Zakai equation, whose solution is the unnormalized conditional probability distribution of the state of the system. Zakai equation is a stochastic Fokker-Planck equation. Therefore, the problem to be solved is similar to that met in Mean Field Control theory. Since Mean Field Control theor… ▽ More The classical stochastic control problem under partial information can be formulated as a control problem for Zakai equation, whose solution is the unnormalized conditional probability distribution of the state of the system. Zakai equation is a stochastic Fokker-Planck equation. Therefore, the problem to be solved is similar to that met in Mean Field Control theory. Since Mean Field Control theory is much posterior to the development of Stochastic Control with partial information, the tools, techniques, and concepts obtained in the last decade, for Mean Field Games and Mean field type Control theory, have not been used for the control of Zakai equation. Our objective is to connect the two theories. We get the power of new tools, and we get new insights for the problem of stochastic control with partial information. For mean field theory, we get new interesting applications, but also new problems. Indeed, Mean Field Control Theory leads to very complex equations, like the Master equation, which is a nonlinear infinite dimensional P.D.E., for which general theorems are hardly available, although active research in this direction is performed. Direct methods are useful to obtain regularity results. We will develop in detail the LQ regulator problem, but since we cannot just consider the Gaussian case, well-known results, such as the separation principle is not available. An important result is available in the literature, due to A. Makowsky. It describes the solution of Zakai equation for linear systems with general initial condition (non-gaussian). We show that the separation principle can be extended for quadratic pay-off functionals, but the Kalman filter is much more complex than in the gaussian case. Finally we compare our work to the work of E. Bandini et al. and we show that the example E. Bandini et al. provided does not cover ours. Our system remains nonlinear in their setting. △ Less

Submitted 26 September, 2019; v1 submitted 23 September, 2019; originally announced September 2019.

Comments: 29 pages

MSC Class: 35Q93; 93E11; 93E20

Stochastic Control on Space of Random Variables

Authors: Alain Bensoussan, P. Jameson Graber, S. C. P. Yam

Abstract: By extending \cite{bensoussan2015control}, we implement the proposal of Lions \cite{lions14} on studying mean field games and their master equations via certain control problems on the Hilbert space of square integrable random variables. In \cite{bensoussan2015control}, the Hilbert space could be quite general in the face of the "deterministic control problem" due to the absence of additional rand… ▽ More By extending \cite{bensoussan2015control}, we implement the proposal of Lions \cite{lions14} on studying mean field games and their master equations via certain control problems on the Hilbert space of square integrable random variables. In \cite{bensoussan2015control}, the Hilbert space could be quite general in the face of the "deterministic control problem" due to the absence of additional randomness; while the special case of $L^2$ space of square integrable random variables was brought in at the interpretation stage. The effectiveness of the approach was demonstrated by deriving Bellman equations and the first order master equations through control theory of dynamical systems valued in the Hilbert space. In our present problem for second order master equations, it connects with a stochastic control problem over the space of random variables, and it possesses an additional randomness generated by the Wiener process which cannot be detached from the randomness caused by the elements in the Hilbert space. Nevertheless, we demonstrate how to tackle this difficulty, while preserving most of the efficiency of the approach suggested by Lions \cite{lions14}. △ Less

Submitted 29 March, 2019; originally announced March 2019.

Comments: 28 pages main body, 9 pages appendix

Mean Field Control and Mean Field Game Models with Several Populations

Authors: Alain Bensoussan, Tao Huang, Mathieu Laurière

Abstract: In this paper, we investigate the interaction of two populations with a large number of indistinguishable agents. The problem consists in two levels: the interaction between agents of a same population, and the interaction between the two populations. In the spirit of mean field type control (MFC) problems and mean field games (MFG), each population is approximated by a continuum of infinitesimal… ▽ More In this paper, we investigate the interaction of two populations with a large number of indistinguishable agents. The problem consists in two levels: the interaction between agents of a same population, and the interaction between the two populations. In the spirit of mean field type control (MFC) problems and mean field games (MFG), each population is approximated by a continuum of infinitesimal agents. We define four different problems in a general context and interpret them in the framework of MFC or MFG. By calculus of variations, we derive formally in each case the adjoint equations for the necessary conditions of optimality. Importantly, we find that in the case of a competition between two coalitions, one needs to rely on a system of Master equations in order to describe the equilibrium. Examples are provided, in particular linear-quadratic models for which we obtain systems of ODEs that can be related to Riccati equations. △ Less

Submitted 29 October, 2018; v1 submitted 1 October, 2018; originally announced October 2018.

Optimal periodic replenishment policies for spectrally positive Lévy demand processes

Authors: José-Luis Pérez, Kazutoshi Yamazaki, Alain Bensoussan

Abstract: We consider a version of the stochastic inventory control problem for a spectrally positive Lévy demand process, in which the inventory can only be replenished at independent exponential times. We show the optimality of a periodic barrier replenishment policy that restocks any shortage below a certain threshold at each replenishment opportunity. The optimal policies and value functions are concise… ▽ More We consider a version of the stochastic inventory control problem for a spectrally positive Lévy demand process, in which the inventory can only be replenished at independent exponential times. We show the optimality of a periodic barrier replenishment policy that restocks any shortage below a certain threshold at each replenishment opportunity. The optimal policies and value functions are concisely written in terms of the scale functions. Numerical results are also provided. △ Less

Submitted 15 September, 2020; v1 submitted 24 June, 2018; originally announced June 2018.

Comments: 27 pages, 3 figures. Forthcoming in SIAM Journal on Control and Optimization

MSC Class: 60G51; 93E20; 90B05

Bellman systems with mean field dependent dynamics

Authors: Alain Bensoussan, Miroslav Bulíček, Jens Frehse

Abstract: We deal with nonlinear elliptic and parabolic systems that are the Bellman like systems associated to stochastic differential games with mean field dependent dynamics. The key novelty of the paper is that we allow heavily mean field dependent dynamics. This in particular leads to a system of PDE's with critical growth, for which it is rare to have an existence and/or regularity result. In the pape… ▽ More We deal with nonlinear elliptic and parabolic systems that are the Bellman like systems associated to stochastic differential games with mean field dependent dynamics. The key novelty of the paper is that we allow heavily mean field dependent dynamics. This in particular leads to a system of PDE's with critical growth, for which it is rare to have an existence and/or regularity result. In the paper, we introduce a structural assumptions that cover many cases in stochastic differential games with mean filed dependent dynamics for which we are able to establish the existence of a weak solution. In addition, we present here a completely new method for obtaining the maximum/minimum principles for systems with critical growths, which is a starting point for further existence and also qualitative analysis. △ Less

Submitted 6 November, 2017; originally announced November 2017.

Risk-Sensitive Mean-Field-Type Control

Authors: Alain Bensoussan, Boualem Djehiche, Hamidou Tembine, Phillip Yam

Abstract: We study risk-sensitive optimal control of a stochastic differential equation (SDE) of mean-field type, where the coefficients are allowed to depend on some functional of the law as well as the state and control processes. Moreover the risk-sensitive cost functional is also of mean-field type. We derive optimality equations in infinite dimensions connecting dual functions associated with Bellman f… ▽ More We study risk-sensitive optimal control of a stochastic differential equation (SDE) of mean-field type, where the coefficients are allowed to depend on some functional of the law as well as the state and control processes. Moreover the risk-sensitive cost functional is also of mean-field type. We derive optimality equations in infinite dimensions connecting dual functions associated with Bellman functional to the adjoint process of the Pontryagin maximum principle. The case of linear-exponentiated quadratic cost and its connection with the risk-neutral solution is discussed. △ Less

Submitted 4 February, 2017; originally announced February 2017.

Comments: 14 pages

Mean-field-game model for Botnet defense in Cyber-security

Authors: Vassili Kolokoltsov, Alain Bensoussan

Abstract: We initiate the analysis of the response of computer owners to various offers of defence systems against a cyber-hacker (for instance, a botnet attack), as a stochastic game of a large number of interacting agents. We introduce a simple mean-field game that models their behavior. It takes into account both the random process of the propagation of the infection (controlled by the botner herder) a… ▽ More We initiate the analysis of the response of computer owners to various offers of defence systems against a cyber-hacker (for instance, a botnet attack), as a stochastic game of a large number of interacting agents. We introduce a simple mean-field game that models their behavior. It takes into account both the random process of the propagation of the infection (controlled by the botner herder) and the decision making process of customers. Its stationary version turns out to be exactly solvable (but not at all trivial) under an additional natural assumption that the execution time of the decisions of the customers (say, switch on or out the defence system) is much faster that the infection rates. △ Less

Submitted 20 November, 2015; originally announced November 2015.

MSC Class: 91A80; 93E20; 91A13

Journal ref: Applied Mathematics and Optimization 74(3), 2016, 669-692

Existence and uniqueness of solutions for Bertrand and Cournot mean field games

Authors: P. Jameson Graber, Alain Bensoussan

Abstract: We study a system of partial differential equations used to describe Bertrand and Cournot competition among a continuum of producers of an exhaustible resource. By deriving new a priori estimates, we prove the existence of classical solutions under general assumptions on the data. Moreover, under an additional hypothesis we prove uniqueness. Keywords: mean field games, Hamilton-Jacobi, Fokker-Pl… ▽ More We study a system of partial differential equations used to describe Bertrand and Cournot competition among a continuum of producers of an exhaustible resource. By deriving new a priori estimates, we prove the existence of classical solutions under general assumptions on the data. Moreover, under an additional hypothesis we prove uniqueness. Keywords: mean field games, Hamilton-Jacobi, Fokker-Planck, coupled systems, optimal control, nonlinear partial differential equations △ Less

Submitted 30 August, 2015; v1 submitted 21 August, 2015; originally announced August 2015.

MSC Class: 35K61

Control Problem on Space of Random Variables and Master Equation

Authors: Alain Bensoussan, Phillip Yam

Abstract: We study in this paper a control problem in a space of random variables. We show that its Hamilton Jacobi Bellman equation is related to the Master equation in Mean field theory. P.L. Lions in [14,15] introduced the Hilbert space of square integrable random variables as a natural space for writing the Master equation which appears in the mean field theory. W. Gangbo and A. Święch [10] considered t… ▽ More We study in this paper a control problem in a space of random variables. We show that its Hamilton Jacobi Bellman equation is related to the Master equation in Mean field theory. P.L. Lions in [14,15] introduced the Hilbert space of square integrable random variables as a natural space for writing the Master equation which appears in the mean field theory. W. Gangbo and A. Święch [10] considered this type of equation in the space of probability measures equipped with the Wasserstein metric and use the concept of Wasserstein gradient. We compare the two approaches and provide some extension of the results of Gangbo and Święch. △ Less

Submitted 4 August, 2015; originally announced August 2015.

On The Interpretation Of The Master Equation

Authors: Alain Bensoussan, Jens Frehse, Phillip Yam

Abstract: Since its introduction by P.L. Lions in his lectures and seminars at the College de France, see [9], and also the very helpful notes of Cardialaguet [4] on Lions' lectures, the Master Equation has attracted a lot of interest, and various points of view have been expressed, see for example Carmona-Delarue [5], Bensoussan-Frehse-Yam [2], Buckdahn-Li-Peng-Rainer [3]. There are several ways to introdu… ▽ More Since its introduction by P.L. Lions in his lectures and seminars at the College de France, see [9], and also the very helpful notes of Cardialaguet [4] on Lions' lectures, the Master Equation has attracted a lot of interest, and various points of view have been expressed, see for example Carmona-Delarue [5], Bensoussan-Frehse-Yam [2], Buckdahn-Li-Peng-Rainer [3]. There are several ways to introduce this type of equation; and in those mentioned works, they involve an argument which is a probability measure, while P.L. Lions has recently proposed the idea of working with the Hilbert space of square integrable random variables. Hence writing the equation is an issue; while another issue is its origin. In this article, we discuss all these various aspects, and our modeling argument relies heavily on a seminar at College de France delivered by P.L. Lions on November 14, 2014. △ Less

Submitted 26 March, 2015; originally announced March 2015.

Linear-Quadratic Mean Field Games

Authors: Alain Bensoussan, Joseph Sung, Phillip Yam, Siu Pang Yung

Abstract: In this article, we provide a comprehensive study of the linear-quadratic mean field games via the adjoint equation approach; although the problem has been considered in the literature by Huang, Caines and Malhame (HCM, 2007a), their method is based on Dynamic Programming. It turns out that two methods are not equivalent, as far as giving sufficient condition for the existence of a solution is con… ▽ More In this article, we provide a comprehensive study of the linear-quadratic mean field games via the adjoint equation approach; although the problem has been considered in the literature by Huang, Caines and Malhame (HCM, 2007a), their method is based on Dynamic Programming. It turns out that two methods are not equivalent, as far as giving sufficient condition for the existence of a solution is concerned. Due to the linearity of the adjoint equations, the optimal mean field term satisfies a linear forward-backward ordinary differential equation. For the one dimensional case, we show that the equilibrium strategy always exists uniquely. For dimension greater than one, by choosing a suitable norm and then applying the Banach Fixed Point Theorem, a sufficient condition, which is independent of the solution of the standard Riccati differential equation, for the unique existence of the equilibrium strategy is provided. As a by-product, we also establish a neat and instructive sufficient condition for the unique existence of the solution for a class of non-trivial nonsymmetric Riccati equations. Numerical examples of non-existence of the equilibrium strategy and the comparison of HCM's approach will also be provided. △ Less

Submitted 23 April, 2014; originally announced April 2014.

The Master Equation in Mean Field Theory

Authors: Alain Bensoussan, Jens Frehse, Phillip Yam

Abstract: In his lectures at College de France, P.L. Lions introduced the concept of Master equation, see [5] for Mean Field Games. It is introduced in a heuristic fashion, from the system of partial differential equations, associated to a Nash equilibrium for a large, but finite, number of players. The method, also explained in[2], consists in a formal analogy of terms. The interest of this equation is tha… ▽ More In his lectures at College de France, P.L. Lions introduced the concept of Master equation, see [5] for Mean Field Games. It is introduced in a heuristic fashion, from the system of partial differential equations, associated to a Nash equilibrium for a large, but finite, number of players. The method, also explained in[2], consists in a formal analogy of terms. The interest of this equation is that it contains interesting particular cases, which can be studied directly, in particular the system of HJB-FP (Hamilton-Jacobi-Bellman, Fokker-Planck) equations obtained as the limit of the finite Nash equilibrium game, when the trajectories are independent, see [4]. Usually, in mean field theory, one can bypass the large Nash equilibrium, by introducing the concept of representative agent, whose action is influenced by a distribution of similar agents, and obtains directly the system of HJB-FP equations of interest, see for instance [1]. Apparently, there is no such approach for the Master equation. We show here that it is possible. We first do it for the Mean Field type control problem, for which we interpret completely the Master equation. For the Mean Field Games itself, we solve a related problem, and obtain again the Master equation. △ Less

Submitted 5 November, 2014; v1 submitted 16 April, 2014; originally announced April 2014.

Mean Field Games with a Dominating Player

Authors: Alain Bensoussan, Michael Chau, Phillip Yam

Abstract: In this article, we consider mean field games between a dominating player and a group of representative agents, each of which acts similarly and also interacts with each other through a mean field term being substantially influenced by the dominating player. We first provide the general theory and discuss the necessary condition for the optimal controls and game condition by adopting adjoint equat… ▽ More In this article, we consider mean field games between a dominating player and a group of representative agents, each of which acts similarly and also interacts with each other through a mean field term being substantially influenced by the dominating player. We first provide the general theory and discuss the necessary condition for the optimal controls and game condition by adopting adjoint equation approach. We then present a special case in the context of linear-quadratic framework, in which a necessary and sufficient condition can be asserted by stochastic maximum principle; we finally establish the sufficient condition that guarantees the unique existence of the equilibrium control. The proof of the convergence result of finite player game to mean field counterpart is provided in Appendix. △ Less

Submitted 25 July, 2014; v1 submitted 16 April, 2014; originally announced April 2014.

The Maximum Principle for Global Solutions of Stochastic Stackelberg Differential Games

Authors: Alain Bensoussan, Shaokuan Chen, Suresh P. Sethi

Abstract: This paper obtains the maximum principle for both stochastic (global) open-loop and stochastic (global) closed-loop Stackelberg differential games. For the closed-loop case, we use the theory of controlled forward-backward stochastic differential equations to derive the maximum principle for the leader's optimal strategy. In the special case of the open-loop linear quadratic Stackelberg game, we c… ▽ More This paper obtains the maximum principle for both stochastic (global) open-loop and stochastic (global) closed-loop Stackelberg differential games. For the closed-loop case, we use the theory of controlled forward-backward stochastic differential equations to derive the maximum principle for the leader's optimal strategy. In the special case of the open-loop linear quadratic Stackelberg game, we consider the follower's Hamiltonian system as the leader's state equation, derive the related stochastic Riccati equation, and show the existence and uniqueness of the solution to the Riccati equation under appropriate assumptions. However, for the closed-loop linear quadratic Stackelberg game, we can write the related Riccati equation consisting of forward-backward stochastic differential equations, while leaving the existence of its solution as an open problem. △ Less

Submitted 28 October, 2012; v1 submitted 11 October, 2012; originally announced October 2012.

Asymptotic Analysis of Stochastic Variational Inequalities Modeling an Elasto-Plastic Problem with Vanishing Jumps

Authors: Alain Bensoussan, Hector Jasso Fuentes, Laurent Mertz

Abstract: In a previous work by the first author with J. Turi (AMO, 08), a stochastic variational inequality has been introduced to model an elasto-plastic oscillator with noise. A major advantage of the stochastic variational inequality is to overcome the need to describe the trajectory by phases (elastic or plastic). This is useful, since the sequence of phases cannot be characterized easily. In particula… ▽ More In a previous work by the first author with J. Turi (AMO, 08), a stochastic variational inequality has been introduced to model an elasto-plastic oscillator with noise. A major advantage of the stochastic variational inequality is to overcome the need to describe the trajectory by phases (elastic or plastic). This is useful, since the sequence of phases cannot be characterized easily. In particular, there are numerous small elastic phases which may appear as an artefact of the Wiener process. However, it remains important to have informations on these phases. In order to reconcile these contradictory issues, we introduce an approximation of stochastic variational inequalities by imposing artificial small jumps between phases allowing a clear separation of the phases. In this work, we prove that the approximate solution converges on any finite time interval, when the size of jumps tends to 0. △ Less

Submitted 20 December, 2011; originally announced December 2011.

Comments: 13 pages

Degenerate Dirichlet Problems Related to the Ergodic Property of an Elasto-Plastic Oscillator Excited by a Filtered White Noise

Authors: Alain Bensoussan, Laurent Mertz

Abstract: A stochastic variational inequality is proposed to model an elasto-plastic oscillator excited by a filtered white noise. We prove the ergodic properties of the process and characterize the corresponding invariant measure. This extends Bensoussan-Turi's method (Degenerate Dirichlet Problems Related to the Invariant Measure of Elasto-Plastic Oscillators, AMO, 2008) with a significant additional diff… ▽ More A stochastic variational inequality is proposed to model an elasto-plastic oscillator excited by a filtered white noise. We prove the ergodic properties of the process and characterize the corresponding invariant measure. This extends Bensoussan-Turi's method (Degenerate Dirichlet Problems Related to the Invariant Measure of Elasto-Plastic Oscillators, AMO, 2008) with a significant additional difficulty of increasing the dimension. Two points boundary value problem in dimension 1 is replaced by elliptic equations in dimension 2. In the present context, Khasminskii's method (Stochastic Stability of Differential Equations, Sijthoff and Noordhof,1980) leads to the study of degenerate Dirichlet problems with partial differential equations and nonlocal boundary conditions. △ Less

Submitted 18 December, 2011; originally announced December 2011.

Behavior of the plastic deformation of an elasto-perfectly-plastic oscillator with noise

Authors: Alain Bensoussan, Laurent Mertz

Abstract: Earlier works in engineering, partly experimental, partly computational have revealed that asymptotically, when the excitation is a white noise, plastic deformation and total deformation for an elasto-perfectly-plastic oscillator have a variance which increases linearly with time with the same coefficient. In this work, we prove this result and we characterize the corresponding drift coefficient.… ▽ More Earlier works in engineering, partly experimental, partly computational have revealed that asymptotically, when the excitation is a white noise, plastic deformation and total deformation for an elasto-perfectly-plastic oscillator have a variance which increases linearly with time with the same coefficient. In this work, we prove this result and we characterize the corresponding drift coefficient. Our study relies on a stochastic variational inequality governing the evolution between the velocity of the oscillator and the non-linear restoring force. We then define long cycles behavior of the Markov process solution of the stochastic variational inequality which is the key concept to obtain the result. An important question in engineering is to compute this coefficient. Also, we provide numerical simulations which show successful agreement with our theoretical prediction and previous empirical studies made by engineers. △ Less

Submitted 18 December, 2011; originally announced December 2011.

An analytic approach to the ergodic theory of stochastic variational inequalities

Authors: Alain Bensoussan, Laurent Mertz

Abstract: In an earlier work made by the first author with J. Turi (Degenerate Dirichlet Problems Related to the Invariant Measure of Elasto-Plastic Oscillators, AMO, 2008), the solution of a stochastic variational inequality modeling an elasto-perfectly-plastic oscillator has been studied. The existence and uniqueness of an invariant measure have been proven. Nonlocal problems have been introduced in this… ▽ More In an earlier work made by the first author with J. Turi (Degenerate Dirichlet Problems Related to the Invariant Measure of Elasto-Plastic Oscillators, AMO, 2008), the solution of a stochastic variational inequality modeling an elasto-perfectly-plastic oscillator has been studied. The existence and uniqueness of an invariant measure have been proven. Nonlocal problems have been introduced in this context. In this work, we present a new characterization of the invariant measure. The key finding is the connection between nonlocal PDEs and local PDEs which can be interpreted with short cycles of the Markov process solution of the stochastic variational inequality. △ Less

Submitted 18 December, 2011; originally announced December 2011.