-
Optimal Savings and Value of Population in A Stochastic Environment: Transient Behavior
Authors:
Hao Liu,
Suresh P. Sethi,
Tak Kwong Wong,
Sheung Chi Phillip Yam
Abstract:
We extend the work on optimal investment and consumption of a population considered in [2] to a general stochastic setting over a finite time horizon. We incorporate the Cobb-Douglas production function in the capital dynamics while the consumption utility function and the drift rate in the population dynamics can be general, in contrast with [2, 30, 31]. The dynamic programming formulation yields…
▽ More
We extend the work on optimal investment and consumption of a population considered in [2] to a general stochastic setting over a finite time horizon. We incorporate the Cobb-Douglas production function in the capital dynamics while the consumption utility function and the drift rate in the population dynamics can be general, in contrast with [2, 30, 31]. The dynamic programming formulation yields an unconventional nonlinear Hamilton-Jacobi-Bellman (HJB) equation, in which the Cobb-Douglas production function as the coefficient of the gradient of the value function induces the mismatching of power rates between capital and population. Moreover, the equation has a very singular term, essentially a very negative power of the partial derivative of the value function with respect to the capital, coming from the optimization of control, and their resolution turns out to be a complex problem not amenable to classical analysis. To show that this singular term, which has not been studied in any physical systems yet, does not actually blow up, we establish new pointwise generalized power laws for the partial derivative of the value function. Our contribution lies in providing a theoretical treatment that combines both the probabilistic approach and theory of partial differential equations to derive the pointwise upper and lower bounds as well as energy estimates in weighted Sobolev spaces. By then, we accomplish showing the well-posedness of classical solutions to a non-canonical parabolic equation arising from a long-lasting problem in macroeconomics.
△ Less
Submitted 16 February, 2024;
originally announced February 2024.
-
A Control Theoretical Approach to Mean Field Games and Associated Master Equations
Authors:
Alain Bensoussan,
Ho Man Tai,
Tak Kwong Wong,
Sheung Chi Phillip Yam
Abstract:
We prove the global-in-time well-posedness for a broad class of mean field game problems, which is beyond the special linear-quadratic setting, as long as the mean field sensitivity is not too large. Through the stochastic maximum principle, we adopt the FBSDE approach to investigate the unique existence of the corresponding equilibrium strategies. The corresponding FBSDEs are first solved locally…
▽ More
We prove the global-in-time well-posedness for a broad class of mean field game problems, which is beyond the special linear-quadratic setting, as long as the mean field sensitivity is not too large. Through the stochastic maximum principle, we adopt the FBSDE approach to investigate the unique existence of the corresponding equilibrium strategies. The corresponding FBSDEs are first solved locally in time, then by controlling the sensitivity of the backward solutions with respect to the initial condition via some suitable apriori estimates for the corresponding Jacobian flows, the global-in-time solution is warranted. Further analysis on these Jacobian flows will be discussed to establish the regularities, such as linear functional differentiability, of the respective value functions that leads to the ultimate classical well-posedness of the master equation on $\mathbb{R}^d$. To the best of our knowledge, it is the first article to deal with the mean field game problem, as well as its associated master equation, with general cost functionals having quadratic growth under the small mean field effect. In this current approach, we directly impose the structural conditions on the cost functionals, rather than conditions on the Hamiltonian. The advantages of this are threefold: (i) compared with imposing conditions on Hamiltonian, the structural conditions imposed in this work are easily verified, and less demanding on the regularity requirements of the cost functionals while solving the master equation; (ii) the displacement monotonicity is basically just a direct consequence of small mean field effect in the structural conditions; and (iii) when the mean field effect is not that small, we can still provide an accurate lifespan for the local existence. The method in this work can be readily extended to the case with nonlinear drift and non-separable cost functionals.
△ Less
Submitted 2 February, 2024;
originally announced February 2024.
-
Global Well-Posedness of First-Order Mean Field Games and Master Equations with Nonlinear Dynamics
Authors:
Alain Bensoussan,
Tak Kwong Wong,
Sheung Chi Phillip Yam,
Hongwei Yuan
Abstract:
This article presents the variant of the approach introduced in the recent work of Bensoussan, Wong, Yam and Yuan [13] to the generic first-order mean field game problem. A major contribution here is the provision of new crucial a priori estimates, whose establishment is fundamentally different from the mentioned work since the associated forward-backward ordinary differential equation (FBODE) sys…
▽ More
This article presents the variant of the approach introduced in the recent work of Bensoussan, Wong, Yam and Yuan [13] to the generic first-order mean field game problem. A major contribution here is the provision of new crucial a priori estimates, whose establishment is fundamentally different from the mentioned work since the associated forward-backward ordinary differential equation (FBODE) system is notably different. In addition, we require monotonicity conditions intimately on the coefficient functions but not on the Hamiltonians to handle their non-separable nature and nonlinear dynamics; as tackling Hamiltonians directly, it potentially dissolves much useful information. Compared with the assumptions used in [13], we introduce an additional requirement that the first-order derivative of the drift function in the measure variable cannot be too large relative to the convexity of the running cost function; this requirement only arises when the Hamiltonian is non-separable, and this phenomenon can also be seen in the existing literature. On the other hand, we require less here for the second-order differentiability of the coefficient functions in comparison to that in [13]. Our approach involves first demonstrating the local existence of a solution over small time interval, followed by the provision of new crucial a priori estimates for the sensitivity of the backward equation with respect to the initial condition of forward dynamics; and finally, smoothly gluing the local solutions together to form a global solution. In addition, we establish the local and global existence and uniqueness of classical solutions for the mean field game and its master equation.
△ Less
Submitted 14 November, 2023;
originally announced November 2023.
-
Mean Field Analysis of Two-Party Governance: Competition versus Cooperation among Leaders
Authors:
Dantong Chu,
Kenneth Tsz Hin Ng,
Sheung Chi Phillip Yam,
Harry Zheng
Abstract:
This article studies linear-quadratic Stackelberg games between two dominating players (or equivalently, leaders) and a large group of followers, each of whom interacts under a mean field game (MFG) framework. Unlike the conventional major-minor player game, the mean field term herein is endogenously affected by the two leaders simultaneously. These homogeneous followers are non-cooperative, where…
▽ More
This article studies linear-quadratic Stackelberg games between two dominating players (or equivalently, leaders) and a large group of followers, each of whom interacts under a mean field game (MFG) framework. Unlike the conventional major-minor player game, the mean field term herein is endogenously affected by the two leaders simultaneously. These homogeneous followers are non-cooperative, whereas the two leaders can either compete or cooperate with each other, which are respectively formulated as a Nash and a Pareto game. The complete solutions of the leader-follower game can be expressed in terms of the solutions of some non-symmetric Riccati equations. Notably, our analysis suggests that both modes of interactions between leaders has their own merits and neither of them is always more favourable to the community of followers. In our knowledge, a comparative study of the effect of different modes of governance on the society is relatively rare in the existing literature, we here provide its first preliminary quantitative analysis; under a broad class of practically relevant models, we provide sufficient conditions to decide whether cooperation or competition between leaders is more favourable to the followers. Being in common with modern folklore, the relative merits of the two Stackelberg games depend on whether the interests between the two leaders and the followers align among themselves. Representative numerical examples are also supplemented.
△ Less
Submitted 1 July, 2024; v1 submitted 17 November, 2023;
originally announced November 2023.
-
Degenerate Mean Field Type Control with Linear and Unbounded Diffusion, and their Associated Equations
Authors:
Alain Bensoussan,
Ziyu Huang,
Shanjian Tang,
Sheung Chi Phillip Yam
Abstract:
We study the well-posedness of a system of forward-backward stochastic differential equations (FBSDEs) corresponding to a degenerate mean field type control problem, when the diffusion coefficient depends on the state together with its measure and also the control. Degenerate mean field type control problems are rarely studied in the literature. Our method is based on a lifting approach which embe…
▽ More
We study the well-posedness of a system of forward-backward stochastic differential equations (FBSDEs) corresponding to a degenerate mean field type control problem, when the diffusion coefficient depends on the state together with its measure and also the control. Degenerate mean field type control problems are rarely studied in the literature. Our method is based on a lifting approach which embeds the control problem and the associated FBSDEs in Wasserstein spaces into certain Hilbert spaces. We use a continuation method to establish the solvability of the FBSDEs and that of the Gâteaux derivatives of this FBSDEs. We then explore the regularity of the value function in time and in measure argument, and we also show that it is the unique classical solution of the associated Bellman equation. We also study the higher regularity of the linear functional derivative of the value function, by then, we obtain the classical solution of the mean field type master equation.
△ Less
Submitted 15 November, 2023;
originally announced November 2023.
-
Linear Quadratic Extended Mean Field Games and Control Problems
Authors:
Alain Bensoussan,
Bohan Li,
Sheung Chi Phillip Yam
Abstract:
We provide a thorough study of a general class of linear-quadratic extended mean field games and control problems in any dimensions where the mean field terms are allowed to be unbounded and there are also presence of cross terms in the objective functionals. Our investigation focuses on the unique existence of equilibrium strategies for the extended mean field problems by employing the stochastic…
▽ More
We provide a thorough study of a general class of linear-quadratic extended mean field games and control problems in any dimensions where the mean field terms are allowed to be unbounded and there are also presence of cross terms in the objective functionals. Our investigation focuses on the unique existence of equilibrium strategies for the extended mean field problems by employing the stochastic maximum principle approach and the appropriate fixed point argument. We provide two distinct proofs, accompanied by two sufficient conditions, that establish the unique existence of the equilibrium strategy over a global time horizon. Both conditions emphasize the importance of sufficiently small coefficients of sensitivity for the cross term, of state and control, and mean field term. To determine the required magnitude of these coefficients, we utilize the singular values of appropriate matrices and Weyl's inequalities. The present proposed theory is consistent with the classical one, namely, our theoretical framework encompasses classical linear-quadratic stochastic control problems as particular cases. Additionally, we establish sufficient conditions for the unique existence of solutions to a particular class of non-symmetric Riccati equations, and we illustrate a counterexample to the existence of equilibrium strategies. Furthermore, we also apply the stochastic maximum principle approach to examine linear-quadratic extended mean field type stochastic control problems. Finally, we conduct a comparative analysis between our method and the alternative master equation approach, specifically addressing the efficacy of the present proposed approach in solving common practical problems, for which the explicit forms of the equilibrium strategies can be obtained directly, even over any global time horizon.
△ Less
Submitted 9 November, 2023;
originally announced November 2023.
-
Maximum Principle for Mean Field Type Control Problems with General Volatility Functions
Authors:
Alain Bensoussan,
Ziyu Huang,
Sheung Chi Phillip Yam
Abstract:
In this paper, we study the maximum principle of mean field type control problems when the volatility function depends on the state and its measure and also the control, by using our recently developed method. Our method is to embed the mean field type control problem into a Hilbert space to bypass the evolution in the Wasserstein space. We here give a necessary condition and a sufficient conditio…
▽ More
In this paper, we study the maximum principle of mean field type control problems when the volatility function depends on the state and its measure and also the control, by using our recently developed method. Our method is to embed the mean field type control problem into a Hilbert space to bypass the evolution in the Wasserstein space. We here give a necessary condition and a sufficient condition for these control problems in Hilbert spaces, and we also derive a system of forward-backward stochastic differential equations.
△ Less
Submitted 13 September, 2023;
originally announced September 2023.
-
A Theory of First Order Mean Field Type Control Problems and their Equations
Authors:
Alain Bensoussan,
Tak Kwong Wong,
Sheung Chi Phillip Yam,
Hongwei Yuan
Abstract:
In this article, by using several new crucial {\it a priori} estimates which are still absent in the literature, we provide a comprehensive resolution of the first order generic mean field type control problems and also establish the global-in-time classical solutions of their Bellman and master equations. Rather than develo** the analytical approach via tackling the Bellman and master equation…
▽ More
In this article, by using several new crucial {\it a priori} estimates which are still absent in the literature, we provide a comprehensive resolution of the first order generic mean field type control problems and also establish the global-in-time classical solutions of their Bellman and master equations. Rather than develo** the analytical approach via tackling the Bellman and master equation directly, we apply the maximum principle approach by considering the induced forward-backward ordinary differential equation (FBODE) system; indeed, we first show the local-in-time unique existence of the solution of the FBODE system for a variety of terminal data by Banach fixed point argument, and then provide crucial a priori estimates of bounding the sensitivity of the terminal data for the backward equation by utilizing a monotonicity condition that can be deduced from the positive definiteness of the Schur complement of the Hessian matrix of the Lagrangian in the lifted version and manipulating first order condition appropriately; this uniform bound over the whole planning horizon $[0,T]$ allows us to partition $[0,T]$ into a number of sub-intervals with a common small length and then glue the consecutive local-in-time solutions together to form the unique global-in-time solution of the FBODE system. The regularity of the global-in-time solution follows from that of the local ones due to the regularity assumptions on the coefficient functions. Moreover, the regularity of the value function will also be shown with the aid of the regularity of the solution couple of the FBODE system and the regularity assumptions on the coefficient functions, with which we can further deduce that this value function and its linear functional derivative satisfy the Bellman and master equations, respectively.
△ Less
Submitted 15 September, 2023; v1 submitted 19 May, 2023;
originally announced May 2023.
-
Mean Field Type Control Problems, Some Hilbert-space-valued FBSDEs, and Related Equations
Authors:
Alain Bensoussan,
Ho Man Tai,
Sheung Chi Phillip Yam
Abstract:
In this article, we provide an original systematic global-in-time analysis of mean field type control problems on $\mathbb{R}^n$ with generic cost functionals by the modified approach but not the same, firstly proposed in [7], as the ``lifting'' idea introduced by P. L. Lions. As an alternative to the recent popular analytical method by tackling the master equation, we resolve the control problem…
▽ More
In this article, we provide an original systematic global-in-time analysis of mean field type control problems on $\mathbb{R}^n$ with generic cost functionals by the modified approach but not the same, firstly proposed in [7], as the ``lifting'' idea introduced by P. L. Lions. As an alternative to the recent popular analytical method by tackling the master equation, we resolve the control problem in a certain proper Hilbert subspace of the whole space of $L^2$ random variables, it can be regarded as tangent space attached at the initial probability measure. The present work also fills the gap of the global-in-time solvability and extends the previous works of [7,11] which only dealt with quadratic cost functionals in control; the problem is linked to the global solvability of the Hilbert-space-valued forward-backward stochastic differential equation (FBSDE), which is solved by variational techniques here. We also rely on the Jacobian flow of the solution to this FBSDE to establish the regularities of the value function, including its linearly functional differentiability, which leads to the classical well-posedness of the Bellman equation. Together with the linear functional derivatives and the gradient of the linear functional derivatives of the solution to the FBSDE, we also obtain the classical well-posedness of the master equation.
△ Less
Submitted 6 May, 2023;
originally announced May 2023.
-
An Effective Multivariate Normality Test via Hessians of Empirical Cumulant Generating Functions
Authors:
Kwun Chuen Gary Chan,
Hok Kan Ling,
Chuan-Fa Tang,
Sheung Chi Phillip Yam
Abstract:
In this article, we propose a new class of consistent tests for $p$-variate normality. These tests are based on the characterization of the standard multivariate normal distribution, that the Hessian of the corresponding cumulant generating function is identical to the $p\times p$ identity matrix and the idea of decomposing the information from the joint distribution into the dependence copula and…
▽ More
In this article, we propose a new class of consistent tests for $p$-variate normality. These tests are based on the characterization of the standard multivariate normal distribution, that the Hessian of the corresponding cumulant generating function is identical to the $p\times p$ identity matrix and the idea of decomposing the information from the joint distribution into the dependence copula and all marginal distributions. Under the null hypothesis of multivariate normality, our proposed test statistic is independent of the unknown mean vector and covariance matrix so that the distribution-free critical value of the test can be obtained by Monte Carlo simulation. We also derive the asymptotic null distribution of proposed test statistic and establish the consistency of the test against different fixed alternatives. Last but not least, a comprehensive and extensive Monte Carlo study also illustrates that our test is a superb yet computationally convenient competitor to many well-known existing test statistics.
△ Less
Submitted 20 March, 2023;
originally announced March 2023.
-
Nonparametric Likelihood Ratio Test for Univariate Shape-constrained Densities
Authors:
Kwun Chuen Gary Chan,
Hok Kan Ling,
Chuan-Fa Tang,
Sheung Chi Phillip Yam
Abstract:
We provide a comprehensive study of a nonparametric likelihood ratio test on whether a random sample follows a distribution in a prespecified class of shape-constrained densities. While the conventional definition of likelihood ratio is not well-defined for general nonparametric problems, we consider a working sub-class of alternative densities that leads to test statistics with desirable properti…
▽ More
We provide a comprehensive study of a nonparametric likelihood ratio test on whether a random sample follows a distribution in a prespecified class of shape-constrained densities. While the conventional definition of likelihood ratio is not well-defined for general nonparametric problems, we consider a working sub-class of alternative densities that leads to test statistics with desirable properties. Under the null, a scaled and centered version of the test statistic is asymptotic normal and distribution-free, which comes from the fact that the asymptotic dominant term under the null depends only on a function of spacings of transformed outcomes that are uniform distributed. The nonparametric maximum likelihood estimator (NPMLE) under the hypothesis class appears only in an average log-density ratio which often converges to zero at a faster rate than the asymptotic normal term under the null, while diverges in general test so that the test is consistent. The main technicality is to show these results for log-density ratio which requires a case-by-case analysis, including new results for k-monotone densities with unbounded support and completely monotone densities that are of independent interest. A bootstrap method by simulating from the NPMLE is shown to have the same limiting distribution as the test statistic.
△ Less
Submitted 23 November, 2022;
originally announced November 2022.
-
Control in Hilbert Space and First Order Mean Field Type Problem
Authors:
Alain Bensoussan,
Henry Hang Cheung,
Sheung Chi Phillip Yam
Abstract:
We extend the work \cite{bensoussan2019control} by two of the coauthors, which dealt with a deterministic control problem for which the Hilbert space could be generic and investigated a novel form of the `lifting' technique proposed by P. L. Lions. In \cite{bensoussan2019control}, we only showed the local existence and uniqueness of solutions to the FBODEs in the Hilbert space which were associate…
▽ More
We extend the work \cite{bensoussan2019control} by two of the coauthors, which dealt with a deterministic control problem for which the Hilbert space could be generic and investigated a novel form of the `lifting' technique proposed by P. L. Lions. In \cite{bensoussan2019control}, we only showed the local existence and uniqueness of solutions to the FBODEs in the Hilbert space which were associated to the control problems with drift function consisting of the control only. In this article, we establish the global existence and uniqueness of the solutions to the FBODEs in Hilbert space corresponding to control problems with separable drift function which is nonlinear in state and linear in control. We shall also prove the sufficiency of the Pontryagin Maximum Principle and derive the corresponding Bellman equation. Besides, we shall show an analogue in the stationary case. Finally, by using the `lifting' idea as in \cite{stochasticv2,stochasticv1}, we shall apply the result to solve the linear quadratic mean field type control problems, and to show the global existence of the corresponding Bellman equations.
△ Less
Submitted 25 August, 2021;
originally announced August 2021.
-
Value-Gradient based Formulation of Optimal Control Problem and Machine Learning Algorithm
Authors:
Alain Bensoussan,
Jiayue Han,
Sheung Chi Phillip Yam,
Xiang Zhou
Abstract:
Optimal control problem is typically solved by first finding the value function through Hamilton-Jacobi equation (HJE) and then taking the minimizer of the Hamiltonian to obtain the control. In this work, instead of focusing on the value function, we propose a new formulation for the gradient of the value function (value-gradient) as a decoupled system of partial differential equations in the cont…
▽ More
Optimal control problem is typically solved by first finding the value function through Hamilton-Jacobi equation (HJE) and then taking the minimizer of the Hamiltonian to obtain the control. In this work, instead of focusing on the value function, we propose a new formulation for the gradient of the value function (value-gradient) as a decoupled system of partial differential equations in the context of continuous-time deterministic discounted optimal control problem. We develop an efficient iterative scheme for this system of equations in parallel by utilizing the properties that they share the same characteristic curves as the HJE for the value function. For the theoretical part, we prove that this iterative scheme converges linearly in $L_α^2$ sense for some suitable exponent $α$ in a weight function. For the numerical method, we combine characteristic line method with machine learning techniques. Specifically, we generate multiple characteristic curves at each policy iteration from an ensemble of initial states, and compute both the value function and its gradient simultaneously on each curve as the labelled data. Then supervised machine learning is applied to minimize the weighted squared loss for both the value function and its gradients. Experimental results demonstrate that this new method not only significantly increases the accuracy but also improves the efficiency and robustness of the numerical estimates, particularly with less amount of characteristics data or fewer training steps.
△ Less
Submitted 9 September, 2021; v1 submitted 16 March, 2021;
originally announced March 2021.
-
PhishGAN: Data Augmentation and Identification of Homoglpyh Attacks
Authors:
Joon Sern Lee,
Gui Peng David Yam,
** Hao Chan
Abstract:
Homoglyph attacks are a common technique used by hackers to conduct phishing. Domain names or links that are visually similar to actual ones are created via punycode to obfuscate the attack, making the victim more susceptible to phishing. For example, victims may mistake "|inkedin.com" for "linkedin.com" and in the process, divulge personal details to the fake website. Current State of The Art (SO…
▽ More
Homoglyph attacks are a common technique used by hackers to conduct phishing. Domain names or links that are visually similar to actual ones are created via punycode to obfuscate the attack, making the victim more susceptible to phishing. For example, victims may mistake "|inkedin.com" for "linkedin.com" and in the process, divulge personal details to the fake website. Current State of The Art (SOTA) typically make use of string comparison algorithms (e.g. Levenshtein Distance), which are computationally heavy. One reason for this is the lack of publicly available datasets thus hindering the training of more advanced Machine Learning (ML) models. Furthermore, no one font is able to render all types of punycode correctly, posing a significant challenge to the creation of a dataset that is unbiased toward any particular font. This coupled with the vast number of internet domains pose a challenge in creating a dataset that can capture all possible variations. Here, we show how a conditional Generative Adversarial Network (GAN), PhishGAN, can be used to generate images of hieroglyphs, conditioned on non-homoglpyh input text images. Practical changes to current SOTA were required to facilitate the generation of more varied homoglyph text-based images. We also demonstrate a workflow of how PhishGAN together with a Homoglyph Identifier (HI) model can be used to identify the domain the homoglyph was trying to imitate. Furthermore, we demonstrate how PhishGAN's ability to generate datasets on the fly facilitate the quick adaptation of cybersecurity systems to detect new threats as they emerge.
△ Less
Submitted 28 September, 2020; v1 submitted 24 June, 2020;
originally announced June 2020.
-
Machine Learning and Control Theory
Authors:
Alain Bensoussan,
Yiqun Li,
Dinh Phan Cao Nguyen,
Minh-Binh Tran,
Sheung Chi Phillip Yam,
Xiang Zhou
Abstract:
We survey in this article the connections between Machine Learning and Control Theory. Control Theory provide useful concepts and tools for Machine Learning. Conversely Machine Learning can be used to solve large control problems. In the first part of the paper, we develop the connections between reinforcement learning and Markov Decision Processes, which are discrete time control problems. In the…
▽ More
We survey in this article the connections between Machine Learning and Control Theory. Control Theory provide useful concepts and tools for Machine Learning. Conversely Machine Learning can be used to solve large control problems. In the first part of the paper, we develop the connections between reinforcement learning and Markov Decision Processes, which are discrete time control problems. In the second part, we review the concept of supervised learning and the relation with static optimization. Deep learning which extends supervised learning, can be viewed as a control problem. In the third part, we present the links between stochastic gradient descent and mean-field theory. Conversely, in the fourth and fifth parts, we review machine learning approaches to stochastic control problems, and focus on the deterministic case, to explain, more easily, the numerical algorithms.
△ Less
Submitted 9 June, 2020;
originally announced June 2020.
-
Control on Hilbert Spaces and Application to Some Mean Field Type Control Problems
Authors:
Alain Bensoussan,
P. Jameson Graber,
Sheung Chi Phillip Yam
Abstract:
We propose a new approach to studying classical solutions of the Bellman equation and Master equation for mean field type control problems, using a novel form of the "lifting" idea introduced by P.-L. Lions. Rather than studying the usual system of Hamilton-Jacobi/Fokker-Planck PDEs using analytic techniques, we instead study a stochastic control problem on a specially constructed Hilbert space, w…
▽ More
We propose a new approach to studying classical solutions of the Bellman equation and Master equation for mean field type control problems, using a novel form of the "lifting" idea introduced by P.-L. Lions. Rather than studying the usual system of Hamilton-Jacobi/Fokker-Planck PDEs using analytic techniques, we instead study a stochastic control problem on a specially constructed Hilbert space, which is reminiscent of a tangent space on the Wasserstein space in optimal transport. On this Hilbert space we can use classical control theory techniques, despite the fact that it is infinite dimensional. A consequence of our construction is that the mean field type control problem appears as a special case. Thus we preserve the advantages of the lifiting procedure, while removing some of the difficulties. Our approach extends previous work by two of the coauthors, which dealt with a deterministic control problem for which the Hilbert space could be generic.
△ Less
Submitted 9 May, 2023; v1 submitted 21 May, 2020;
originally announced May 2020.
-
Mean Field approach to stochastic control with partial information
Authors:
Alain Bensoussan,
Sheung Chi Phillip Yam
Abstract:
The classical stochastic control problem under partial information can be formulated as a control problem for Zakai equation, whose solution is the unnormalized conditional probability distribution of the state of the system. Zakai equation is a stochastic Fokker-Planck equation. Therefore, the problem to be solved is similar to that met in Mean Field Control theory. Since Mean Field Control theor…
▽ More
The classical stochastic control problem under partial information can be formulated as a control problem for Zakai equation, whose solution is the unnormalized conditional probability distribution of the state of the system. Zakai equation is a stochastic Fokker-Planck equation. Therefore, the problem to be solved is similar to that met in Mean Field Control theory. Since Mean Field Control theory is much posterior to the development of Stochastic Control with partial information, the tools, techniques, and concepts obtained in the last decade, for Mean Field Games and Mean field type Control theory, have not been used for the control of Zakai equation. Our objective is to connect the two theories. We get the power of new tools, and we get new insights for the problem of stochastic control with partial information. For mean field theory, we get new interesting applications, but also new problems. Indeed, Mean Field Control Theory leads to very complex equations, like the Master equation, which is a nonlinear infinite dimensional P.D.E., for which general theorems are hardly available, although active research in this direction is performed. Direct methods are useful to obtain regularity results. We will develop in detail the LQ regulator problem, but since we cannot just consider the Gaussian case, well-known results, such as the separation principle is not available. An important result is available in the literature, due to A. Makowsky. It describes the solution of Zakai equation for linear systems with general initial condition (non-gaussian). We show that the separation principle can be extended for quadratic pay-off functionals, but the Kalman filter is much more complex than in the gaussian case. Finally we compare our work to the work of E. Bandini et al. and we show that the example E. Bandini et al. provided does not cover ours. Our system remains nonlinear in their setting.
△ Less
Submitted 26 September, 2019; v1 submitted 23 September, 2019;
originally announced September 2019.
-
Stochastic Control on Space of Random Variables
Authors:
Alain Bensoussan,
P. Jameson Graber,
S. C. P. Yam
Abstract:
By extending \cite{bensoussan2015control}, we implement the proposal of Lions \cite{lions14} on studying mean field games and their master equations via certain control problems on the Hilbert space of square integrable random variables. In \cite{bensoussan2015control}, the Hilbert space could be quite general in the face of the "deterministic control problem" due to the absence of additional rand…
▽ More
By extending \cite{bensoussan2015control}, we implement the proposal of Lions \cite{lions14} on studying mean field games and their master equations via certain control problems on the Hilbert space of square integrable random variables. In \cite{bensoussan2015control}, the Hilbert space could be quite general in the face of the "deterministic control problem" due to the absence of additional randomness; while the special case of $L^2$ space of square integrable random variables was brought in at the interpretation stage. The effectiveness of the approach was demonstrated by deriving Bellman equations and the first order master equations through control theory of dynamical systems valued in the Hilbert space. In our present problem for second order master equations, it connects with a stochastic control problem over the space of random variables, and it possesses an additional randomness generated by the Wiener process which cannot be detached from the randomness caused by the elements in the Hilbert space. Nevertheless, we demonstrate how to tackle this difficulty, while preserving most of the efficiency of the approach suggested by Lions \cite{lions14}.
△ Less
Submitted 29 March, 2019;
originally announced March 2019.
-
Risk-Sensitive Mean-Field-Type Control
Authors:
Alain Bensoussan,
Boualem Djehiche,
Hamidou Tembine,
Phillip Yam
Abstract:
We study risk-sensitive optimal control of a stochastic differential equation (SDE) of mean-field type, where the coefficients are allowed to depend on some functional of the law as well as the state and control processes. Moreover the risk-sensitive cost functional is also of mean-field type. We derive optimality equations in infinite dimensions connecting dual functions associated with Bellman f…
▽ More
We study risk-sensitive optimal control of a stochastic differential equation (SDE) of mean-field type, where the coefficients are allowed to depend on some functional of the law as well as the state and control processes. Moreover the risk-sensitive cost functional is also of mean-field type. We derive optimality equations in infinite dimensions connecting dual functions associated with Bellman functional to the adjoint process of the Pontryagin maximum principle. The case of linear-exponentiated quadratic cost and its connection with the risk-neutral solution is discussed.
△ Less
Submitted 4 February, 2017;
originally announced February 2017.
-
Efficient nonparametric estimation of causal mediation effects
Authors:
K. C. G. Chan,
K. Imai,
S. C. P. Yam,
Z. Zhang
Abstract:
An essential goal of program evaluation and scientific research is the investigation of causal mechanisms. Over the past several decades, causal mediation analysis has been used in medical and social sciences to decompose the treatment effect into the natural direct and indirect effects. However, all of the existing mediation analysis methods rely on parametric modeling assumptions in one way or a…
▽ More
An essential goal of program evaluation and scientific research is the investigation of causal mechanisms. Over the past several decades, causal mediation analysis has been used in medical and social sciences to decompose the treatment effect into the natural direct and indirect effects. However, all of the existing mediation analysis methods rely on parametric modeling assumptions in one way or another, typically requiring researchers to specify multiple regression models involving the treatment, mediator, outcome, and pre-treatment confounders. To overcome this limitation, we propose a novel nonparametric estimation method for causal mediation analysis that eliminates the need for applied researchers to model multiple conditional distributions. The proposed method balances a certain set of empirical moments between the treatment and control groups by weighting each observation; in particular, we establish that the proposed estimator is globally semiparametric efficient. We also show how to consistently estimate the asymptotic variance of the proposed estimator without additional efforts. Finally, we extend the proposed method to other relevant settings including the causal mediation analysis with multiple mediators.
△ Less
Submitted 14 January, 2016;
originally announced January 2016.
-
Control Problem on Space of Random Variables and Master Equation
Authors:
Alain Bensoussan,
Phillip Yam
Abstract:
We study in this paper a control problem in a space of random variables. We show that its Hamilton Jacobi Bellman equation is related to the Master equation in Mean field theory. P.L. Lions in [14,15] introduced the Hilbert space of square integrable random variables as a natural space for writing the Master equation which appears in the mean field theory. W. Gangbo and A. Święch [10] considered t…
▽ More
We study in this paper a control problem in a space of random variables. We show that its Hamilton Jacobi Bellman equation is related to the Master equation in Mean field theory. P.L. Lions in [14,15] introduced the Hilbert space of square integrable random variables as a natural space for writing the Master equation which appears in the mean field theory. W. Gangbo and A. Święch [10] considered this type of equation in the space of probability measures equipped with the Wasserstein metric and use the concept of Wasserstein gradient. We compare the two approaches and provide some extension of the results of Gangbo and Święch.
△ Less
Submitted 4 August, 2015;
originally announced August 2015.
-
On The Interpretation Of The Master Equation
Authors:
Alain Bensoussan,
Jens Frehse,
Phillip Yam
Abstract:
Since its introduction by P.L. Lions in his lectures and seminars at the College de France, see [9], and also the very helpful notes of Cardialaguet [4] on Lions' lectures, the Master Equation has attracted a lot of interest, and various points of view have been expressed, see for example Carmona-Delarue [5], Bensoussan-Frehse-Yam [2], Buckdahn-Li-Peng-Rainer [3]. There are several ways to introdu…
▽ More
Since its introduction by P.L. Lions in his lectures and seminars at the College de France, see [9], and also the very helpful notes of Cardialaguet [4] on Lions' lectures, the Master Equation has attracted a lot of interest, and various points of view have been expressed, see for example Carmona-Delarue [5], Bensoussan-Frehse-Yam [2], Buckdahn-Li-Peng-Rainer [3]. There are several ways to introduce this type of equation; and in those mentioned works, they involve an argument which is a probability measure, while P.L. Lions has recently proposed the idea of working with the Hilbert space of square integrable random variables. Hence writing the equation is an issue; while another issue is its origin. In this article, we discuss all these various aspects, and our modeling argument relies heavily on a seminar at College de France delivered by P.L. Lions on November 14, 2014.
△ Less
Submitted 26 March, 2015;
originally announced March 2015.
-
Oracle, Multiple Robust and Multipurpose Calibration in a Missing Response Problem
Authors:
Kwun Chuen Gary Chan,
Sheung Chi Phillip Yam
Abstract:
In the presence of a missing response, reweighting the complete case subsample by the inverse of nonmissing probability is both intuitive and easy to implement. When the population totals of some auxiliary variables are known and when the inclusion probabilities are known by design, survey statisticians have developed calibration methods for improving efficiencies of the inverse probability weight…
▽ More
In the presence of a missing response, reweighting the complete case subsample by the inverse of nonmissing probability is both intuitive and easy to implement. When the population totals of some auxiliary variables are known and when the inclusion probabilities are known by design, survey statisticians have developed calibration methods for improving efficiencies of the inverse probability weighting estimators and the methods can be applied to missing data analysis. Model-based calibration has been proposed in the survey sampling literature, where multidimensional auxiliary variables are first summarized into a predictor function from a working regression model. Usually, one working model is being proposed for each parameter of interest and results in different sets of calibration weights for estimating different parameters. This paper considers calibration using multiple working regression models for estimating a single or multiple parameters. Contrary to a common belief that overfitting hurts efficiency, we present three rather unexpected results. First, when the missing probability is correctly specified and multiple working regression models for the conditional mean are posited, calibration enjoys an oracle property: the same semiparametric efficiency bound is attained as if the true outcome model is known in advance. Second, when the missing data mechanism is misspecified, calibration can still be a consistent estimator when any one of the outcome regression models is correctly specified. Third, a common set of calibration weights can be used to improve efficiency in estimating multiple parameters of interest and can simultaneously attain semiparametric efficiency bounds for all parameters of interest. We provide connections of a wide class of calibration estimators, constructed based on generalized empirical likelihood, to many existing estimators in biostatistics, econometrics and survey sampling and perform simulation studies to show that the finite sample properties of calibration estimators conform well with the theoretical results being studied.
△ Less
Submitted 15 October, 2014;
originally announced October 2014.
-
Higher Order, Polar and Sz.-Nagy's Generalized Derivatives of Random Polynomials with Independent and Identically Distributed Zeros on the Unit Circle
Authors:
Pak-Leong Cheung,
Tuen Wai Ng,
Jonathan Tsai,
S. C. P. Yam
Abstract:
For random polynomials with i.i.d. (independent and identically distribu-ted) zeros following any common probability distribution $μ$ with support contained in the unit circle, the empirical measures of the zeros of their first and higher order derivatives will be proved to converge weakly to $μ$ a.s. (almost sure(ly)). This, in particular, completes a recent work of Subramanian on the first order…
▽ More
For random polynomials with i.i.d. (independent and identically distribu-ted) zeros following any common probability distribution $μ$ with support contained in the unit circle, the empirical measures of the zeros of their first and higher order derivatives will be proved to converge weakly to $μ$ a.s. (almost sure(ly)). This, in particular, completes a recent work of Subramanian on the first order derivative case where $μ$ was assumed to be non-uniform. The same a.s. weak convergence will also be shown for polar and Sz.-Nagy's generalized derivatives, on some mild conditions.
△ Less
Submitted 25 September, 2014;
originally announced September 2014.
-
Linear-Quadratic Mean Field Games
Authors:
Alain Bensoussan,
Joseph Sung,
Phillip Yam,
Siu Pang Yung
Abstract:
In this article, we provide a comprehensive study of the linear-quadratic mean field games via the adjoint equation approach; although the problem has been considered in the literature by Huang, Caines and Malhame (HCM, 2007a), their method is based on Dynamic Programming. It turns out that two methods are not equivalent, as far as giving sufficient condition for the existence of a solution is con…
▽ More
In this article, we provide a comprehensive study of the linear-quadratic mean field games via the adjoint equation approach; although the problem has been considered in the literature by Huang, Caines and Malhame (HCM, 2007a), their method is based on Dynamic Programming. It turns out that two methods are not equivalent, as far as giving sufficient condition for the existence of a solution is concerned. Due to the linearity of the adjoint equations, the optimal mean field term satisfies a linear forward-backward ordinary differential equation. For the one dimensional case, we show that the equilibrium strategy always exists uniquely. For dimension greater than one, by choosing a suitable norm and then applying the Banach Fixed Point Theorem, a sufficient condition, which is independent of the solution of the standard Riccati differential equation, for the unique existence of the equilibrium strategy is provided. As a by-product, we also establish a neat and instructive sufficient condition for the unique existence of the solution for a class of non-trivial nonsymmetric Riccati equations. Numerical examples of non-existence of the equilibrium strategy and the comparison of HCM's approach will also be provided.
△ Less
Submitted 23 April, 2014;
originally announced April 2014.
-
The Master Equation in Mean Field Theory
Authors:
Alain Bensoussan,
Jens Frehse,
Phillip Yam
Abstract:
In his lectures at College de France, P.L. Lions introduced the concept of Master equation, see [5] for Mean Field Games. It is introduced in a heuristic fashion, from the system of partial differential equations, associated to a Nash equilibrium for a large, but finite, number of players. The method, also explained in[2], consists in a formal analogy of terms. The interest of this equation is tha…
▽ More
In his lectures at College de France, P.L. Lions introduced the concept of Master equation, see [5] for Mean Field Games. It is introduced in a heuristic fashion, from the system of partial differential equations, associated to a Nash equilibrium for a large, but finite, number of players. The method, also explained in[2], consists in a formal analogy of terms. The interest of this equation is that it contains interesting particular cases, which can be studied directly, in particular the system of HJB-FP (Hamilton-Jacobi-Bellman, Fokker-Planck) equations obtained as the limit of the finite Nash equilibrium game, when the trajectories are independent, see [4]. Usually, in mean field theory, one can bypass the large Nash equilibrium, by introducing the concept of representative agent, whose action is influenced by a distribution of similar agents, and obtains directly the system of HJB-FP equations of interest, see for instance [1]. Apparently, there is no such approach for the Master equation. We show here that it is possible. We first do it for the Mean Field type control problem, for which we interpret completely the Master equation. For the Mean Field Games itself, we solve a related problem, and obtain again the Master equation.
△ Less
Submitted 5 November, 2014; v1 submitted 16 April, 2014;
originally announced April 2014.
-
Mean Field Games with a Dominating Player
Authors:
Alain Bensoussan,
Michael Chau,
Phillip Yam
Abstract:
In this article, we consider mean field games between a dominating player and a group of representative agents, each of which acts similarly and also interacts with each other through a mean field term being substantially influenced by the dominating player. We first provide the general theory and discuss the necessary condition for the optimal controls and game condition by adopting adjoint equat…
▽ More
In this article, we consider mean field games between a dominating player and a group of representative agents, each of which acts similarly and also interacts with each other through a mean field term being substantially influenced by the dominating player. We first provide the general theory and discuss the necessary condition for the optimal controls and game condition by adopting adjoint equation approach. We then present a special case in the context of linear-quadratic framework, in which a necessary and sufficient condition can be asserted by stochastic maximum principle; we finally establish the sufficient condition that guarantees the unique existence of the equilibrium control. The proof of the convergence result of finite player game to mean field counterpart is provided in Appendix.
△ Less
Submitted 25 July, 2014; v1 submitted 16 April, 2014;
originally announced April 2014.
-
Conformal invariance of the exploration path in 2-d critical bond percolation in the square lattice
Authors:
Jonathan Tsai,
S. C. P. Yam,
Wang Zhou
Abstract:
In this paper we present the proof of the convergence of the critical bond percolation exploration process on the square lattice to the trace of SLE$_{6}$. This is an important conjecture in mathematical physics and probability. The case of critical site percolation on the hexagonal lattice was established in the seminal work of Smirnov via proving Cardy's formula. Our proof uses a series of trans…
▽ More
In this paper we present the proof of the convergence of the critical bond percolation exploration process on the square lattice to the trace of SLE$_{6}$. This is an important conjecture in mathematical physics and probability. The case of critical site percolation on the hexagonal lattice was established in the seminal work of Smirnov via proving Cardy's formula. Our proof uses a series of transformations and conditioning to construct a pair of paths: the $+\partial$CBP and the $-\partial$CBP. The convergence in the site percolation case on the hexagonal lattice allows us to obtain certain estimates on the scaling limit of the $+\partial$CBP and the $-\partial$CBP. By considering a path which is the concatenation of $+\partial$CBPs and $-\partial$CBPs in an alternating manner, we can prove the convergence in the case of bond percolation on the square lattice.
△ Less
Submitted 10 January, 2013; v1 submitted 8 December, 2011;
originally announced December 2011.
