-
Stochastic Delay Differential Games: Financial Modeling and Machine Learning Algorithms
Authors:
Robert Balkin,
Hector D. Ceniceros,
Ruimeng Hu
Abstract:
In this paper, we propose a numerical methodology for finding the closed-loop Nash equilibrium of stochastic delay differential games through deep learning. These games are prevalent in finance and economics where multi-agent interaction and delayed effects are often desired features in a model, but are introduced at the expense of increased dimensionality of the problem. This increased dimensiona…
▽ More
In this paper, we propose a numerical methodology for finding the closed-loop Nash equilibrium of stochastic delay differential games through deep learning. These games are prevalent in finance and economics where multi-agent interaction and delayed effects are often desired features in a model, but are introduced at the expense of increased dimensionality of the problem. This increased dimensionality is especially significant as that arising from the number of players is coupled with the potential infinite dimensionality caused by the delay. Our approach involves parameterizing the controls of each player using distinct recurrent neural networks. These recurrent neural network-based controls are then trained using a modified version of Brown's fictitious play, incorporating deep learning techniques. To evaluate the effectiveness of our methodology, we test it on finance-related problems with known solutions. Furthermore, we also develop new problems and derive their analytical Nash equilibrium solutions, which serve as additional benchmarks for assessing the performance of our proposed deep learning approach.
△ Less
Submitted 12 July, 2023;
originally announced July 2023.
-
Systemic Risk Models for Disjoint and Overlap** Groups with Equilibrium Strategies
Authors:
Yichen Feng,
Jean-Pierre Fouque,
Ruimeng Hu,
Tomoyuki Ichiba
Abstract:
We analyze the systemic risk for disjoint and overlap** groups (e.g., central clearing counterparties (CCP)) by proposing new models with realistic game features. Specifically, we generalize the systemic risk measure proposed in [F. Biagini, J.-P. Fouque, M. Frittelli, and T. Meyer-Brandis, Finance and Stochastics, 24(2020), 513--564] by allowing individual banks to choose their preferred groups…
▽ More
We analyze the systemic risk for disjoint and overlap** groups (e.g., central clearing counterparties (CCP)) by proposing new models with realistic game features. Specifically, we generalize the systemic risk measure proposed in [F. Biagini, J.-P. Fouque, M. Frittelli, and T. Meyer-Brandis, Finance and Stochastics, 24(2020), 513--564] by allowing individual banks to choose their preferred groups instead of being assigned to certain groups. We introduce the concept of Nash equilibrium for these new models, and analyze the optimal solution under Gaussian distribution of the risk factor. We also provide an explicit solution for the risk allocation of the individual banks, and study the existence and uniqueness of Nash equilibrium both theoretically and numerically. The developed numerical algorithm can simulate scenarios of equilibrium, and we apply it to study the bank-CCP structure with real data and show the validity of the proposed model.
△ Less
Submitted 1 February, 2022;
originally announced February 2022.
-
Sub- and Super-solution Approach to Accuracy Analysis of Portfolio Optimization Asymptotics in Multiscale Stochastic Factor Market
Authors:
Jean-Pierre Fouque,
Ruimeng Hu,
Ronnie Sircar
Abstract:
The problem of portfolio optimization when stochastic factors drive returns and volatilities has been studied in previous works by the authors. In particular, they proposed asymptotic approximations for value functions and optimal strategies in the regime where these factors are running on both slow and fast timescales. However, the rigorous justification of the accuracy of these approximations ha…
▽ More
The problem of portfolio optimization when stochastic factors drive returns and volatilities has been studied in previous works by the authors. In particular, they proposed asymptotic approximations for value functions and optimal strategies in the regime where these factors are running on both slow and fast timescales. However, the rigorous justification of the accuracy of these approximations has been limited to power utilities and a single factor. In this paper, we provide an accurate analysis for cases with general utility functions and two timescale factors by constructing sub- and super-solutions to the fully nonlinear problem so that their difference is at the desired level of accuracy. This approach will be valuable in various related stochastic control problems.
△ Less
Submitted 13 October, 2021; v1 submitted 21 June, 2021;
originally announced June 2021.
-
Signatured Deep Fictitious Play for Mean Field Games with Common Noise
Authors:
Ming Min,
Ruimeng Hu
Abstract:
Existing deep learning methods for solving mean-field games (MFGs) with common noise fix the sampling common noise paths and then solve the corresponding MFGs. This leads to a nested-loop structure with millions of simulations of common noise paths in order to produce accurate solutions, which results in prohibitive computational cost and limits the applications to a large extent. In this paper, b…
▽ More
Existing deep learning methods for solving mean-field games (MFGs) with common noise fix the sampling common noise paths and then solve the corresponding MFGs. This leads to a nested-loop structure with millions of simulations of common noise paths in order to produce accurate solutions, which results in prohibitive computational cost and limits the applications to a large extent. In this paper, based on the rough path theory, we propose a novel single-loop algorithm, named signatured deep fictitious play, by which we can work with the unfixed common noise setup to avoid the nested-loop structure and reduce the computational complexity significantly. The proposed algorithm can accurately capture the effect of common uncertainty changes on mean-field equilibria without further training of neural networks, as previously needed in the existing machine learning algorithms. The efficiency is supported by three applications, including linear-quadratic MFGs, mean-field portfolio game, and mean-field game of optimal consumption and investment. Overall, we provide a new point of view from the rough path theory to solve MFGs with common noise with significantly improved efficiency and an extensive range of applications. In addition, we report the first deep learning work to deal with extended MFGs (a mean-field interaction via both the states and controls) with common noise.
△ Less
Submitted 6 June, 2021;
originally announced June 2021.
-
$N$-player and Mean-field Games in Itô-diffusion Markets with Competitive or Homophilous Interaction
Authors:
Ruimeng Hu,
Thaleia Zariphopoulou
Abstract:
In Itô-diffusion environments, we introduce and analyze $N$-player and common-noise mean-field games in the context of optimal portfolio choice in a common market. The players invest in a finite horizon and also interact, driven either by competition or homophily. We study an incomplete market model in which the players have constant individual risk tolerance coefficients (CARA utilities). We also…
▽ More
In Itô-diffusion environments, we introduce and analyze $N$-player and common-noise mean-field games in the context of optimal portfolio choice in a common market. The players invest in a finite horizon and also interact, driven either by competition or homophily. We study an incomplete market model in which the players have constant individual risk tolerance coefficients (CARA utilities). We also consider the general case of random individual risk tolerances and analyze the related games in a complete market setting. This randomness makes the problem substantially more complex as it leads to ($N$ or a continuum of) auxiliary ''individual'' Itô-diffusion markets. For all cases, we derive explicit or closed-form solutions for the equilibrium stochastic processes, the optimal state processes, and the values of the games.
△ Less
Submitted 14 June, 2021; v1 submitted 1 June, 2021;
originally announced June 2021.
-
Recurrent Neural Networks for Stochastic Control Problems with Delay
Authors:
Jiequn Han,
Ruimeng Hu
Abstract:
Stochastic control problems with delay are challenging due to the path-dependent feature of the system and thus its intrinsic high dimensions. In this paper, we propose and systematically study deep neural networks-based algorithms to solve stochastic control problems with delay features. Specifically, we employ neural networks for sequence modeling (\emph{e.g.}, recurrent neural networks such as…
▽ More
Stochastic control problems with delay are challenging due to the path-dependent feature of the system and thus its intrinsic high dimensions. In this paper, we propose and systematically study deep neural networks-based algorithms to solve stochastic control problems with delay features. Specifically, we employ neural networks for sequence modeling (\emph{e.g.}, recurrent neural networks such as long short-term memory) to parameterize the policy and optimize the objective function. The proposed algorithms are tested on three benchmark examples: a linear-quadratic problem, optimal consumption with fixed finite delay, and portfolio optimization with complete memory. Particularly, we notice that the architecture of recurrent neural networks naturally captures the path-dependent feature with much flexibility and yields better performance with more efficient and stable training of the network compared to feedforward networks. The superiority is even evident in the case of portfolio optimization with complete memory, which features infinite delay.
△ Less
Submitted 16 June, 2021; v1 submitted 5 January, 2021;
originally announced January 2021.
-
Convergence of Deep Fictitious Play for Stochastic Differential Games
Authors:
Jiequn Han,
Ruimeng Hu,
Jihao Long
Abstract:
Stochastic differential games have been used extensively to model agents' competitions in Finance, for instance, in P2P lending platforms from the Fintech industry, the banking system for systemic risk, and insurance markets. The recently proposed machine learning algorithm, deep fictitious play, provides a novel efficient tool for finding Markovian Nash equilibrium of large $N$-player asymmetric…
▽ More
Stochastic differential games have been used extensively to model agents' competitions in Finance, for instance, in P2P lending platforms from the Fintech industry, the banking system for systemic risk, and insurance markets. The recently proposed machine learning algorithm, deep fictitious play, provides a novel efficient tool for finding Markovian Nash equilibrium of large $N$-player asymmetric stochastic differential games [J. Han and R. Hu, Mathematical and Scientific Machine Learning Conference, pages 221-245, PMLR, 2020]. By incorporating the idea of fictitious play, the algorithm decouples the game into $N$ sub-optimization problems, and identifies each player's optimal strategy with the deep backward stochastic differential equation (BSDE) method parallelly and repeatedly. In this paper, we prove the convergence of deep fictitious play (DFP) to the true Nash equilibrium. We can also show that the strategy based on DFP forms an $\eps$-Nash equilibrium. We generalize the algorithm by proposing a new approach to decouple the games, and present numerical results of large population games showing the empirical convergence of the algorithm beyond the technical assumptions in the theorems.
△ Less
Submitted 21 March, 2021; v1 submitted 12 August, 2020;
originally announced August 2020.
-
Deep Fictitious Play for Finding Markovian Nash Equilibrium in Multi-Agent Games
Authors:
Jiequn Han,
Ruimeng Hu
Abstract:
We propose a deep neural network-based algorithm to identify the Markovian Nash equilibrium of general large $N$-player stochastic differential games. Following the idea of fictitious play, we recast the $N$-player game into $N$ decoupled decision problems (one for each player) and solve them iteratively. The individual decision problem is characterized by a semilinear Hamilton-Jacobi-Bellman equa…
▽ More
We propose a deep neural network-based algorithm to identify the Markovian Nash equilibrium of general large $N$-player stochastic differential games. Following the idea of fictitious play, we recast the $N$-player game into $N$ decoupled decision problems (one for each player) and solve them iteratively. The individual decision problem is characterized by a semilinear Hamilton-Jacobi-Bellman equation, to solve which we employ the recently developed deep BSDE method. The resulted algorithm can solve large $N$-player games for which conventional numerical methods would suffer from the curse of dimensionality. Multiple numerical examples involving identical or heterogeneous agents, with risk-neutral or risk-sensitive objectives, are tested to validate the accuracy of the proposed algorithm in large group games. Even for a fifty-player game with the presence of common noise, the proposed algorithm still finds the approximate Nash equilibrium accurately, which, to our best knowledge, is difficult to achieve by other numerical algorithms.
△ Less
Submitted 4 June, 2020; v1 submitted 4 December, 2019;
originally announced December 2019.
-
Multiscale Asymptotic Analysis for Portfolio Optimization under Stochastic Environment
Authors:
Jean-Pierre Fouque,
Ruimeng Hu
Abstract:
Empirical studies indicate the presence of multi-scales in the volatility of underlying assets: a fast-scale on the order of days and a slow-scale on the order of months. In our previous works, we have studied the portfolio optimization problem in a Markovian setting under each single scale, the slow one in [Fouque and Hu, SIAM J. Control Optim., 55 (2017), 1990-2023], and the fast one in [Hu, Pro…
▽ More
Empirical studies indicate the presence of multi-scales in the volatility of underlying assets: a fast-scale on the order of days and a slow-scale on the order of months. In our previous works, we have studied the portfolio optimization problem in a Markovian setting under each single scale, the slow one in [Fouque and Hu, SIAM J. Control Optim., 55 (2017), 1990-2023], and the fast one in [Hu, Proceedings of IEEE CDC 2018, accepted]. This paper is dedicated to the analysis when the two scales coexist in a Markovian setting. We study the terminal wealth utility maximization problem when the volatility is driven by both fast- and slow-scale factors. We first propose a zeroth-order strategy, and rigorously establish the first order approximation of the associated problem value. This is done by analyzing the corresponding linear partial differential equation (PDE) via regular and singular perturbation techniques, as in the single-scale cases. Then, we show the asymptotic optimality of our proposed strategy within a specific family of admissible controls. Interestingly, we highlight that a pure PDE approach does not work in the multi-scale case and, instead, we use the so-called epsilon-martingale decomposition. This completes the analysis of portfolio optimization in both fast mean-reverting and slowly-varying Markovian stochastic environments.
△ Less
Submitted 1 September, 2019; v1 submitted 18 February, 2019;
originally announced February 2019.
-
Deep Learning for Ranking Response Surfaces with Applications to Optimal Stop** Problems
Authors:
Ruimeng Hu
Abstract:
In this paper, we propose deep learning algorithms for ranking response surfaces, with applications to optimal stop** problems in financial mathematics. The problem of ranking response surfaces is motivated by estimating optimal feedback policy maps in stochastic control problems, aiming to efficiently find the index associated to the minimal response across the entire continuous input space…
▽ More
In this paper, we propose deep learning algorithms for ranking response surfaces, with applications to optimal stop** problems in financial mathematics. The problem of ranking response surfaces is motivated by estimating optimal feedback policy maps in stochastic control problems, aiming to efficiently find the index associated to the minimal response across the entire continuous input space $\mathcal{X} \subseteq \mathbb{R}^d$. By considering points in $\mathcal{X}$ as pixels and indices of the minimal surfaces as labels, we recast the problem as an image segmentation problem, which assigns a label to every pixel in an image such that pixels with the same label share certain characteristics. This provides an alternative method for efficiently solving the problem instead of using sequential design in our previous work [R. Hu and M. Ludkovski, SIAM/ASA Journal on Uncertainty Quantification, 5 (2017), 212--239].
Deep learning algorithms are scalable, parallel and model-free, i.e., no parametric assumptions needed on the response surfaces. Considering ranking response surfaces as image segmentation allows one to use a broad class of deep neural networks, e.g., UNet, SegNet, DeconvNet, which have been widely applied and numerically proved to possess high accuracy in the field. We also systematically study the dependence of deep learning algorithms on the input data generated on uniform grids or by sequential design sampling, and observe that the performance of deep learning is {\it not} sensitive to the noise and locations (close to/away from boundaries) of training data. We present a few examples including synthetic ones and the Bermudan option pricing problem to show the efficiency and accuracy of this method.
△ Less
Submitted 10 March, 2020; v1 submitted 10 January, 2019;
originally announced January 2019.
-
Portfolio Optimization under Fast Mean-reverting and Rough Fractional Stochastic Environment
Authors:
Jean-Pierre Fouque,
Ruimeng Hu
Abstract:
Fractional stochastic volatility models have been widely used to capture the non-Markovian structure revealed from financial time series of realized volatility. On the other hand, empirical studies have identified scales in stock price volatility: both fast-time scale on the order of days and slow-scale on the order of months. So, it is natural to study the portfolio optimization problem under the…
▽ More
Fractional stochastic volatility models have been widely used to capture the non-Markovian structure revealed from financial time series of realized volatility. On the other hand, empirical studies have identified scales in stock price volatility: both fast-time scale on the order of days and slow-scale on the order of months. So, it is natural to study the portfolio optimization problem under the effects of dependence behavior which we will model by fractional Brownian motions with Hurst index $H$, and in the fast or slow regimes characterized by small parameters $\eps$ or $δ$. For the slowly varying volatility with $H \in (0,1)$, it was shown that the first order correction to the problem value contains two terms of order $δ^H$, one random component and one deterministic function of state processes, while for the fast varying case with $H > \half$, the same form holds at order $\eps^{1-H}$. This paper is dedicated to the remaining case of a fast-varying rough environment ($H < \half$) which exhibits a different behavior. We show that, in the expansion, only one deterministic term of order $\sqrt{\eps}$ appears in the first order correction.
△ Less
Submitted 24 January, 2019; v1 submitted 6 April, 2018;
originally announced April 2018.
-
Asymptotic Optimal Portfolio in Fast Mean-reverting Stochastic Environments
Authors:
Ruimeng Hu
Abstract:
This paper studies the portfolio optimization problem when the investor's utility is general and the return and volatility of the risky asset are fast mean-reverting, which are important to capture the fast-time scale in the modeling of stock price volatility. Motivated by the heuristic derivation in [J.-P. Fouque, R. Sircar and T. Zariphopoulou, \emph{Mathematical Finance}, 2016], we propose a ze…
▽ More
This paper studies the portfolio optimization problem when the investor's utility is general and the return and volatility of the risky asset are fast mean-reverting, which are important to capture the fast-time scale in the modeling of stock price volatility. Motivated by the heuristic derivation in [J.-P. Fouque, R. Sircar and T. Zariphopoulou, \emph{Mathematical Finance}, 2016], we propose a zeroth order strategy, and show its asymptotic optimality within a specific (smaller) family of admissible strategies under proper assumptions. This optimality result is achieved by establishing a first order approximation of the problem value associated to this proposed strategy using singular perturbation method, and estimating the risk-tolerance functions. The results are natural extensions of our previous work on portfolio optimization in a slowly varying stochastic environment [J.-P. Fouque and R. Hu, \emph{SIAM Journal on Control and Optimization}, 2017], and together they form a whole picture of analyzing portfolio optimization in both fast and slow environments.
△ Less
Submitted 29 January, 2019; v1 submitted 20 March, 2018;
originally announced March 2018.
-
Optimal Portfolio under Fast Mean-reverting Fractional Stochastic Environment
Authors:
Jean-Pierre Fouque,
Ruimeng Hu
Abstract:
Empirical studies indicate the existence of long range dependence in the volatility of the underlying asset. This feature can be captured by modeling its return and volatility using functions of a stationary fractional Ornstein--Uhlenbeck (fOU) process with Hurst index $H \in (\frac{1}{2}, 1)$. In this paper, we analyze the nonlinear optimal portfolio allocation problem under this model and in the…
▽ More
Empirical studies indicate the existence of long range dependence in the volatility of the underlying asset. This feature can be captured by modeling its return and volatility using functions of a stationary fractional Ornstein--Uhlenbeck (fOU) process with Hurst index $H \in (\frac{1}{2}, 1)$. In this paper, we analyze the nonlinear optimal portfolio allocation problem under this model and in the regime where the fOU process is fast mean-reverting. We first consider the case of power utility, and rigorously give first order approximations of the value and the optimal strategy by a martingale distortion transformation. We also establish the asymptotic optimality in all admissible controls of a zeroth order trading strategy. Then, we extend the discussions to general utility functions using the epsilon-martingale decomposition technique, and we obtain similar asymptotic optimality results within a specific family of admissible strategies.
△ Less
Submitted 8 February, 2018; v1 submitted 9 June, 2017;
originally announced June 2017.
-
Optimal Portfolio under Fractional Stochastic Environment
Authors:
Jean-Pierre Fouque,
Ruimeng Hu
Abstract:
Rough stochastic volatility models have attracted a lot of attentions recently, in particular for the linear option pricing problem. In this paper, starting with power utilities, we propose to use a martingale distortion representation of the optimal value function for the nonlinear asset allocation problem in a (non-Markovian) fractional stochastic environment (for all Hurst index $H \in (0,1)$).…
▽ More
Rough stochastic volatility models have attracted a lot of attentions recently, in particular for the linear option pricing problem. In this paper, starting with power utilities, we propose to use a martingale distortion representation of the optimal value function for the nonlinear asset allocation problem in a (non-Markovian) fractional stochastic environment (for all Hurst index $H \in (0,1)$). We rigorously establish a first order approximation of the optimal value, where the return and volatility of the underlying asset are functions of a stationary slowly varying fractional Ornstein-Uhlenbeck process. We prove that this approximation can be also generated by a fixed zeroth order trading strategy providing an explicit strategy which is asymptotically optimal in all admissible controls. Furthermore, we extend the discussion to general utility functions, and obtain the asymptotic optimality of this fixed strategy in a specific family of admissible strategies.
△ Less
Submitted 9 December, 2017; v1 submitted 20 March, 2017;
originally announced March 2017.
-
Asymptotic Optimal Strategy for Portfolio Optimization in a Slowly Varying Stochastic Environment
Authors:
Jean-Pierre Fouque,
Ruimeng Hu
Abstract:
In this paper, we study the portfolio optimization problem with general utility functions and when the return and volatility of underlying asset are slowly varying. An asymptotic optimal strategy is provided within a specific class of admissible controls under this problem setup. Specifically, we first establish a rigorous first order approximation of the value function associated to a fixed zerot…
▽ More
In this paper, we study the portfolio optimization problem with general utility functions and when the return and volatility of underlying asset are slowly varying. An asymptotic optimal strategy is provided within a specific class of admissible controls under this problem setup. Specifically, we first establish a rigorous first order approximation of the value function associated to a fixed zeroth order suboptimal trading strategy, which is given by the heuristic argument in [J.-P. Fouque, R. Sircar and T. Zariphopoulou, {\it Mathematical Finance}, 2016]. Then, we show that this zeroth order suboptimal strategy is asymptotically optimal in a specific family of admissible trading strategies. Finally, we show that our assumptions are satisfied by a particular fully solvable model.
△ Less
Submitted 7 November, 2016; v1 submitted 11 March, 2016;
originally announced March 2016.
-
Sequential Design for Ranking Response Surfaces
Authors:
Ruimeng Hu,
Mike Ludkovski
Abstract:
We propose and analyze sequential design methods for the problem of ranking several response surfaces. Namely, given $L \ge 2$ response surfaces over a continuous input space $\cal X$, the aim is to efficiently find the index of the minimal response across the entire $\cal X$. The response surfaces are not known and have to be noisily sampled one-at-a-time. This setting is motivated by stochastic…
▽ More
We propose and analyze sequential design methods for the problem of ranking several response surfaces. Namely, given $L \ge 2$ response surfaces over a continuous input space $\cal X$, the aim is to efficiently find the index of the minimal response across the entire $\cal X$. The response surfaces are not known and have to be noisily sampled one-at-a-time. This setting is motivated by stochastic control applications and requires joint experimental design both in space and response-index dimensions. To generate sequential design heuristics we investigate stepwise uncertainty reduction approaches, as well as sampling based on posterior classification complexity. We also make connections between our continuous-input formulation and the discrete framework of pure regret in multi-armed bandits. To model the response surfaces we utilize kriging surrogates. Several numerical examples using both synthetic data and an epidemics control problem are provided to illustrate our approach and the efficacy of respective adaptive designs.
△ Less
Submitted 12 July, 2016; v1 submitted 3 September, 2015;
originally announced September 2015.
