Showing 1–50 of 62 results for author: Reisinger, C

Mean-field games of speedy information access with observation costs

Authors: Dirk Becherer, Christoph Reisinger, Jonathan Tam

Abstract: We investigate mean-field games (MFG) in which agents can actively control their speed of access to information. Specifically, the agents can dynamically decide to obtain observations with reduced delay by accepting higher observation costs. Agents seek to exploit their active information acquisition by making further decisions to influence their state dynamics so as to maximise rewards. In a mean… ▽ More We investigate mean-field games (MFG) in which agents can actively control their speed of access to information. Specifically, the agents can dynamically decide to obtain observations with reduced delay by accepting higher observation costs. Agents seek to exploit their active information acquisition by making further decisions to influence their state dynamics so as to maximise rewards. In a mean-field equilibrium, each generic agent solves individually a partially observed Markov decision problem in which the way partial observations are obtained is itself subject to dynamic control actions, while no agent can improve unilaterally given the actions of all others. Based on a finite characterisation of belief states, we show how the mean-field game with controlled costly information access can be formulated as an equivalent standard mean-field game on an augmented but finite state space. With sufficient entropy regularisation, a fixed point iteration converges to the unique MFG equilibrium. Moreover, we derive an approximate $\varepsilon$-Nash equilibrium for a large but finite population size and small regularisation parameter. We illustrate our (extended) MFG of information access and of controls by an example from epidemiology, where medical testing results can be procured at different speeds and costs. △ Less

Submitted 3 May, 2024; v1 submitted 14 September, 2023; originally announced September 2023.

Comments: 31 pages, 4 figures

MSC Class: 93C43; 93C41; 91A16

Contagious McKean--Vlasov problems with common noise: from smooth to singular feedback through hitting times

Authors: Ben Hambly, Aldaïr Petronilia, Christoph Reisinger, Stefan Rigger, Andreas Søjmark

Abstract: We consider a family of McKean-Vlasov equations arising as the large particle limit of a system of interacting particles on the positive half-line with common noise and feedback. Such systems are motivated by structural models for systemic risk with contagion. This contagious interaction is such that when a particle hits zero, the impact is to move all the others toward the origin through a kernel… ▽ More We consider a family of McKean-Vlasov equations arising as the large particle limit of a system of interacting particles on the positive half-line with common noise and feedback. Such systems are motivated by structural models for systemic risk with contagion. This contagious interaction is such that when a particle hits zero, the impact is to move all the others toward the origin through a kernel which smooths the impact over time. We study a rescaling of the impact kernel under which it converges to the Dirac delta function so that the interaction happens instantaneously and the limiting singular McKean--Vlasov equation can exhibit jumps. Our approach provides a novel method to construct solutions to such singular problems that allows for more general drift and diffusion coefficients and we establish weak convergence to relaxed solutions in this setting. With more restrictions on the coefficients we can establish an almost sure version showing convergence to strong solutions. Under some regularity conditions on the contagion, we also show a rate of convergence up to the time the regularity of the contagion breaks down. Lastly, we perform some numerical experiments to investigate the sharpness of our bounds for the rate of convergence. △ Less

Submitted 20 July, 2023; originally announced July 2023.

Comments: 43 pages, 4 figures

Randomness and early termination: what makes a game exciting?

Authors: Gaoyue Guo, Sam D. Howison, Dylan Possamaï, Christoph Reisinger

Abstract: In this paper we revisit an open problem posed by Aldous on the max-entropy win-probability martingale: given two players of equal strength, such that the win-probability is a martingale diffusion, which of these processes has maximum entropy and hence gives the most excitement for the spectators? We study a terminal-boundary value problem for the nonlinear parabolic PDE… ▽ More In this paper we revisit an open problem posed by Aldous on the max-entropy win-probability martingale: given two players of equal strength, such that the win-probability is a martingale diffusion, which of these processes has maximum entropy and hence gives the most excitement for the spectators? We study a terminal-boundary value problem for the nonlinear parabolic PDE $2\partial_te(t,x)=\log(-\partial_{xx}e(t,x))$ derived by Aldous and prove its wellposedness and regularity of its solution by combining PDE analysis and probabilistic tools, in particular the reformulation as a stochastic control problem with restricted control set, which allows us to deduce strict ellipticity. We establish key qualitative properties of the solution including concavity, monotonicity, convergence to a steady state for long remaining time and the asymptotic behaviour shortly before the terminal time. Moreover, we construct convergent numerical approximations. The analytical and numerical results allow us to highlight the behaviour of the win-probability process in the present case where the match may end early, in contrast to recent work by Backhoff-Veraguas and Beiglböck where the match always runs the full length. △ Less

Submitted 20 September, 2023; v1 submitted 12 June, 2023; originally announced June 2023.

$K$-Nearest-Neighbor Resampling for Off-Policy Evaluation in Stochastic Control

Authors: Michael Giegrich, Roel Oomen, Christoph Reisinger

Abstract: In this paper, we propose a novel $K$-nearest neighbor resampling procedure for estimating the performance of a policy from historical data containing realized episodes of a decision process generated under a different policy. We provide statistical consistency results under weak conditions. In particular, we avoid the common assumption of identically and independently distributed transitions and… ▽ More In this paper, we propose a novel $K$-nearest neighbor resampling procedure for estimating the performance of a policy from historical data containing realized episodes of a decision process generated under a different policy. We provide statistical consistency results under weak conditions. In particular, we avoid the common assumption of identically and independently distributed transitions and rewards. Instead, our analysis allows for the sampling of entire episodes, as is common practice in most applications. To establish the consistency in this setting, we generalize Stone's Theorem, a well-known result in nonparametric statistics on local averaging, to include episodic data and the counterfactual estimation underlying off-policy evaluation (OPE). By focusing on feedback policies that depend deterministically on the current state in environments with continuous state-action spaces and system-inherent stochasticity effected by chosen actions, and relying on trajectory simulation similar to Monte Carlo methods, the proposed method is particularly well suited for stochastic control environments. Compared to other OPE methods, our algorithm does not require optimization, can be efficiently implemented via tree-based nearest neighbor search and parallelization, and does not explicitly assume a parametric model for the environment's dynamics. Numerical experiments demonstrate the effectiveness of the algorithm compared to existing baselines in a variety of stochastic control settings, including a linear quadratic regulator, trade execution in limit order books, and online stochastic bin packing. △ Less

Submitted 10 January, 2024; v1 submitted 7 June, 2023; originally announced June 2023.

Convergence of policy gradient methods for finite-horizon exploratory linear-quadratic control problems

Authors: Michael Giegrich, Christoph Reisinger, Yufei Zhang

Abstract: We study the global linear convergence of policy gradient (PG) methods for finite-horizon continuous-time exploratory linear-quadratic control (LQC) problems. The setting includes stochastic LQC problems with indefinite costs and allows additional entropy regularisers in the objective. We consider a continuous-time Gaussian policy whose mean is linear in the state variable and whose covariance is… ▽ More We study the global linear convergence of policy gradient (PG) methods for finite-horizon continuous-time exploratory linear-quadratic control (LQC) problems. The setting includes stochastic LQC problems with indefinite costs and allows additional entropy regularisers in the objective. We consider a continuous-time Gaussian policy whose mean is linear in the state variable and whose covariance is state-independent. Contrary to discrete-time problems, the cost is noncoercive in the policy and not all descent directions lead to bounded iterates. We propose geometry-aware gradient descents for the mean and covariance of the policy using the Fisher geometry and the Bures-Wasserstein geometry, respectively. The policy iterates are shown to satisfy an a-priori bound, and converge globally to the optimal policy with a linear rate. We further propose a novel PG method with discrete-time policies. The algorithm leverages the continuous-time analysis, and achieves a robust linear convergence across different action frequencies. A numerical experiment confirms the convergence and robustness of the proposed algorithm. △ Less

Submitted 1 March, 2024; v1 submitted 1 November, 2022; originally announced November 2022.

Comments: To be published in SIAM Journal on Control and Optimization

MSC Class: 68Q25; 93E20

An explicit Milstein-type scheme for interacting particle systems and McKean--Vlasov SDEs with common noise and non-differentiable drift coefficients

Authors: Sani Biswas, Chaman Kumar, Neelima, Gonçalo dos Reis, Christoph Reisinger

Abstract: We propose an explicit drift-randomised Milstein scheme for both McKean--Vlasov stochastic differential equations and associated high-dimensional interacting particle systems with common noise. By using a drift-randomisation step in space and measure, we establish the scheme's strong convergence rate of $1$ under reduced regularity assumptions on the drift coefficient: no classical (Euclidean) der… ▽ More We propose an explicit drift-randomised Milstein scheme for both McKean--Vlasov stochastic differential equations and associated high-dimensional interacting particle systems with common noise. By using a drift-randomisation step in space and measure, we establish the scheme's strong convergence rate of $1$ under reduced regularity assumptions on the drift coefficient: no classical (Euclidean) derivatives in space or measure derivatives (e.g., Lions/Fréchet) are required. The main result is established by enriching the concepts of bistability and consistency of numerical schemes used previously for standard SDE. We introduce certain Spijker-type norms (and associated Banach spaces) to deal with the interaction of particles present in the stochastic systems being analysed. A discussion of the scheme's complexity is provided. △ Less

Submitted 16 June, 2023; v1 submitted 22 August, 2022; originally announced August 2022.

Comments: 36 pages including appendix; minor revision from earlier version

MSC Class: 65C05; 65C30; 65C35; 60H35

Implicit and fully discrete approximation of the supercooled Stefan problem in the presence of blow-ups

Authors: Christa Cuchiero, Christoph Reisinger, Stefan Rigger

Abstract: We consider two implicit approximation schemes of the one-dimensional supercooled Stefan problem and prove their convergence, even in the presence of finite time blow-ups. All proofs are based on a probabilistic reformulation recently considered in the literature. The first scheme is a version of the time-step** scheme studied in V. Kaushansky, C. Reisinger, M. Shkolnikov, and Z. Q. Song, arXiv:… ▽ More We consider two implicit approximation schemes of the one-dimensional supercooled Stefan problem and prove their convergence, even in the presence of finite time blow-ups. All proofs are based on a probabilistic reformulation recently considered in the literature. The first scheme is a version of the time-step** scheme studied in V. Kaushansky, C. Reisinger, M. Shkolnikov, and Z. Q. Song, arXiv:2010.05281, 2020, but here the flux over the free boundary and its velocity are coupled implicitly. Moreover, we extend the analysis to more general driving processes than Brownian motion. The second scheme is a Donsker-type approximation, also interpretable as an implicit finite difference scheme, for which global convergence is shown under minor technical conditions. With stronger assumptions, which apply in cases without blow-ups, we obtain additionally a convergence rate arbitrarily close to 1/2. Our numerical results suggest that this rate also holds for less regular solutions, in contrast to explicit schemes, and allow a sharper resolution of the discontinuous free boundary in the blow-up regime. △ Less

Submitted 29 June, 2022; originally announced June 2022.

Hedging option books using neural-SDE market models

Authors: Samuel N. Cohen, Christoph Reisinger, Sheng Wang

Abstract: We study the capability of arbitrage-free neural-SDE market models to yield effective strategies for hedging options. In particular, we derive sensitivity-based and minimum-variance-based hedging strategies using these models and examine their performance when applied to various option portfolios using real-world data. Through backtesting analysis over typical and stressed market periods, we show… ▽ More We study the capability of arbitrage-free neural-SDE market models to yield effective strategies for hedging options. In particular, we derive sensitivity-based and minimum-variance-based hedging strategies using these models and examine their performance when applied to various option portfolios using real-world data. Through backtesting analysis over typical and stressed market periods, we show that neural-SDE market models achieve lower hedging errors than Black--Scholes delta and delta-vega hedging consistently over time, and are less sensitive to the tenor choice of hedging instruments. In addition, hedging using market models leads to similar performance to hedging using Heston models, while the former tends to be more robust during stressed market periods. △ Less

Submitted 31 May, 2022; originally announced May 2022.

MSC Class: 91B28; 91B70; 62M45; 62P05

Linear convergence of a policy gradient method for some finite horizon continuous time control problems

Authors: Christoph Reisinger, Wolfgang Stockinger, Yufei Zhang

Abstract: Despite its popularity in the reinforcement learning community, a provably convergent policy gradient method for continuous space-time control problems with nonlinear state dynamics has been elusive. This paper proposes proximal gradient algorithms for feedback controls of finite-time horizon stochastic control problems. The state dynamics are nonlinear diffusions with control-affine drift, and th… ▽ More Despite its popularity in the reinforcement learning community, a provably convergent policy gradient method for continuous space-time control problems with nonlinear state dynamics has been elusive. This paper proposes proximal gradient algorithms for feedback controls of finite-time horizon stochastic control problems. The state dynamics are nonlinear diffusions with control-affine drift, and the cost functions are nonconvex in the state and nonsmooth in the control. The system noise can degenerate, which allows for deterministic control problems as special cases. We prove under suitable conditions that the algorithm converges linearly to a stationary point of the control problem, and is stable with respect to policy updates by approximate gradient steps. The convergence result justifies the recent reinforcement learning heuristics that adding entropy regularization or a fictitious discount factor to the optimization objective accelerates the convergence of policy gradient methods. The proof exploits careful regularity estimates of backward stochastic differential equations. △ Less

Submitted 23 December, 2022; v1 submitted 22 March, 2022; originally announced March 2022.

Comments: Highlight the importance of the gradient direction on the convergence analysis on page 8

MSC Class: 68Q25; 93E20; 49M05

Estimating risks of option books using neural-SDE market models

Authors: Samuel N. Cohen, Christoph Reisinger, Sheng Wang

Abstract: In this paper, we examine the capacity of an arbitrage-free neural-SDE market model to produce realistic scenarios for the joint dynamics of multiple European options on a single underlying. We subsequently demonstrate its use as a risk simulation engine for option portfolios. Through backtesting analysis, we show that our models are more computationally efficient and accurate for evaluating the V… ▽ More In this paper, we examine the capacity of an arbitrage-free neural-SDE market model to produce realistic scenarios for the joint dynamics of multiple European options on a single underlying. We subsequently demonstrate its use as a risk simulation engine for option portfolios. Through backtesting analysis, we show that our models are more computationally efficient and accurate for evaluating the Value-at-Risk (VaR) of option portfolios, with better coverage performance and less procyclicality than standard filtered historical simulation approaches. △ Less

Submitted 14 February, 2022; originally announced February 2022.

MSC Class: 91B28; 91B70; 62M45; 62P05

Markov decision processes with observation costs: framework and computation with a penalty scheme

Authors: Christoph Reisinger, Jonathan Tam

Abstract: We consider Markov decision processes where the state of the chain is only given at chosen observation times and of a cost. Optimal strategies involve the optimisation of observation times as well as the subsequent action values. We consider the finite horizon and discounted infinite horizon problems, as well as an extension with parameter uncertainty. By including the time elapsed from observatio… ▽ More We consider Markov decision processes where the state of the chain is only given at chosen observation times and of a cost. Optimal strategies involve the optimisation of observation times as well as the subsequent action values. We consider the finite horizon and discounted infinite horizon problems, as well as an extension with parameter uncertainty. By including the time elapsed from observations as part of the augmented Markov system, the value function satisfies a system of quasi-variational inequalities (QVIs). Such a class of QVIs can be seen as an extension to the interconnected obstacle problem. We prove a comparison principle for this class of QVIs, which implies uniqueness of solutions to our proposed problem. Penalty methods are then utilised to obtain arbitrarily accurate solutions. Finally, we perform numerical experiments on three applications which illustrate our framework. △ Less

Submitted 5 December, 2023; v1 submitted 19 January, 2022; originally announced January 2022.

Comments: 35 pages, 8 figures, 3 tables

MSC Class: 93C41; 49N30; 49L20; 65K15

Optimal bailout strategies resulting from the drift controlled supercooled Stefan problem

Authors: Christa Cuchiero, Christoph Reisinger, Stefan Rigger

Abstract: We consider the problem faced by a central bank which bails out distressed financial institutions that pose systemic risk to the banking sector. In a structural default model with mutual obligations, the central agent seeks to inject a minimum amount of cash in order to limit defaults to a given proportion of entities. We prove that the value of the central agent's control problem converges as the… ▽ More We consider the problem faced by a central bank which bails out distressed financial institutions that pose systemic risk to the banking sector. In a structural default model with mutual obligations, the central agent seeks to inject a minimum amount of cash in order to limit defaults to a given proportion of entities. We prove that the value of the central agent's control problem converges as the number of defaultable institutions goes to infinity, and that it satisfies a drift controlled version of the supercooled Stefan problem. We compute optimal strategies in feedback form by solving numerically a regularized version of the corresponding mean field control problem using a policy gradient method. Our simulations show that the central agent's optimal strategy is to subsidise banks whose equity values lie in a non-trivial time-dependent region. △ Less

Submitted 18 October, 2022; v1 submitted 2 November, 2021; originally announced November 2021.

A fast iterative PDE-based algorithm for feedback controls of nonsmooth mean-field control problems

Authors: Christoph Reisinger, Wolfgang Stockinger, Yufei Zhang

Abstract: We propose a PDE-based accelerated gradient algorithm for optimal feedback controls of McKean-Vlasov dynamics that involve mean-field interactions both in the state and action. The method exploits a forward-backward splitting approach and iteratively refines the approximate controls based on the gradients of smooth costs, the proximal maps of nonsmooth costs, and dynamically updated momentum param… ▽ More We propose a PDE-based accelerated gradient algorithm for optimal feedback controls of McKean-Vlasov dynamics that involve mean-field interactions both in the state and action. The method exploits a forward-backward splitting approach and iteratively refines the approximate controls based on the gradients of smooth costs, the proximal maps of nonsmooth costs, and dynamically updated momentum parameters. At each step, the state dynamics is approximated via a particle system, and the required gradient is evaluated through a coupled system of nonlocal linear PDEs. The latter is solved by finite difference approximation or neural network-based residual approximation, depending on the state dimension. We present exhaustive numerical experiments for low and high-dimensional mean-field control problems, including sparse stabilization of stochastic Cucker-Smale models, which reveal that our algorithm captures important structures of the optimal feedback control and achieves a robust performance with respect to parameter perturbation. △ Less

Submitted 1 May, 2024; v1 submitted 15 August, 2021; originally announced August 2021.

Comments: Accepted for publication by SIAM Journal on Scientific Computing

MSC Class: 49N80; 60H35; 35Q93; 93A16

Arbitrage-free neural-SDE market models

Authors: Samuel N. Cohen, Christoph Reisinger, Sheng Wang

Abstract: Modelling joint dynamics of liquid vanilla options is crucial for arbitrage-free pricing of illiquid derivatives and managing risks of option trade books. This paper develops a nonparametric model for the European options book respecting underlying financial constraints and while being practically implementable. We derive a state space for prices which are free from static (or model-independent) a… ▽ More Modelling joint dynamics of liquid vanilla options is crucial for arbitrage-free pricing of illiquid derivatives and managing risks of option trade books. This paper develops a nonparametric model for the European options book respecting underlying financial constraints and while being practically implementable. We derive a state space for prices which are free from static (or model-independent) arbitrage and study the inference problem where a model is learnt from discrete time series data of stock and option prices. We use neural networks as function approximators for the drift and diffusion of the modelled SDE system, and impose constraints on the neural nets such that no-arbitrage conditions are preserved. In particular, we give methods to calibrate \textit{neural SDE} models which are guaranteed to satisfy a set of linear inequalities. We validate our approach with numerical experiments using data generated from a Heston stochastic local volatility model. △ Less

Submitted 23 August, 2021; v1 submitted 23 May, 2021; originally announced May 2021.

MSC Class: 91B28; 91B70; 62M45; 62P05

Simulation of conditional expectations under fast mean-reverting stochastic volatility models

Authors: Andrei Cozma, Christoph Reisinger

Abstract: In this short paper, we study the simulation of a large system of stochastic processes subject to a common driving noise and fast mean-reverting stochastic volatilities. This model may be used to describe the firm values of a large pool of financial entities. We then seek an efficient estimator for the probability of a default, indicated by a firm value below a certain threshold, conditional on co… ▽ More In this short paper, we study the simulation of a large system of stochastic processes subject to a common driving noise and fast mean-reverting stochastic volatilities. This model may be used to describe the firm values of a large pool of financial entities. We then seek an efficient estimator for the probability of a default, indicated by a firm value below a certain threshold, conditional on common factors. We consider approximations where coefficients containing the fast volatility are replaced by certain ergodic averages (a type of law of large numbers), and study a correction term (of central limit theorem-type). The accuracy of these approximations is assessed by numerical simulation of pathwise losses and the estimation of payoff functions as they appear in basket credit derivatives. △ Less

Submitted 12 October, 2021; v1 submitted 17 December, 2020; originally announced December 2020.

Path regularity of coupled McKean-Vlasov FBSDEs

Authors: Christoph Reisinger, Wolfgang Stockinger, Yufei Zhang

Abstract: This paper establishes Hölder time regularity of solutions to coupled McKean-Vlasov forward-backward stochastic differential equations (MV-FBSDEs). This is not only of fundamental mathematical interest, but also essential for their numerical approximations. We show that a solution triple to a MV-FBSDE with Lipschitz coefficients is 1/2-Hölder continuous in time in the $L^p$-norm provided that it a… ▽ More This paper establishes Hölder time regularity of solutions to coupled McKean-Vlasov forward-backward stochastic differential equations (MV-FBSDEs). This is not only of fundamental mathematical interest, but also essential for their numerical approximations. We show that a solution triple to a MV-FBSDE with Lipschitz coefficients is 1/2-Hölder continuous in time in the $L^p$-norm provided that it admits a Lipschitz decoupling field. Special examples include decoupled MV-FBSDEs, coupled MV-FBSDEs with a small time horizon and coupled stochastic Pontryagin systems arsing from mean field control problems. △ Less

Submitted 12 November, 2020; originally announced November 2020.

Comments: The results in this paper replace Sections 2 and 5 of arXiv:2009.08175v1

MSC Class: 60G17; 60H07; 49N60

Well-posedness and tamed Euler schemes for McKean-Vlasov equations driven by Lévy noise

Authors: Neelima, Sani Biswas, Chaman Kumar, Gonçalo dos Reis, Christoph Reisinger

Abstract: We prove the well-posedness of solutions to McKean-Vlasov stochastic differential equations driven by Lévy noise under mild assumptions where, in particular, the Lévy measure is not required to be finite. The drift, diffusion and jump coefficients are allowed to be random, can grow super-linearly in the state variable, and all may depend on the marginal law of the solution process. We provide a pr… ▽ More We prove the well-posedness of solutions to McKean-Vlasov stochastic differential equations driven by Lévy noise under mild assumptions where, in particular, the Lévy measure is not required to be finite. The drift, diffusion and jump coefficients are allowed to be random, can grow super-linearly in the state variable, and all may depend on the marginal law of the solution process. We provide a propagation of chaos result under more relaxed conditions than those existing in the literature, and consistent with our well-posedness result. We propose a tamed Euler scheme for the associated interacting particle system and prove that the rate of its strong convergence is arbitrarily close to $1/2$. As a by-product, we also obtain the corresponding results on well-posedness, propagation of chaos and strong convergence of the tamed Euler scheme for McKean-Vlasov stochastic delay differential equations (SDDE) and McKean-Vlasov stochastic differential equations with Markovian switching (SDEwMS), both driven by Lévy noise. Furthermore, our results on tamed Euler schemes are new even for ordinary SDEs driven by Lévy noise and with super-linearly growing coefficients. △ Less

Submitted 16 October, 2020; originally announced October 2020.

Comments: 33

MSC Class: 65C05; 65C30; 65C35; 60H35

Convergence of a time-step** scheme to the free boundary in the supercooled Stefan problem

Authors: Vadim Kaushansky, Christoph Reisinger, Mykhaylo Shkolnikov, Zhuo Qun Song

Abstract: The supercooled Stefan problem and its variants describe the freezing of a supercooled liquid in physics, as well as the large system limits of systemic risk models in finance and of integrate-and-fire models in neuroscience. Adopting the physics terminology, the supercooled Stefan problem is known to feature a finite-time blow-up of the freezing rate for a wide range of initial temperature distri… ▽ More The supercooled Stefan problem and its variants describe the freezing of a supercooled liquid in physics, as well as the large system limits of systemic risk models in finance and of integrate-and-fire models in neuroscience. Adopting the physics terminology, the supercooled Stefan problem is known to feature a finite-time blow-up of the freezing rate for a wide range of initial temperature distributions in the liquid. Such a blow-up can result in a discontinuity of the liquid-solid boundary. In this paper, we prove that the natural Euler time-step** scheme applied to a probabilistic formulation of the supercooled Stefan problem converges to the liquid-solid boundary of its physical solution globally in time, in the Skorokhod M1 topology. In the course of the proof, we give an explicit bound on the rate of local convergence for the time-step** scheme. We also run numerical tests to compare our theoretical results to the practically observed convergence behavior. △ Less

Submitted 18 March, 2022; v1 submitted 11 October, 2020; originally announced October 2020.

Comments: 23 pages

MSC Class: 80A22; 35B44; 65N20; 60H30

Optimal regularity of extended mean field controls and their piecewise constant approximation

Authors: Christoph Reisinger, Wolfgang Stockinger, Yufei Zhang

Abstract: We consider the control of McKean-Vlasov dynamics whose coefficients have mean field interactions in the state and control. We show that for a class of linear-convex mean field control problems, the unique optimal open-loop control admits the optimal 1/2-Hölder regularity in time. Consequently, we prove that the value function can be approximated by one with piecewise constant controls and discret… ▽ More We consider the control of McKean-Vlasov dynamics whose coefficients have mean field interactions in the state and control. We show that for a class of linear-convex mean field control problems, the unique optimal open-loop control admits the optimal 1/2-Hölder regularity in time. Consequently, we prove that the value function can be approximated by one with piecewise constant controls and discrete-time state processes arising from Euler-Maruyama time step**, up to an order 1/2 error, and the optimal control can be approximated up to an order 1/4 error. These results are novel even for the case without mean field interaction. △ Less

Submitted 23 September, 2021; v1 submitted 17 September, 2020; originally announced September 2020.

Comments: The weak convergence of the optimal control approximation has been improved to the strong sense

MSC Class: 49N80; 49N60; 60H35; 65L70

A posteriori error estimates for fully coupled McKean-Vlasov forward-backward SDEs

Authors: Christoph Reisinger, Wolfgang Stockinger, Yufei Zhang

Abstract: Fully coupled McKean-Vlasov forward-backward stochastic differential equations (MV-FBSDEs) arise naturally from large population optimization problems. Judging the quality of given numerical solutions for MV-FBSDEs, which usually require Picard iterations and approximations of nested conditional expectations, is typically difficult. This paper proposes an a posteriori error estimator to quantify t… ▽ More Fully coupled McKean-Vlasov forward-backward stochastic differential equations (MV-FBSDEs) arise naturally from large population optimization problems. Judging the quality of given numerical solutions for MV-FBSDEs, which usually require Picard iterations and approximations of nested conditional expectations, is typically difficult. This paper proposes an a posteriori error estimator to quantify the $L^2$-approximation error of an arbitrarily generated approximation on a time grid. We establish that the error estimator is equivalent to the global approximation error between the given numerical solution and the solution of a forward Euler discretized MV-FBSDE. A crucial and challenging step in the analysis is the proof of stability of this Euler approximation to the MV-FBSDE, which is of independent interest. We further demonstrate that, for sufficiently fine time grids, the accuracy of numerical solutions for solving the continuous MV-FBSDE can also be measured by the error estimator. The error estimates justify the use of residual-based algorithms for solving MV-FBSDEs. Numerical experiments for MV-FBSDEs arising from mean field control and games confirm the effectiveness and practical applicability of the error estimator. △ Less

Submitted 7 June, 2023; v1 submitted 15 July, 2020; originally announced July 2020.

Comments: The effectiveness of the error estimator is demonstrated in high-dimensional and nonlinear examples

MSC Class: 65C30; 60H10; 65C05; 91A13

Well-posedness and numerical schemes for one-dimensional McKean-Vlasov equations and interacting particle systems with discontinuous drift

Authors: Gunther Leobacher, Christoph Reisinger, Wolfgang Stockinger

Abstract: In this paper, we first establish well-posedness results for one-dimensional McKean-Vlasov stochastic differential equations (SDEs) and related particle systems with a measure-dependent drift coefficient that is discontinuous in the spatial component, and a diffusion coefficient which is a Lipschitz function of the state only. We only require a fairly mild condition on the diffusion coefficient, n… ▽ More In this paper, we first establish well-posedness results for one-dimensional McKean-Vlasov stochastic differential equations (SDEs) and related particle systems with a measure-dependent drift coefficient that is discontinuous in the spatial component, and a diffusion coefficient which is a Lipschitz function of the state only. We only require a fairly mild condition on the diffusion coefficient, namely to be non-zero in a point of discontinuity of the drift, while we need to impose certain structural assumptions on the measure-dependence of the drift. Second, we study Euler-Maruyama type schemes for the particle system to approximate the solution of the one-dimensional McKean-Vlasov SDE. Here, we will prove strong convergence results in terms of the number of time-steps and number of particles. Due to the discontinuity of the drift, the convergence analysis is non-standard and the usual strong convergence order $1/2$ known for the Lipschitz case cannot be recovered for all schemes. △ Less

Submitted 15 March, 2022; v1 submitted 26 June, 2020; originally announced June 2020.

Comments: 33 pages, 4 figures

MSC Class: 65C20; 65C30; 65C35; 60H30; 60H35; 60K40

Well-posedness and tamed schemes for McKean-Vlasov Equations with Common Noise

Authors: Chaman Kumar, Neelima, Christoph Reisinger, Wolfgang Stockinger

Abstract: In this paper, we first establish well-posedness of McKean-Vlasov stochastic differential equations (McKean-Vlasov SDEs) with common noise, possibly with coefficients having super-linear growth in the state variable. Second, we present stable time-step** schemes for this class of McKean-Vlasov SDEs. Specifically, we propose an explicit tamed Euler and tamed Milstein scheme for an interacting par… ▽ More In this paper, we first establish well-posedness of McKean-Vlasov stochastic differential equations (McKean-Vlasov SDEs) with common noise, possibly with coefficients having super-linear growth in the state variable. Second, we present stable time-step** schemes for this class of McKean-Vlasov SDEs. Specifically, we propose an explicit tamed Euler and tamed Milstein scheme for an interacting particle system associated with the McKean-Vlasov equation. We prove stability and strong convergence of order $1/2$ and $1$, respectively. To obtain our main results, we employ techniques from calculus on the Wasserstein space. The proof for the strong convergence of the tamed Milstein scheme only requires the coefficients to be once continuously differentiable in the state and measure component. To demonstrate our theoretical findings, we present several numerical examples, including mean-field versions of the stochastic $3/2$ volatility model and the stochastic double well dynamics with multiplicative noise. △ Less

Submitted 31 May, 2020; originally announced June 2020.

Comments: 36 pages, 3 figures

MSC Class: 65C05; 65C30; 65C35; 60H35

An adaptive Euler-Maruyama scheme for McKean-Vlasov SDEs with super-linear growth and application to the mean-field FitzHugh-Nagumo model

Authors: Christoph Reisinger, Wolfgang Stockinger

Abstract: In this paper, we introduce adaptive Euler-Maruyama schemes for McKean-Vlasov stochastic differential equations (SDEs) assuming only a standard monotonicity condition on the drift and diffusion coefficients but no global Lipschitz continuity in the state variable for either, while global Lipschitz continuity is required for the measure component only. We prove moment stability of the discretised p… ▽ More In this paper, we introduce adaptive Euler-Maruyama schemes for McKean-Vlasov stochastic differential equations (SDEs) assuming only a standard monotonicity condition on the drift and diffusion coefficients but no global Lipschitz continuity in the state variable for either, while global Lipschitz continuity is required for the measure component only. We prove moment stability of the discretised processes and a strong convergence rate of $1/2$. Several numerical examples, centred around a mean-field model for FitzHugh-Nagumo neurons, illustrate that the standard uniform scheme fails and that the adaptive approach shows in most cases superior performance to tamed approximation schemes. In addition, we introduce and analyse an adaptive Milstein scheme for a certain sub-class of McKean-Vlasov SDEs with linear measure-dependence of the drift. △ Less

Submitted 31 October, 2021; v1 submitted 12 May, 2020; originally announced May 2020.

Comments: 29 pages, 12 figures

Milstein schemes and antithetic multilevel Monte Carlo sampling for delay McKean-Vlasov equations and interacting particle systems

Authors: Jianhai Bao, Christoph Reisinger, Panpan Ren, Wolfgang Stockinger

Abstract: In this paper, we first derive Milstein schemes for an interacting particle system associated with point delay McKean-Vlasov stochastic differential equations (McKean-Vlasov SDEs), possibly with a drift term exhibiting super-linear growth in the state component. We prove strong convergence of order one and moment stability, making use of techniques from variational calculus on the space of probabi… ▽ More In this paper, we first derive Milstein schemes for an interacting particle system associated with point delay McKean-Vlasov stochastic differential equations (McKean-Vlasov SDEs), possibly with a drift term exhibiting super-linear growth in the state component. We prove strong convergence of order one and moment stability, making use of techniques from variational calculus on the space of probability measures with finite second order moments. Then, we introduce an antithetic multilevel Milstein scheme, which leads to optimal complexity estimators for expected functionals of solutions to delay McKean-Vlasov equations without the need to simulate Lévy areas. △ Less

Submitted 18 June, 2023; v1 submitted 3 May, 2020; originally announced May 2020.

Comments: 32 pages, 4 figures

MSC Class: 65C20; 65C30; 65C35; 60H30

First order convergence of Milstein schemes for McKean-Vlasov equations and interacting particle systems

Authors: Jianhai Bao, Christoph Reisinger, Panpan Ren, Wolfgang Stockinger

Abstract: In this paper, we derive fully implementable first order time-step** schemes for McKean--Vlasov stochastic differential equations (McKean--Vlasov SDEs), allowing for a drift term with super-linear growth in the state component. We propose Milstein schemes for a time-discretised interacting particle system associated with the McKean--Vlasov equation and prove strong convergence of order 1 and mom… ▽ More In this paper, we derive fully implementable first order time-step** schemes for McKean--Vlasov stochastic differential equations (McKean--Vlasov SDEs), allowing for a drift term with super-linear growth in the state component. We propose Milstein schemes for a time-discretised interacting particle system associated with the McKean--Vlasov equation and prove strong convergence of order 1 and moment stability, taming the drift if only a one-sided Lipschitz condition holds. To derive our main results on strong convergence rates, we make use of calculus on the space of probability measures with finite second order moments. In addition, numerical examples are presented which support our theoretical findings. △ Less

Submitted 7 December, 2020; v1 submitted 7 April, 2020; originally announced April 2020.

Comments: 28 pages, 10 figures

MSC Class: 65C20; 65C30; 65C35; 60H30; 60H35; 60K40

Regularity and stability of feedback relaxed controls

Authors: Christoph Reisinger, Yufei Zhang

Abstract: This paper proposes a relaxed control regularization with general exploration rewards to design robust feedback controls for multi-dimensional continuous-time stochastic exit time problems. We establish that the regularized control problem admits a Hölder continuous feedback control, and demonstrate that both the value function and the feedback control of the regularized control problem are Lipsch… ▽ More This paper proposes a relaxed control regularization with general exploration rewards to design robust feedback controls for multi-dimensional continuous-time stochastic exit time problems. We establish that the regularized control problem admits a Hölder continuous feedback control, and demonstrate that both the value function and the feedback control of the regularized control problem are Lipschitz stable with respect to parameter perturbations. Moreover, we show that a pre-computed feedback relaxed control has a robust performance in a perturbed system, and derive a first-order sensitivity equation for both the value function and optimal feedback relaxed control. These stability results provide a theoretical justification for recent reinforcement learning heuristics that including an exploration reward in the optimization objective leads to more robust decision making. We finally prove first-order monotone convergence of the value functions for relaxed control problems with vanishing exploration parameters, which subsequently enables us to construct the pure exploitation strategy of the original control problem based on the feedback relaxed controls. △ Less

Submitted 23 July, 2021; v1 submitted 9 January, 2020; originally announced January 2020.

Comments: Additional comments have been included, such that the importance of stable feedback controls for reinforcement learning. The manuscript will be published in SIAM Journal on Control and Optimization

MSC Class: 3B52; 93B35; 93E20; 68Q32

Duality-based a posteriori error estimates for some approximation schemes for optimal investment problems

Authors: Athena Picarelli, Christoph Reisinger

Abstract: We consider a Markov chain approximation scheme for utility maximization problems in continuous time, which uses, in turn, a piecewise constant policy approximation, Euler-Maruyama time step**, and a Gauss-Hermite approximation of the Gaussian increments. The error estimates previously derived in Picarelli and Reisinger (2019) are asymmetric between lower and upper bounds due to the control appr… ▽ More We consider a Markov chain approximation scheme for utility maximization problems in continuous time, which uses, in turn, a piecewise constant policy approximation, Euler-Maruyama time step**, and a Gauss-Hermite approximation of the Gaussian increments. The error estimates previously derived in Picarelli and Reisinger (2019) are asymmetric between lower and upper bounds due to the control approximation and improve on known results in the literature in the lower case only. In the present paper, we use duality results to obtain a posteriori upper error bounds which are empirically of the same order as the lower bounds. The theoretical results are confirmed by our numerical tests. △ Less

Submitted 4 January, 2020; originally announced January 2020.

A neural network based policy iteration algorithm with global $H^2$-superlinear convergence for stochastic games on domains

Authors: Kazufumi Ito, Christoph Reisinger, Yufei Zhang

Abstract: In this work, we propose a class of numerical schemes for solving semilinear Hamilton-Jacobi-Bellman-Isaacs (HJBI) boundary value problems which arise naturally from exit time problems of diffusion processes with controlled drift. We exploit policy iteration to reduce the semilinear problem into a sequence of linear Dirichlet problems, which are subsequently approximated by a multilayer feedforwar… ▽ More In this work, we propose a class of numerical schemes for solving semilinear Hamilton-Jacobi-Bellman-Isaacs (HJBI) boundary value problems which arise naturally from exit time problems of diffusion processes with controlled drift. We exploit policy iteration to reduce the semilinear problem into a sequence of linear Dirichlet problems, which are subsequently approximated by a multilayer feedforward neural network ansatz. We establish that the numerical solutions converge globally in the $H^2$-norm, and further demonstrate that this convergence is superlinear, by interpreting the algorithm as an inexact Newton iteration for the HJBI equation. Moreover, we construct the optimal feedback controls from the numerical value functions and deduce convergence. The numerical schemes and convergence results are then extended to HJBI boundary value problems corresponding to controlled diffusion processes with oblique boundary reflection. Numerical experiments on the stochastic Zermelo navigation problem are presented to illustrate the theoretical results and to demonstrate the effectiveness of the method. △ Less

Submitted 13 February, 2020; v1 submitted 5 June, 2019; originally announced June 2019.

Comments: Additional numerical experiments have been included (on Pages 27-31) to show the proposed algorithm achieves a more stable and more rapid convergence than the existing neural network based methods within similar computational time

MSC Class: 82C32; 91A15; 65M12

Analysis of sparse grid multilevel estimators for multi-dimensional Zakai equations

Authors: Christoph Reisinger, Zhenru Wang

Abstract: In this article, we analyse the accuracy and computational complexity of estimators for expected functionals of the solution to multi-dimensional parabolic stochastic partial differential equations (SPDE) of Zakai-type. Here, we use the Milstein scheme for time integration and an alternating direction implicit (ADI) splitting of the spatial finite difference discretisation, coupled with the sparse… ▽ More In this article, we analyse the accuracy and computational complexity of estimators for expected functionals of the solution to multi-dimensional parabolic stochastic partial differential equations (SPDE) of Zakai-type. Here, we use the Milstein scheme for time integration and an alternating direction implicit (ADI) splitting of the spatial finite difference discretisation, coupled with the sparse grid combination technique and multilevel Monte Carlo sampling (MLMC). In the two-dimensional case, we find by detailed Fourier analysis that for a root-mean-square error (RMSE) $\varepsilon$, MLMC on sparse grids has the optimal complexity $O(\varepsilon^{-2})$, whereas MLMC on regular grids has $O(\varepsilon^{-2}(\log\varepsilon)^2)$, standard MC on sparse grids $O(\varepsilon^{-7/2}(|\log\varepsilon|)^{5/2})$, and MC on regular grids $O(\varepsilon^{-4})$. Numerical tests confirm these findings empirically. We give a discussion of the higher-dimensional setting without detailed proofs, which suggests that MLMC on sparse grids always leads to the optimal complexity, standard MC on sparse grids has a fixed complexity order independent of the dimension (up to a logarithmic term), whereas the cost of MLMC and MC on regular grids increases exponentially with the dimension. △ Less

Submitted 17 April, 2019; originally announced April 2019.

Rectified deep neural networks overcome the curse of dimensionality for nonsmooth value functions in zero-sum games of nonlinear stiff systems

Authors: Christoph Reisinger, Yufei Zhang

Abstract: In this paper, we establish that for a wide class of controlled stochastic differential equations (SDEs) with stiff coefficients, the value functions of corresponding zero-sum games can be represented by a deep artificial neural network (DNN), whose complexity grows at most polynomially in both the dimension of the state equation and the reciprocal of the required accuracy. Such nonlinear stiff sy… ▽ More In this paper, we establish that for a wide class of controlled stochastic differential equations (SDEs) with stiff coefficients, the value functions of corresponding zero-sum games can be represented by a deep artificial neural network (DNN), whose complexity grows at most polynomially in both the dimension of the state equation and the reciprocal of the required accuracy. Such nonlinear stiff systems may arise, for example, from Galerkin approximations of controlled stochastic partial differential equations (SPDEs), or controlled PDEs with uncertain initial conditions and source terms. This implies that DNNs can break the curse of dimensionality in numerical approximations and optimal control of PDEs and SPDEs. The main ingredient of our proof is to construct a suitable discrete-time system to effectively approximate the evolution of the underlying stochastic dynamics. Similar ideas can also be applied to obtain expression rates of DNNs for value functions induced by stiff systems with regime switching coefficients and driven by general Lévy noise. △ Less

Submitted 13 May, 2020; v1 submitted 15 March, 2019; originally announced March 2019.

Comments: This revised version has been accepted for publication in Analysis and Applications

MSC Class: 82C32; 41A25; 35R60

A numerical scheme for the quantile hedging problem

Authors: Cyril Bénézet, Jean-François Chassagneux, Christoph Reisinger

Abstract: We consider the numerical approximation of the quantile hedging price in a non-linear market. In a Markovian framework, we propose a numerical method based on a Piecewise Constant Policy Timestep** (PCPT) scheme coupled with a monotone finite difference approximation. We prove the convergence of our algorithm combining BSDE arguments with the Barles & Jakobsen and Barles & Souganidis approaches… ▽ More We consider the numerical approximation of the quantile hedging price in a non-linear market. In a Markovian framework, we propose a numerical method based on a Piecewise Constant Policy Timestep** (PCPT) scheme coupled with a monotone finite difference approximation. We prove the convergence of our algorithm combining BSDE arguments with the Barles & Jakobsen and Barles & Souganidis approaches for non-linear equations. In a numerical section, we illustrate the efficiency of our scheme by considering a financial example in a market with imperfections. △ Less

Submitted 28 February, 2019; originally announced February 2019.

Comments: 47 pages, 6 figures

Journal ref: SIAM J. Finan. Math. 12-1 (2021), pp. 110-157

Error estimates of penalty schemes for quasi-variational inequalities arising from impulse control problems

Authors: Christoph Reisinger, Yufei Zhang

Abstract: This paper proposes penalty schemes for a class of weakly coupled systems of Hamilton-Jacobi-Bellman quasi-variational inequalities (HJBQVIs) arising from stochastic hybrid control problems of regime-switching models with both continuous and impulse controls. We show that the solutions of the penalized equations converge monotonically to those of the HJBQVIs. We further establish that the schemes… ▽ More This paper proposes penalty schemes for a class of weakly coupled systems of Hamilton-Jacobi-Bellman quasi-variational inequalities (HJBQVIs) arising from stochastic hybrid control problems of regime-switching models with both continuous and impulse controls. We show that the solutions of the penalized equations converge monotonically to those of the HJBQVIs. We further establish that the schemes are half-order accurate for HJBQVIs with Lipschitz coefficients, and first-order accurate for equations with more regular coefficients. Moreover, we construct the action regions and optimal impulse controls based on the error estimates and the penalized solutions. The penalty schemes and convergence results are then extended to HJBQVIs with possibly negative impulse costs. We also demonstrate the convergence of monotone discretizations of the penalized equations, and establish that policy iteration applied to the discrete equation is monotonically convergent with an arbitrary initial guess in an infinite dimensional setting. Numerical examples for infinite-horizon optimal switching problems are presented to illustrate the effectiveness of the penalty schemes over the conventional direct control scheme. △ Less

Submitted 2 January, 2020; v1 submitted 23 January, 2019; originally announced January 2019.

Comments: Accepted for publication in SIAM Journal on Control and Optimization

Improved order 1/4 convergence for piecewise constant policy approximation of stochastic control problems

Authors: Espen R. Jakobsen, Athena Picarelli, Christoph Reisinger

Abstract: In N.V. Krylov, Approximating value functions for controlled degenerate diffusion processes by using piece-wise constant policies, Electron. J. Probab., 4(2), 1999, it is proved under standard assumptions that the value functions of controlled diffusion processes can be approximated with order 1/6 error by those with controls which are constant on uniform time intervals. In this note we refine the… ▽ More In N.V. Krylov, Approximating value functions for controlled degenerate diffusion processes by using piece-wise constant policies, Electron. J. Probab., 4(2), 1999, it is proved under standard assumptions that the value functions of controlled diffusion processes can be approximated with order 1/6 error by those with controls which are constant on uniform time intervals. In this note we refine the proof and show that the provable rate can be improved to 1/4, which is optimal in our setting. Moreover, we demonstrate the improvements this implies for error estimates derived by similar techniques for approximation schemes, bringing these in line with the best available results from the PDE literature. △ Less

Submitted 4 January, 2019; originally announced January 2019.

Probabilistic error analysis for some approximation schemes to optimal control problems

Authors: Athena Picarelli, Christoph Reisinger

Abstract: We introduce a class of numerical schemes for optimal control problems based on a novel Markov chain approximation, which uses, in turn, a piecewise constant policy approximation, Euler-Maruyama time step**, and a Gauss-Hermite approximation of the Gaussian increments. We provide lower error bounds of order arbitrarily close to 1/2 in time and 1/3 in space for Lipschitz viscosity solutions, coup… ▽ More We introduce a class of numerical schemes for optimal control problems based on a novel Markov chain approximation, which uses, in turn, a piecewise constant policy approximation, Euler-Maruyama time step**, and a Gauss-Hermite approximation of the Gaussian increments. We provide lower error bounds of order arbitrarily close to 1/2 in time and 1/3 in space for Lipschitz viscosity solutions, coupling probabilistic arguments with regularization techniques as introduced by Krylov. The corresponding order of the upper bounds is 1/4 in time and 1/5 in space. For sufficiently regular solutions, the order is 1 in both time and space for both bounds. Finally, we propose techniques for further improving the accuracy of the individual components of the approximation. △ Less

Submitted 4 January, 2020; v1 submitted 10 October, 2018; originally announced October 2018.

Semi-analytical solution of a McKean-Vlasov equation with feedback through hitting a boundary

Authors: Alexander Lipton, Vadim Kaushansky, Christoph Reisinger

Abstract: In this paper, we study the non-linear diffusion equation associated with a particle system where the common drift depends on the rate of absorption of particles at a boundary. We provide an interpretation as a structural credit risk model with default contagion in a large interconnected banking system. Using the method of heat potentials, we derive a coupled system of Volterra integral equations… ▽ More In this paper, we study the non-linear diffusion equation associated with a particle system where the common drift depends on the rate of absorption of particles at a boundary. We provide an interpretation as a structural credit risk model with default contagion in a large interconnected banking system. Using the method of heat potentials, we derive a coupled system of Volterra integral equations for the transition density and for the loss through absorption. An approximation by expansion is given for a small interaction parameter. We also present a numerical solution algorithm and conduct computational tests. △ Less

Submitted 25 August, 2018; v1 submitted 15 August, 2018; originally announced August 2018.

A penalty scheme for monotone systems with interconnected obstacles: convergence and error estimates

Authors: Christoph Reisinger, Yufei Zhang

Abstract: We present a novel penalty approach for a class of quasi-variational inequalities (QVIs) involving monotone systems and interconnected obstacles. We show that for any given positive switching cost, the solutions of the penalized equations converge monotonically to those of the QVIs. We estimate the penalization errors and are able to deduce that the optimal switching regions are constructed exactl… ▽ More We present a novel penalty approach for a class of quasi-variational inequalities (QVIs) involving monotone systems and interconnected obstacles. We show that for any given positive switching cost, the solutions of the penalized equations converge monotonically to those of the QVIs. We estimate the penalization errors and are able to deduce that the optimal switching regions are constructed exactly. We further demonstrate that as the switching cost tends to zero, the QVI degenerates into an equation of HJB type, which is approximated by the penalized equation at the same order (up to a log factor) as that for positive switching cost. Numerical experiments on optimal switching problems are presented to illustrate the theoretical results and to demonstrate the effectiveness of the method. △ Less

Submitted 4 July, 2019; v1 submitted 14 August, 2018; originally announced August 2018.

Comments: Accepted for publication (in this revised form) in SIAM Journal on Numerical Analysis

MSC Class: 34A38; 65M12; 65K15

Simulation of particle systems interacting through hitting times

Authors: Vadim Kaushansky, Christoph Reisinger

Abstract: We develop an Euler-type particle method for the simulation of a McKean--Vlasov equation arising from a mean-field model with positive feedback from hitting a boundary. Under assumptions on the parameters which ensure differentiable solutions, we establish convergence of order $1/2$ in the time step. Moreover, we give a modification of the scheme using Brownian bridges and local mesh refinement, w… ▽ More We develop an Euler-type particle method for the simulation of a McKean--Vlasov equation arising from a mean-field model with positive feedback from hitting a boundary. Under assumptions on the parameters which ensure differentiable solutions, we establish convergence of order $1/2$ in the time step. Moreover, we give a modification of the scheme using Brownian bridges and local mesh refinement, which improves the order to $1$. We confirm our theoretical results with numerical tests and empirically investigate cases with blow-up. △ Less

Submitted 29 May, 2018; originally announced May 2018.

A penalty scheme and policy iteration for nonlocal HJB variational inequalities with monotone drivers

Authors: Christoph Reisinger, Yufei Zhang

Abstract: We propose a class of numerical schemes for nonlocal HJB variational inequalities (HJBVIs) with monotone drivers. The solution and free boundary of the HJBVI are constructed from a sequence of penalized equations, for which a continuous dependence result is derived and the penalization error is estimated. The penalized equation is then discretized by a class of semi-implicit monotone approximation… ▽ More We propose a class of numerical schemes for nonlocal HJB variational inequalities (HJBVIs) with monotone drivers. The solution and free boundary of the HJBVI are constructed from a sequence of penalized equations, for which a continuous dependence result is derived and the penalization error is estimated. The penalized equation is then discretized by a class of semi-implicit monotone approximations. We present a novel analysis technique for the well-posedness of the discrete equation, and demonstrate the convergence of the scheme, which subsequently gives a constructive proof for the existence of a solution to the penalized equation and variational inequality. We further propose an efficient iterative algorithm with local superlinear convergence for solving the discrete equation. Numerical experiments are presented for an optimal investment problem under ambiguity and a recursive consumption-portfolio allocation problem. △ Less

Submitted 16 May, 2018; originally announced May 2018.

MSC Class: 65M06; 65M12; 62L15; 93E20; 91G80

Approximation schemes for mixed optimal stop** and control problems with nonlinear expectations and jumps

Authors: Roxana Dumitrescu, Christoph Reisinger, Yufei Zhang

Abstract: We propose a class of numerical schemes for mixed optimal stop** and control of processes with infinite activity jumps and where the objective is evaluated by a nonlinear expectation. Exploiting an approximation by switching systems, piecewise constant policy timestep** reduces the problem to nonlocal semi-linear equations with different control parameters, uncoupled over individual time steps… ▽ More We propose a class of numerical schemes for mixed optimal stop** and control of processes with infinite activity jumps and where the objective is evaluated by a nonlinear expectation. Exploiting an approximation by switching systems, piecewise constant policy timestep** reduces the problem to nonlocal semi-linear equations with different control parameters, uncoupled over individual time steps, which we solve by fully implicit monotone approximations to the controlled diffusion and the nonlocal term, and specifically the Lax-Friedrichs scheme for the nonlinearity in the gradient. We establish a comparison principle for the switching system and demonstrate the convergence of the schemes, which subsequently gives a constructive proof for the existence of a solution to the switching system. Numerical experiments are presented for a recursive utility maximization problem to demonstrate the effectiveness of the new schemes. △ Less

Submitted 10 March, 2018; originally announced March 2018.

Stability and error analysis of an implicit Milstein finite difference scheme for a two-dimensional Zakai SPDE

Authors: Christoph Reisinger, Zhenru Wang

Abstract: In this article, we propose an implicit finite difference scheme for a two-dimensional parabolic stochastic partial differential equation (SPDE) of Zakai type. The scheme is based on a Milstein approximation to the stochastic integral and an alternating direction implicit (ADI) discretisation of the elliptic term. We prove its mean-square stability and convergence in L2 of first order in time and… ▽ More In this article, we propose an implicit finite difference scheme for a two-dimensional parabolic stochastic partial differential equation (SPDE) of Zakai type. The scheme is based on a Milstein approximation to the stochastic integral and an alternating direction implicit (ADI) discretisation of the elliptic term. We prove its mean-square stability and convergence in L2 of first order in time and second order in space, by Fourier analysis, in the presence of Dirac initial data. Numerical tests confirm these findings empirically. △ Less

Submitted 28 November, 2018; v1 submitted 21 February, 2018; originally announced February 2018.

Comments: 31 pages

MSC Class: 65T50; 65N06; 65N12

Stability and convergence of second order backward differentiation schemes for parabolic Hamilton-Jacobi-Bellman equations

Authors: Olivier Bokanowski, Athena Picarelli, Christoph Reisinger

Abstract: We study a second order BDF (Backward Differentiation Formula) scheme for the numerical approximation of parabolic HJB (Hamilton-Jacobi-Bellman) equations. The scheme under consideration is implicit, non-monotone, and second order accurate in time and space. The lack of monotonicity prevents the use of well-known convergence results for solutions in the viscosity sense. In this work, we establish… ▽ More We study a second order BDF (Backward Differentiation Formula) scheme for the numerical approximation of parabolic HJB (Hamilton-Jacobi-Bellman) equations. The scheme under consideration is implicit, non-monotone, and second order accurate in time and space. The lack of monotonicity prevents the use of well-known convergence results for solutions in the viscosity sense. In this work, we establish rigorous stability results in a general nonlinear setting as well as convergence results for some particular cases with additional regularity assumptions. While most results are presented for one-dimensional, linear parabolic and non-linear HJB equations, some results are also extended to multiple dimensions and to Isaacs equations. Numerical tests are included to validate the method. △ Less

Submitted 20 February, 2018; originally announced February 2018.

Some regularity and convergence results for parabolic Hamilton-Jacobi-Bellman equations in bounded domains

Authors: Athena Picarelli, Christoph Reisinger, Julen Rotaetxe Arto

Abstract: We study the approximation of parabolic Hamilton-Jacobi-Bellman (HJB) equations in bounded domains with strong Dirichlet boundary conditions. We work under the assumption of the existence of a sufficiently regular barrier function for the problem to obtain well-posedness and regularity of a related switching system and the convergence of its components to the HJB equation. In particular, we show e… ▽ More We study the approximation of parabolic Hamilton-Jacobi-Bellman (HJB) equations in bounded domains with strong Dirichlet boundary conditions. We work under the assumption of the existence of a sufficiently regular barrier function for the problem to obtain well-posedness and regularity of a related switching system and the convergence of its components to the HJB equation. In particular, we show existence of a viscosity solution to the switching system by a novel construction of sub- and supersolutions and application of Perron's method. Error bounds for monotone schemes for the HJB equation are then derived from estimates near the boundary, where the standard regularisation procedure for viscosity solutions is not applicable, and are found to be of the same order as known results for the whole space. We deduce error bounds for some common finite difference and truncated semi-Lagrangian schemes. △ Less

Submitted 15 July, 2019; v1 submitted 30 October, 2017; originally announced October 2017.

Analysis of Multi-Index Monte Carlo Estimators for a Zakai SPDE

Authors: Zhenru Wang, Christoph Reisinger

Abstract: In this article, we propose a space-time Multi-Index Monte Carlo (MIMC) estimator for a one-dimensional parabolic stochastic partial differential equation (SPDE) of Zakai type. We compare the complexity with the Multilevel Monte Carlo (MLMC) method of Giles and Reisinger (2012), and find, by means of Fourier analysis, that the MIMC method: (i) has suboptimal complexity of… ▽ More In this article, we propose a space-time Multi-Index Monte Carlo (MIMC) estimator for a one-dimensional parabolic stochastic partial differential equation (SPDE) of Zakai type. We compare the complexity with the Multilevel Monte Carlo (MLMC) method of Giles and Reisinger (2012), and find, by means of Fourier analysis, that the MIMC method: (i) has suboptimal complexity of $O(\varepsilon^{-2}|\log\varepsilon|^3)$ for a root mean square error (RMSE) $\varepsilon$ if the same spatial discretisation as in the MLMC method is used; (ii) has a better complexity of $O(\varepsilon^{-2}|\log\varepsilon|)$ if a carefully adapted discretisation is used; (iii) has to be adapted for non-smooth functionals. Numerical tests confirm these findings empirically. △ Less

Submitted 8 December, 2016; originally announced December 2016.

High-order filtered schemes for time-dependent second order HJB equations

Authors: Olivier Bokanowski, Athena Picarelli, Christoph Reisinger

Abstract: In this paper, we present and analyse a class of "filtered" numerical schemes for second order Hamilton-Jacobi-Bellman equations. Our approach follows the ideas introduced in B.D. Froese and A.M. Oberman, Convergent filtered schemes for the Monge-Ampère partial differential equation, SIAM J. Numer. Anal., 51(1):423--444, 2013, and more recently applied by other authors to stationary or time-depend… ▽ More In this paper, we present and analyse a class of "filtered" numerical schemes for second order Hamilton-Jacobi-Bellman equations. Our approach follows the ideas introduced in B.D. Froese and A.M. Oberman, Convergent filtered schemes for the Monge-Ampère partial differential equation, SIAM J. Numer. Anal., 51(1):423--444, 2013, and more recently applied by other authors to stationary or time-dependent first order Hamilton-Jacobi equations. For high order approximation schemes (where "high" stands for greater than one), the inevitable loss of monotonicity prevents the use of the classical theoretical results for convergence to viscosity solutions. The work introduces a suitable local modification of these schemes by "filtering" them with a monotone scheme, such that they can be proven convergent and still show an overall high order behaviour for smooth enough solutions. We give theoretical proofs of these claims and illustrate the behaviour with numerical tests from mathematical finance, focussing also on the use of backward difference formulae (BDF) for constructing the high order schemes. △ Less

Submitted 15 November, 2016; originally announced November 2016.

Comments: 27 pages, 16 figures, 4 tables

MSC Class: 65M06; 91G80

The non-locality of Markov chain approximations to two-dimensional diffusions

Authors: Christoph Reisinger

Abstract: In this short paper, we consider discrete-time Markov chains on lattices as approximations to continuous-time diffusion processes. The approximations can be interpreted as finite difference schemes for the generator of the process. We derive conditions on the diffusion coefficients which permit transition probabilities to match locally first and second moments. We derive a novel formula which expr… ▽ More In this short paper, we consider discrete-time Markov chains on lattices as approximations to continuous-time diffusion processes. The approximations can be interpreted as finite difference schemes for the generator of the process. We derive conditions on the diffusion coefficients which permit transition probabilities to match locally first and second moments. We derive a novel formula which expresses how the matching becomes more difficult for larger (absolute) correlations and strongly anisotropic processes, such that instantaneous moves to more distant neighbours on the lattice have to be allowed. Roughly speaking, for non-zero correlations, the distance covered in one timestep is proportional to the ratio of volatilities in the two directions. We discuss the implications to Markov decision processes and the convergence analysis of approximations to Hamilton-Jacobi-Bellman equations in the Barles-Souganidis framework. △ Less

Submitted 7 November, 2016; v1 submitted 20 May, 2016; originally announced May 2016.

Comments: Corrected two errata from previous and journal version: definition of R in (5) and summations in (7)

Boundary Treatment and Multigrid Preconditioning for Semi-Lagrangian Schemes Applied to Hamilton-Jacobi-Bellman Equations

Authors: Christoph Reisinger, Julen Rotaetxe Arto

Abstract: We analyse two practical aspects that arise in the numerical solution of Hamilton-Jacobi-Bellman (HJB) equations by a particular class of monotone approximation schemes known as semi-Lagrangian schemes. These schemes make use of a wide stencil to achieve convergence and result in discretization matrices that are less sparse and less local than those coming from standard finite difference schemes.… ▽ More We analyse two practical aspects that arise in the numerical solution of Hamilton-Jacobi-Bellman (HJB) equations by a particular class of monotone approximation schemes known as semi-Lagrangian schemes. These schemes make use of a wide stencil to achieve convergence and result in discretization matrices that are less sparse and less local than those coming from standard finite difference schemes. This leads to computational difficulties not encountered there. In particular, we consider the overstep** of the domain boundary and analyse the accuracy and stability of stencil truncation. This truncation imposes a stricter CFL condition for explicit schemes in the vicinity of boundaries than in the interior, such that implicit schemes become attractive. We then study the use of geometric, algebraic and aggregation-based multigrid preconditioners to solve the resulting discretised systems from implicit time step** schemes efficiently. Finally, we illustrate the performance of these techniques numerically for benchmark test cases from the literature. △ Less

Submitted 7 November, 2016; v1 submitted 16 May, 2016; originally announced May 2016.

A partial Fourier transform method for a class of hypoelliptic Kolmogorov equations

Authors: Christoph Reisinger, Endre Süli, Alan Whitley

Abstract: We consider hypoelliptic Kolmogorov equations in $n+1$ spatial dimensions, with $n\geq 1$, where the differential operator in the first $n$ spatial variables featuring in the equation is second-order elliptic, and with respect to the $(n+1)$st spatial variable the equation contains a pure transport term only and is therefore first-order hyperbolic. If the two differential operators, in the first… ▽ More We consider hypoelliptic Kolmogorov equations in $n+1$ spatial dimensions, with $n\geq 1$, where the differential operator in the first $n$ spatial variables featuring in the equation is second-order elliptic, and with respect to the $(n+1)$st spatial variable the equation contains a pure transport term only and is therefore first-order hyperbolic. If the two differential operators, in the first $n$ and in the $(n+1)$st co-ordinate directions, do not commute, we benefit from hypoelliptic regularization in time, and the solution for $t>0$ is smooth even for a Dirac initial datum prescribed at $t=0$. We study specifically the case where the coefficients depend only on the first $n$ variables. In that case, a Fourier transform in the last variable and standard central finite difference approximation in the other variables can be applied for the numerical solution. We prove second-order convergence in the spatial mesh size for the model hypoelliptic equation $\frac{\partial u}{\partial t} + x \frac{\partial u}{\partial y} = \frac{\partial^2 u}{\partial x^2}$ subject to the initial condition $u(x,y,0) = δ(x) δ(y)$, with $(x,y) \in \mathbb{R} \times\mathbb{R}$ and $t>0$, proposed by Kolmogorov, and for an extension with $n=2$. We also demonstrate exponential convergence of an approximation of the inverse Fourier transform based on the trapezium rule. Lastly, we apply the method to a PDE arising in mathematical finance, which models the distribution of the hedging error under a mis-specified derivative pricing model. △ Less

Submitted 24 May, 2016; v1 submitted 18 April, 2016; originally announced April 2016.

Error analysis of truncated expansion solutions to high-dimensional parabolic PDEs

Authors: Christoph Reisinger, Rasmus Wissmann

Abstract: We study an expansion method for high-dimensional parabolic PDEs which constructs accurate approximate solutions by decomposition into solutions to lower-dimensional PDEs, and which is particularly effective if there are a low number of dominant principal components. The focus of the present article is the derivation of sharp error bounds for the constant coefficient case and a first and second or… ▽ More We study an expansion method for high-dimensional parabolic PDEs which constructs accurate approximate solutions by decomposition into solutions to lower-dimensional PDEs, and which is particularly effective if there are a low number of dominant principal components. The focus of the present article is the derivation of sharp error bounds for the constant coefficient case and a first and second order approximation. We give a precise characterisation when these bounds hold for (non-smooth) option pricing applications and provide numerical results demonstrating that the practically observed convergence speed is in agreement with the theoretical predictions. △ Less

Submitted 7 November, 2016; v1 submitted 18 May, 2015; originally announced May 2015.

Piecewise Constant Policy Approximations to Hamilton-Jacobi-Bellman Equations

Authors: Christoph Reisinger, Peter Forsyth

Abstract: An advantageous feature of piecewise constant policy timestep** for Hamilton-Jacobi-Bellman (HJB) equations is that different linear approximation schemes, and indeed different meshes, can be used for the resulting linear equations for different control parameters. Standard convergence analysis suggests that monotone (i.e., linear) interpolation must be used to transfer data between meshes. Usin… ▽ More An advantageous feature of piecewise constant policy timestep** for Hamilton-Jacobi-Bellman (HJB) equations is that different linear approximation schemes, and indeed different meshes, can be used for the resulting linear equations for different control parameters. Standard convergence analysis suggests that monotone (i.e., linear) interpolation must be used to transfer data between meshes. Using the equivalence to a switching system and an adaptation of the usual arguments based on consistency, stability and monotonicity, we show that if limited, potentially higher order interpolation is used for the mesh transfer, convergence is guaranteed. We provide numerical tests for the mean-variance optimal investment problem and the uncertain volatility option pricing model, and compare the results to published test cases. △ Less

Submitted 20 January, 2016; v1 submitted 19 March, 2015; originally announced March 2015.

A Forward Equation for Barrier Options under the Brunick&Shreve Markovian Projection

Authors: Ben Hambly, Matthieu Mariapragassam, Christoph Reisinger

Abstract: We derive a forward equation for arbitrage-free barrier option prices, in terms of Markovian projections of the stochastic volatility process, in continuous semi-martingale models. This provides a Dupire-type formula for the coefficient derived by Brunick and Shreve for their mimicking diffusion and can be interpreted as the canonical extension of local volatility for barrier options. Alternativel… ▽ More We derive a forward equation for arbitrage-free barrier option prices, in terms of Markovian projections of the stochastic volatility process, in continuous semi-martingale models. This provides a Dupire-type formula for the coefficient derived by Brunick and Shreve for their mimicking diffusion and can be interpreted as the canonical extension of local volatility for barrier options. Alternatively, a forward partial-integro differential equation (PIDE) is introduced which provides up-and-out call prices, under a Brunick-Shreve model, for the complete set of strikes, barriers and maturities in one solution step. Similar to the vanilla forward PDE, the above-named forward PIDE can serve as a building block for an efficient calibration routine including barrier option quotes. We provide a discretisation scheme for the PIDE as well as a numerical validation. △ Less

Submitted 16 September, 2016; v1 submitted 13 November, 2014; originally announced November 2014.

Comments: 20 pages, Quantitative Finance Volume 16, 2016 - Issue 6