-
Entropy annealing for policy mirror descent in continuous time and space
Authors:
Deven Sethi,
David Šiška,
Yufei Zhang
Abstract:
Entropy regularization has been extensively used in policy optimization algorithms to regularize the optimization landscape and accelerate convergence; however, it comes at the cost of introducing an additional regularization bias. This work quantifies the impact of entropy regularization on the convergence of policy gradient methods for stochastic exit time control problems. We analyze a continuo…
▽ More
Entropy regularization has been extensively used in policy optimization algorithms to regularize the optimization landscape and accelerate convergence; however, it comes at the cost of introducing an additional regularization bias. This work quantifies the impact of entropy regularization on the convergence of policy gradient methods for stochastic exit time control problems. We analyze a continuous-time policy mirror descent dynamics, which updates the policy based on the gradient of an entropy-regularized value function and adjusts the strength of entropy regularization as the algorithm progresses. We prove that with a fixed entropy level, the dynamics converges exponentially to the optimal solution of the regularized problem. We further show that when the entropy level decays at suitable polynomial rates, the annealed flow converges to the solution of the unregularized problem at a rate of $\mathcal O(1/S)$ for discrete action spaces and, under suitable conditions, at a rate of $\mathcal O(1/\sqrt{S})$ for general action spaces, with $S$ being the gradient flow time. This paper explains how entropy regularization improves policy optimization, even with the true gradient, from the perspective of convergence rate.
△ Less
Submitted 6 June, 2024; v1 submitted 30 May, 2024;
originally announced May 2024.
-
Mirror Descent for Stochastic Control Problems with Measure-valued Controls
Authors:
Bekzhan Kerimkulov,
David Šiška,
Łukasz Szpruch,
Yufei Zhang
Abstract:
This paper studies the convergence of the mirror descent algorithm for finite horizon stochastic control problems with measure-valued control processes. The control objective involves a convex regularisation function, denoted as $h$, with regularisation strength determined by the weight $τ\ge 0$. The setting covers regularised relaxed control problems. Under suitable conditions, we establish the r…
▽ More
This paper studies the convergence of the mirror descent algorithm for finite horizon stochastic control problems with measure-valued control processes. The control objective involves a convex regularisation function, denoted as $h$, with regularisation strength determined by the weight $τ\ge 0$. The setting covers regularised relaxed control problems. Under suitable conditions, we establish the relative smoothness and convexity of the control objective with respect to the Bregman divergence of $h$, and prove linear convergence of the algorithm for $τ=0$ and exponential convergence for $τ>0$. The results apply to common regularisers including relative entropy, $χ^2$-divergence, and entropic Wasserstein costs. This validates recent reinforcement learning heuristics that adding regularisation accelerates the convergence of gradient methods. The proof exploits careful regularity estimates of backward stochastic differential equations in the bounded mean oscillation norm.
△ Less
Submitted 2 January, 2024;
originally announced January 2024.
-
A Fisher-Rao gradient flow for entropy-regularised Markov decision processes in Polish spaces
Authors:
Bekzhan Kerimkulov,
James-Michael Leahy,
David Siska,
Lukasz Szpruch,
Yufei Zhang
Abstract:
We study the global convergence of a Fisher-Rao policy gradient flow for infinite-horizon entropy-regularised Markov decision processes with Polish state and action space. The flow is a continuous-time analogue of a policy mirror descent method. We establish the global well-posedness of the gradient flow and demonstrate its exponential convergence to the optimal policy. Moreover, we prove the flow…
▽ More
We study the global convergence of a Fisher-Rao policy gradient flow for infinite-horizon entropy-regularised Markov decision processes with Polish state and action space. The flow is a continuous-time analogue of a policy mirror descent method. We establish the global well-posedness of the gradient flow and demonstrate its exponential convergence to the optimal policy. Moreover, we prove the flow is stable with respect to gradient evaluation, offering insights into the performance of a natural policy gradient flow with log-linear policy parameterisation. To overcome challenges stemming from the lack of the convexity of the objective function and the discontinuity arising from the entropy regulariser, we leverage the performance difference lemma and the duality relationship between the gradient and mirror descent flows.
△ Less
Submitted 4 October, 2023;
originally announced October 2023.
-
Inefficiency of CFMs: hedging perspective and agent-based simulations
Authors:
Samuel Cohen,
Marc Sabaté Vidales,
David Šiška,
Łukasz Szpruch
Abstract:
We investigate whether the fee income from trades on the CFM is sufficient for the liquidity providers to hedge away the exposure to market risk. We first analyse this problem through the lens of continuous-time financial mathematics and derive an upper bound for not-arbitrage fee income that would make CFM efficient and liquidity provision fair. We then evaluate our findings by performing multi-a…
▽ More
We investigate whether the fee income from trades on the CFM is sufficient for the liquidity providers to hedge away the exposure to market risk. We first analyse this problem through the lens of continuous-time financial mathematics and derive an upper bound for not-arbitrage fee income that would make CFM efficient and liquidity provision fair. We then evaluate our findings by performing multi-agent simulations by varying CFM fees, market volatility, and rate of arrival of liquidity takers. We observe that, on average, fee income generated from liquidity provision is insufficient to compensate for market risk.
△ Less
Submitted 8 February, 2023;
originally announced February 2023.
-
The Modified MSA, a Gradient Flow and Convergence
Authors:
Deven Sethi,
David Šiška
Abstract:
The modified Method of Successive Approximations (MSA) is an iterative scheme for approximating solutions to stochastic control problems in continuous time based on Pontryagin Optimality Principle which, starting with an initial open loop control, solves the forward equation, the backward adjoint equation and then performs a static minimization step. We observe that this is an implicit Euler schem…
▽ More
The modified Method of Successive Approximations (MSA) is an iterative scheme for approximating solutions to stochastic control problems in continuous time based on Pontryagin Optimality Principle which, starting with an initial open loop control, solves the forward equation, the backward adjoint equation and then performs a static minimization step. We observe that this is an implicit Euler scheme for a gradient flow system. We prove that appropriate interpolations of the iterates of the modified MSA converge to a gradient flow with rate $τ$. We then study the convergence of this gradient flow as time goes to infinity. In the general (non-convex) case we prove that the gradient term itself converges to zero. This is a consequence of an energy identity which shows that the optimization objective decreases along the gradient flow. Moreover, in the convex case, when Pontryagin Optimality Principle provides a sufficient condition for optimality, we prove that the optimization objective converges at rate $\tfrac{1}{S}$ to its optimal value and at exponential rate under strong convexity. The main technical difficulties lie in obtaining appropriate properties of the Hamiltonian (growth, continuity). These are obtained by utilising the theory of Bounded Mean Oscillation (BMO) martingales required for estimates on the adjoint Backward Stochastic Differential Equation (BSDE).
△ Less
Submitted 7 October, 2023; v1 submitted 12 December, 2022;
originally announced December 2022.
-
Decaying derivative estimates for functions of solutions to non-autonomous SDEs
Authors:
Maria Lefter,
David Šiška,
Łukasz Szpruch
Abstract:
We produce uniform and decaying bounds in time for derivatives of the solution to the backwards Kolmogorov equation associated to a stochastic processes governed by a time dependent dynamics. These hold under assumptions over the integrability properties in finite time of the derivatives of the transition density associated to the process, together with the assumption of remaining close over all…
▽ More
We produce uniform and decaying bounds in time for derivatives of the solution to the backwards Kolmogorov equation associated to a stochastic processes governed by a time dependent dynamics. These hold under assumptions over the integrability properties in finite time of the derivatives of the transition density associated to the process, together with the assumption of remaining close over all $[0,\infty)$, or decaying in time, to some static measure. We moreover provide examples which satisfy such a set of assumptions. Finally, the results are interpreted in the McKean-Vlasov context for monotonic coefficients by introducing an auxiliary non-autonomous stochastic process.
△ Less
Submitted 26 July, 2022;
originally announced July 2022.
-
Convergence of Policy Gradient for Entropy Regularized MDPs with Neural Network Approximation in the Mean-Field Regime
Authors:
Bekzhan Kerimkulov,
James-Michael Leahy,
David Šiška,
Lukasz Szpruch
Abstract:
We study the global convergence of policy gradient for infinite-horizon, continuous state and action space, and entropy-regularized Markov decision processes (MDPs). We consider a softmax policy with (one-hidden layer) neural network approximation in a mean-field regime. Additional entropic regularization in the associated mean-field probability measure is added, and the corresponding gradient flo…
▽ More
We study the global convergence of policy gradient for infinite-horizon, continuous state and action space, and entropy-regularized Markov decision processes (MDPs). We consider a softmax policy with (one-hidden layer) neural network approximation in a mean-field regime. Additional entropic regularization in the associated mean-field probability measure is added, and the corresponding gradient flow is studied in the 2-Wasserstein metric. We show that the objective function is increasing along the gradient flow. Further, we prove that if the regularization in terms of the mean-field measure is sufficient, the gradient flow converges exponentially fast to the unique stationary solution, which is the unique maximizer of the regularized MDP objective. Lastly, we study the sensitivity of the value function along the gradient flow with respect to regularization parameters and the initial condition. Our results rely on the careful analysis of the non-linear Fokker-Planck-Kolmogorov equation and extend the pioneering work of Mei et al. 2020 and Agarwal et al. 2020, which quantify the global convergence rate of policy gradient for entropy-regularized MDPs in the tabular setting.
△ Less
Submitted 16 June, 2022; v1 submitted 18 January, 2022;
originally announced January 2022.
-
Solving path dependent PDEs with LSTM networks and path signatures
Authors:
Marc Sabate-Vidales,
David Šiška,
Lukasz Szpruch
Abstract:
Using a combination of recurrent neural networks and signature methods from the rough paths theory we design efficient algorithms for solving parametric families of path dependent partial differential equations (PPDEs) that arise in pricing and hedging of path-dependent derivatives or from use of non-Markovian model, such as rough volatility models in Jacquier and Oumgari, 2019. The solutions of P…
▽ More
Using a combination of recurrent neural networks and signature methods from the rough paths theory we design efficient algorithms for solving parametric families of path dependent partial differential equations (PPDEs) that arise in pricing and hedging of path-dependent derivatives or from use of non-Markovian model, such as rough volatility models in Jacquier and Oumgari, 2019. The solutions of PPDEs are functions of time, a continuous path (the asset price history) and model parameters. As the domain of the solution is infinite dimensional many recently developed deep learning techniques for solving PDEs do not apply. Similarly as in Vidales et al. 2018, we identify the objective function used to learn the PPDE by using martingale representation theorem. As a result we can de-bias and provide confidence intervals for then neural network-based algorithm. We validate our algorithm using classical models for pricing lookback and auto-callable options and report errors for approximating both prices and hedging strategies.
△ Less
Submitted 20 November, 2020;
originally announced November 2020.
-
A modified MSA for stochastic control problems
Authors:
Bekzhan Kerimkulov,
David Šiška,
Łukasz Szpruch
Abstract:
The classical Method of Successive Approximations (MSA) is an iterative method for solving stochastic control problems and is derived from Pontryagin's optimality principle. It is known that the MSA may fail to converge. Using careful estimates for the backward stochastic differential equation (BSDE) this paper suggests a modification to the MSA algorithm. This modified MSA is shown to converge fo…
▽ More
The classical Method of Successive Approximations (MSA) is an iterative method for solving stochastic control problems and is derived from Pontryagin's optimality principle. It is known that the MSA may fail to converge. Using careful estimates for the backward stochastic differential equation (BSDE) this paper suggests a modification to the MSA algorithm. This modified MSA is shown to converge for general stochastic control problems with control in both the drift and diffusion coefficients. Under some additional assumptions the rate of convergence is shown. The results are valid without restrictions on the time horizon of the control problem, in contrast to iterative methods based on the theory of forward-backward stochastic differential equations.
△ Less
Submitted 17 November, 2020; v1 submitted 10 July, 2020;
originally announced July 2020.
-
Robust pricing and hedging via neural SDEs
Authors:
Patryk Gierjatowicz,
Marc Sabate-Vidales,
David Šiška,
Lukasz Szpruch,
Žan Žurič
Abstract:
Mathematical modelling is ubiquitous in the financial industry and drives key decision processes. Any given model provides only a crude approximation to reality and the risk of using an inadequate model is hard to detect and quantify. By contrast, modern data science techniques are opening the door to more robust and data-driven model selection mechanisms. However, most machine learning models are…
▽ More
Mathematical modelling is ubiquitous in the financial industry and drives key decision processes. Any given model provides only a crude approximation to reality and the risk of using an inadequate model is hard to detect and quantify. By contrast, modern data science techniques are opening the door to more robust and data-driven model selection mechanisms. However, most machine learning models are "black-boxes" as individual parameters do not have meaningful interpretation. The aim of this paper is to combine the above approaches achieving the best of both worlds. Combining neural networks with risk models based on classical stochastic differential equations (SDEs), we find robust bounds for prices of derivatives and the corresponding hedging strategies while incorporating relevant market data. The resulting model called neural SDE is an instantiation of generative models and is closely linked with the theory of causal optimal transport. Neural SDEs allow consistent calibration under both the risk-neutral and the real-world measures. Thus the model can be used to simulate market scenarios needed for assessing risk profiles and hedging strategies. We develop and analyse novel algorithms needed for efficient use of neural SDEs. We validate our approach with numerical experiments using both local and stochastic volatility models.
△ Less
Submitted 8 July, 2020;
originally announced July 2020.
-
Gradient Flows for Regularized Stochastic Control Problems
Authors:
David Šiška,
Łukasz Szpruch
Abstract:
This paper studies stochastic control problems with the action space taken to be probability measures, with the objective penalised by the relative entropy. We identify suitable metric space on which we construct a gradient flow for the measure-valued control process, in the set of admissible controls, along which the cost functional is guaranteed to decrease. It is shown that any invariant measur…
▽ More
This paper studies stochastic control problems with the action space taken to be probability measures, with the objective penalised by the relative entropy. We identify suitable metric space on which we construct a gradient flow for the measure-valued control process, in the set of admissible controls, along which the cost functional is guaranteed to decrease. It is shown that any invariant measure of this gradient flow satisfies the Pontryagin optimality principle. If the problem we work with is sufficiently convex, the gradient flow converges exponentially fast. Furthermore, the optimal measure-valued control process admits a Bayesian interpretation which means that one can incorporate prior knowledge when solving such stochastic control problems. This work is motivated by a desire to extend the theoretical underpinning for the convergence of stochastic gradient type algorithms widely employed in the reinforcement learning community to solve control problems.
△ Less
Submitted 25 January, 2024; v1 submitted 10 June, 2020;
originally announced June 2020.
-
Mean-Field Neural ODEs via Relaxed Optimal Control
Authors:
Jean-François Jabir,
David Šiška,
Łukasz Szpruch
Abstract:
We develop a framework for the analysis of deep neural networks and neural ODE models that are trained with stochastic gradient algorithms. We do that by identifying the connections between control theory, deep learning and theory of statistical sampling. We derive Pontryagin's optimality principle and study the corresponding gradient flow in the form of Mean-Field Langevin dynamics (MFLD) for sol…
▽ More
We develop a framework for the analysis of deep neural networks and neural ODE models that are trained with stochastic gradient algorithms. We do that by identifying the connections between control theory, deep learning and theory of statistical sampling. We derive Pontryagin's optimality principle and study the corresponding gradient flow in the form of Mean-Field Langevin dynamics (MFLD) for solving relaxed data-driven control problems. Subsequently, we study uniform-in-time propagation of chaos of time-discretised MFLD. We derive explicit convergence rate in terms of the learning rate, the number of particles/model parameters and the number of iterations of the gradient algorithm. In addition, we study the error arising when using a finite training data set and thus provide quantitive bounds on the generalisation error. Crucially, the obtained rates are dimension-independent. This is possible by exploiting the regularity of the model with respect to the measure over the parameter space.
△ Less
Submitted 16 March, 2021; v1 submitted 11 December, 2019;
originally announced December 2019.
-
Uniform error estimates for artificial neural network approximations for heat equations
Authors:
Lukas Gonon,
Philipp Grohs,
Arnulf Jentzen,
David Kofler,
David Šiška
Abstract:
Recently, artificial neural networks (ANNs) in conjunction with stochastic gradient descent optimization methods have been employed to approximately compute solutions of possibly rather high-dimensional partial differential equations (PDEs). Very recently, there have also been a number of rigorous mathematical results in the scientific literature which examine the approximation capabilities of suc…
▽ More
Recently, artificial neural networks (ANNs) in conjunction with stochastic gradient descent optimization methods have been employed to approximately compute solutions of possibly rather high-dimensional partial differential equations (PDEs). Very recently, there have also been a number of rigorous mathematical results in the scientific literature which examine the approximation capabilities of such deep learning based approximation algorithms for PDEs. These mathematical results from the scientific literature prove in part that algorithms based on ANNs are capable of overcoming the curse of dimensionality in the numerical approximation of high-dimensional PDEs. In these mathematical results from the scientific literature usually the error between the solution of the PDE and the approximating ANN is measured in the $L^p$-sense with respect to some $p \in [1,\infty)$ and some probability measure. In many applications it is, however, also important to control the error in a uniform $L^\infty$-sense. The key contribution of the main result of this article is to develop the techniques to obtain error estimates between solutions of PDEs and approximating ANNs in the uniform $L^\infty$-sense. In particular, we prove that the number of parameters of an ANN to uniformly approximate the classical solution of the heat equation in a region $ [a,b]^d $ for a fixed time point $ T \in (0,\infty) $ grows at most polynomially in the dimension $ d \in \mathbb{N} $ and the reciprocal of the approximation precision $ \varepsilon > 0 $. This shows that ANNs can overcome the curse of dimensionality in the numerical approximation of the heat equation when the error is measured in the uniform $L^\infty$-norm.
△ Less
Submitted 15 June, 2020; v1 submitted 20 November, 2019;
originally announced November 2019.
-
Weak Existence and Uniqueness for McKean-Vlasov SDEs with Common Noise
Authors:
William R. P. Hammersley,
David Šiška,
Łukasz Szpruch
Abstract:
This paper concerns the McKean-Vlasov stochastic differential equation (SDE) with common noise. An appropriate definition of a weak solution to such an equation is developed. The importance of the notion of compatibility in this definition is highlighted by a demonstration of its rôle in connecting weak solutions to McKean-Vlasov SDEs with common noise and solutions to corresponding stochastic par…
▽ More
This paper concerns the McKean-Vlasov stochastic differential equation (SDE) with common noise. An appropriate definition of a weak solution to such an equation is developed. The importance of the notion of compatibility in this definition is highlighted by a demonstration of its rôle in connecting weak solutions to McKean-Vlasov SDEs with common noise and solutions to corresponding stochastic partial differential equations (SPDEs). By kee** track of the dependence structure between all components in a sequence of approximating processes, a compactness argument is employed to prove the existence of a weak solution assuming boundedness and joint continuity of the coefficients (allowing for degenerate diffusions). Weak uniqueness is established when the private (idiosyncratic) noise's diffusion coefficient is non-degenerate and the drift is regular in the total variation distance. This seems sharp when one considers using finite-dimensional noise to regularise an infinite dimensional problem. The proof relies on a suitably tailored cost function in the Monge-Kantorovich problem and representation of weak solutions via Girsanov transformations.
△ Less
Submitted 26 June, 2020; v1 submitted 2 August, 2019;
originally announced August 2019.
-
Mean-Field Langevin Dynamics and Energy Landscape of Neural Networks
Authors:
Kaitong Hu,
Zhenjie Ren,
David Siska,
Lukasz Szpruch
Abstract:
Our work is motivated by a desire to study the theoretical underpinning for the convergence of stochastic gradient type algorithms widely used for non-convex learning tasks such as training of neural networks. The key insight, already observed in the works of Mei, Montanari and Nguyen (2018), Chizat and Bach (2018) as well as Rotskoff and Vanden-Eijnden (2018), is that a certain class of the finit…
▽ More
Our work is motivated by a desire to study the theoretical underpinning for the convergence of stochastic gradient type algorithms widely used for non-convex learning tasks such as training of neural networks. The key insight, already observed in the works of Mei, Montanari and Nguyen (2018), Chizat and Bach (2018) as well as Rotskoff and Vanden-Eijnden (2018), is that a certain class of the finite-dimensional non-convex problems becomes convex when lifted to infinite-dimensional space of measures. We leverage this observation and show that the corresponding energy functional defined on the space of probability measures has a unique minimiser which can be characterised by a first-order condition using the notion of linear functional derivative. Next, we study the corresponding gradient flow structure in 2-Wasserstein metric, which we call Mean-Field Langevin Dynamics (MFLD), and show that the flow of marginal laws induced by the gradient flow converges to a stationary distribution, which is exactly the minimiser of the energy functional. We observe that this convergence is exponential under conditions that are satisfied for highly regularised learning tasks. Our proof of convergence to stationary probability measure is novel and it relies on a generalisation of LaSalle's invariance principle combined with HWI inequality. Importantly, we assume neither that interaction potential of MFLD is of convolution type nor that it has any particular symmetric structure. Furthermore, we allow for the general convex objective function, unlike, most papers in the literature that focus on quadratic loss. Finally, we show that the error between finite-dimensional optimisation problem and its infinite-dimensional limit is of order one over the number of parameters.
△ Less
Submitted 13 December, 2020; v1 submitted 19 May, 2019;
originally announced May 2019.
-
Exponential Convergence and stability of Howards's Policy Improvement Algorithm for Controlled Diffusions
Authors:
B. Kerimkulov,
D. Šiška,
Ł. Szpruch
Abstract:
Optimal control problems are inherently hard to solve as the optimization must be performed simultaneously with updating the underlying system. Starting from an initial guess, Howard's policy improvement algorithm separates the step of updating the trajectory of the dynamical system from the optimization and iterations of this should converge to the optimal control. In the discrete space-time sett…
▽ More
Optimal control problems are inherently hard to solve as the optimization must be performed simultaneously with updating the underlying system. Starting from an initial guess, Howard's policy improvement algorithm separates the step of updating the trajectory of the dynamical system from the optimization and iterations of this should converge to the optimal control. In the discrete space-time setting this is often the case and even rates of convergence are known. In the continuous space-time setting of controlled diffusion the algorithm consists of solving a linear PDE followed by maximization problem. This has been shown to converge, in some situations, however no global rate of is known. The first main contribution of this paper is to establish global rate of convergence for the policy improvement algorithm and a variant, called here the gradient iteration algorithm. The second main contribution is the proof of stability of the algorithms under perturbations to both the accuracy of the linear PDE solution and the accuracy of the maximization step. The proof technique is new in this context as it uses the theory of backward stochastic differential equations.
△ Less
Submitted 22 May, 2020; v1 submitted 19 December, 2018;
originally announced December 2018.
-
Unbiased deep solvers for linear parametric PDEs
Authors:
Marc Sabate Vidales,
David Siska,
Lukasz Szpruch
Abstract:
We develop several deep learning algorithms for approximating families of parametric PDE solutions. The proposed algorithms approximate solutions together with their gradients, which in the context of mathematical finance means that the derivative prices and hedging strategies are computed simulatenously. Having approximated the gradient of the solution one can combine it with a Monte-Carlo simula…
▽ More
We develop several deep learning algorithms for approximating families of parametric PDE solutions. The proposed algorithms approximate solutions together with their gradients, which in the context of mathematical finance means that the derivative prices and hedging strategies are computed simulatenously. Having approximated the gradient of the solution one can combine it with a Monte-Carlo simulation to remove the bias in the deep network approximation of the PDE solution (derivative price). This is achieved by leveraging the Martingale Representation Theorem and combining the Monte Carlo simulation with the neural network. The resulting algorithm is robust with respect to quality of the neural network approximation and consequently can be used as a black-box in case only limited a priori information about the underlying problem is available. We believe this is important as neural network based algorithms often require fair amount of tuning to produce satisfactory results. The methods are empirically shown to work for high-dimensional problems (e.g. 100 dimensions). We provide diagnostics that shed light on appropriate network architectures.
△ Less
Submitted 17 January, 2022; v1 submitted 11 October, 2018;
originally announced October 2018.
-
McKean-Vlasov SDEs under Measure Dependent Lyapunov Conditions
Authors:
William Hammersley,
David Šiška,
Lukasz Szpruch
Abstract:
We prove the existence of weak solutions to McKean-Vlasov SDEs defined on a domain $D \subseteq \mathbb{R}^d$ with continuous and unbounded coefficients that satisfy Lyapunov type conditions, where the Lyapunov function may depend on measure. We propose a new type of {\em integrated} Lyapunov condition, where the inequality is only required to hold when integrated against the measure on which the…
▽ More
We prove the existence of weak solutions to McKean-Vlasov SDEs defined on a domain $D \subseteq \mathbb{R}^d$ with continuous and unbounded coefficients that satisfy Lyapunov type conditions, where the Lyapunov function may depend on measure. We propose a new type of {\em integrated} Lyapunov condition, where the inequality is only required to hold when integrated against the measure on which the Lyapunov function depends , and we show that this is sufficient for the existence of weak solutions to McKean-Vlasov SDEs defined on $D$. The main tool used in the proofs is the concept of a measure derivative due to Lions. We prove results on uniqueness under weaker assumptions than that of global Lipschitz continuity of the coefficients.
△ Less
Submitted 30 September, 2020; v1 submitted 12 February, 2018;
originally announced February 2018.
-
$L^p$-estimates and regularity for SPDEs with monotone semilinearity
Authors:
Neelima,
David Šiška
Abstract:
Semilinear stochastic partial differential equations on bounded domains $\mathscr{D}$ are considered. The semilinear term may have arbitrary polynomial growth as long as it is continuous and monotone except perhaps near the origin. Typical examples are the stochastic Allen--Cahn and Ginzburg--Landau equations. The first main result of this article are $L^p$-estimates for such equations. The $L^p$-…
▽ More
Semilinear stochastic partial differential equations on bounded domains $\mathscr{D}$ are considered. The semilinear term may have arbitrary polynomial growth as long as it is continuous and monotone except perhaps near the origin. Typical examples are the stochastic Allen--Cahn and Ginzburg--Landau equations. The first main result of this article are $L^p$-estimates for such equations. The $L^p$-estimates are subsequently employed in obtaining higher regularity. This is motivated by ongoing work to obtain rate of convergence estimates for numerical approximations to such equations. It is shown, under appropriate assumptions, that the solution is continuous in time with values in the Sobolev space $H^2(\mathscr{D}')$ and $\ell^2$-integrable with values in $H^3(\mathscr{D}')$, for any compact $\mathscr{D}' \subset \mathscr{D}$. Using results from $L^p$-theory of SPDEs obtained by Kim~\cite{kim04} we get analogous results in weighted Sobolev spaces on the whole $\mathscr{D}$. Finally it is shown that the solution is Hölder continuous in time of order $\frac{1}{2} - \frac{2}{q}$ as a process with values in a weighted $L^q$-space, where $q$ arises from the integrability assumptions imposed on the initial condition and forcing terms.
△ Less
Submitted 24 September, 2019; v1 submitted 29 May, 2017;
originally announced May 2017.
-
Coercivity condition for higher order moments for nonlinear SPDEs and existence of solution under local monotonicity
Authors:
Neelima,
David Šiška
Abstract:
Higher order moment estimates for solutions to nonlinear SPDEs governed by locally-monotone operators are obtained under appropriate coercivity condition. These are then used to extend known existence and uniqueness results for nonlinear SPDEs under local monotonicity conditions to allow derivatives in the operator acting on the solution under the stochastic integral.
Higher order moment estimates for solutions to nonlinear SPDEs governed by locally-monotone operators are obtained under appropriate coercivity condition. These are then used to extend known existence and uniqueness results for nonlinear SPDEs under local monotonicity conditions to allow derivatives in the operator acting on the solution under the stochastic integral.
△ Less
Submitted 9 August, 2019; v1 submitted 18 October, 2016;
originally announced October 2016.
-
Itô Formula for Processes Taking Values in Intersection of Finitely Many Banach Spaces
Authors:
István Gyöngy,
David Šiška
Abstract:
Motivated by applications to SPDEs we extend the Itô formula for the square of the norm of a semimartingale $y(t)$ from Gyöngy and Krylov (Stochastics 6(3):153-173, 1982) to the case \begin{equation*} \sum_{i=1}^m \int_{(0,t]} v_i^{\ast}(s)\,dA(s) + h(t)=:y(t)\in V \quad \text{$dA\times \mathbb{P}$-a.e.}, \end{equation*} where $A$ is an increasing right-continuous adapted process, $v_i^{\ast}$ is…
▽ More
Motivated by applications to SPDEs we extend the Itô formula for the square of the norm of a semimartingale $y(t)$ from Gyöngy and Krylov (Stochastics 6(3):153-173, 1982) to the case \begin{equation*} \sum_{i=1}^m \int_{(0,t]} v_i^{\ast}(s)\,dA(s) + h(t)=:y(t)\in V \quad \text{$dA\times \mathbb{P}$-a.e.}, \end{equation*} where $A$ is an increasing right-continuous adapted process, $v_i^{\ast}$ is a progressively measurable process with values in $V_i^{\ast}$, the dual of a Banach space $V_i$, $h$ is a cadlag martingale with values in a Hilbert space $H$, identified with its dual $H^{\ast}$, and $V:=V_1\cap V_2 \cap \ldots \cap V_m$ is continuously and densely embedded in $H$.
The formula is proved under the condition that $\|y\|_{V_i}^{p_i}$ and $\|v_i^\ast\|_{V_i^\ast}^{q_i}$ are almost surely locally integrable with respect to $dA$ for some conjugate exponents $p_i, q_i$. This condition is essentially weaker than the one which would arise in application of the results in Gyöngy and Krylov (Stochastics 6(3):153-173, 1982) to the semimartingale above.
△ Less
Submitted 20 March, 2017; v1 submitted 5 September, 2016;
originally announced September 2016.
-
Nonlinear stochastic evolution equations of second order with dam**
Authors:
Etienne Emmrich,
David Šiška
Abstract:
Convergence of a full discretization of a second order stochastic evolution equation with nonlinear dam** is shown and thus existence of a solution is established. The discretization scheme combines an implicit time step** scheme with an internal approximation. Uniqueness is proved as well.
Convergence of a full discretization of a second order stochastic evolution equation with nonlinear dam** is shown and thus existence of a solution is established. The discretization scheme combines an implicit time step** scheme with an internal approximation. Uniqueness is proved as well.
△ Less
Submitted 11 October, 2016; v1 submitted 31 December, 2015;
originally announced December 2015.
-
Convergence of tamed Euler schemes for a class of stochastic evolution equations
Authors:
István Gyöngy,
Sotirios Sabanis,
David Šiška
Abstract:
We prove stability and convergence of a full discretization for a class of stochastic evolution equations with super-linearly growing operators appearing in the drift term. This is done using the recently developed tamed Euler method, which uses a fully explicit time step**, coupled with a Galerkin scheme for the spatial discretization.
We prove stability and convergence of a full discretization for a class of stochastic evolution equations with super-linearly growing operators appearing in the drift term. This is done using the recently developed tamed Euler method, which uses a fully explicit time step**, coupled with a Galerkin scheme for the spatial discretization.
△ Less
Submitted 13 August, 2015; v1 submitted 26 July, 2014;
originally announced July 2014.
-
Error estimates for finite difference approximations of American put option price
Authors:
David Šiška
Abstract:
Finite difference approximations to multi-asset American put option price are considered. The assets are modelled as a multi-dimensional diffusion process with variable drift and volatility. Approximation error of order one quarter with respect to the time discretisation parameter and one half with respect to the space discretisation parameter is proved by reformulating the corresponding optimal s…
▽ More
Finite difference approximations to multi-asset American put option price are considered. The assets are modelled as a multi-dimensional diffusion process with variable drift and volatility. Approximation error of order one quarter with respect to the time discretisation parameter and one half with respect to the space discretisation parameter is proved by reformulating the corresponding optimal stop** problem as a solution of a degenerate Hamilton-Jacobi-Bellman equation. Furthermore, the error arising from restricting the discrete problem to a finite grid by reducing the original problem to a bounded domain is estimated.
△ Less
Submitted 30 September, 2011; v1 submitted 19 September, 2011;
originally announced September 2011.
-
On randomized stop**
Authors:
Istvan Gyongy,
David Siska
Abstract:
A general result on the method of randomized stop** is proved. It is applied to optimal stop** of controlled diffusion processes with unbounded coefficients to reduce it to an optimal control problem without stop**. This is motivated by recent results of Krylov on numerical solutions to the Bellman equation.
A general result on the method of randomized stop** is proved. It is applied to optimal stop** of controlled diffusion processes with unbounded coefficients to reduce it to an optimal control problem without stop**. This is motivated by recent results of Krylov on numerical solutions to the Bellman equation.
△ Less
Submitted 15 May, 2008; v1 submitted 16 May, 2007;
originally announced May 2007.
-
On finite-difference approximations for normalized Bellman equations
Authors:
István Gyöngy,
David Šiška
Abstract:
A class of stochastic optimal control problems involving optimal stop** is considered. Methods of Krylov are adapted to investigate the numerical solutions of the corresponding normalized Bellman equations and to estimate the rate of convergence of finite difference approximations for the optimal reward functions.
A class of stochastic optimal control problems involving optimal stop** is considered. Methods of Krylov are adapted to investigate the numerical solutions of the corresponding normalized Bellman equations and to estimate the rate of convergence of finite difference approximations for the optimal reward functions.
△ Less
Submitted 17 December, 2014; v1 submitted 27 October, 2006;
originally announced October 2006.
