-
Weighted mesh algorithms for general Markov decision processes: Convergence and tractability
Authors:
Denis Belomestny,
John Schoenmakers
Abstract:
We introduce a mesh-type approach for tackling discrete-time, finite-horizon Markov Decision Processes (MDPs) characterized by state and action spaces that are general, encompassing both finite and infinite (yet suitably regular) subsets of Euclidean space. In particular, for bounded state and action spaces, our algorithm achieves a computational complexity that is tractable in the sense of Novak…
▽ More
We introduce a mesh-type approach for tackling discrete-time, finite-horizon Markov Decision Processes (MDPs) characterized by state and action spaces that are general, encompassing both finite and infinite (yet suitably regular) subsets of Euclidean space. In particular, for bounded state and action spaces, our algorithm achieves a computational complexity that is tractable in the sense of Novak and Wozniakowski, and is polynomial in the time horizon. For unbounded state space the algorithm is "semi-tractable" in the sense that the complexity is proportional to $ε^{-c}$ with some dimension independent $c\geq2$, for achieving an accuracy $ε$, and polynomial in the time horizon with degree linear in the underlying dimension. As such the proposed approach has some flavor of the randomization method by Rust which deals with infinite horizon MDPs and uniform sampling in compact state space. However, the present approach is essentially different due to the finite horizon and a simulation procedure due to general transition distributions, and more general in the sense that it encompasses unbounded state space. To demonstrate the effectiveness of our algorithm, we provide illustrations based on Linear-Quadratic Gaussian (LQG) control problems.
△ Less
Submitted 29 June, 2024;
originally announced July 2024.
-
Decompounding Under General Mixing Distributions
Authors:
Denis Belomestny,
Ekaterina Morozova,
Vladimir Panov
Abstract:
This study focuses on statistical inference for compound models of the form $X=ξ_1+\ldots+ξ_N$, where $N$ is a random variable denoting the count of summands, which are independent and identically distributed (i.i.d.) random variables $ξ_1, ξ_2, \ldots$. The paper addresses the problem of reconstructing the distribution of $ξ$ from observed samples of $X$'s distribution, a process referred to as d…
▽ More
This study focuses on statistical inference for compound models of the form $X=ξ_1+\ldots+ξ_N$, where $N$ is a random variable denoting the count of summands, which are independent and identically distributed (i.i.d.) random variables $ξ_1, ξ_2, \ldots$. The paper addresses the problem of reconstructing the distribution of $ξ$ from observed samples of $X$'s distribution, a process referred to as decompounding, with the assumption that $N$'s distribution is known. This work diverges from the conventional scope by not limiting $N$'s distribution to the Poisson type, thus embracing a broader context. We propose a nonparametric estimate for the density of $ξ$, derive its rates of convergence and prove that these rates are minimax optimal for suitable classes of distributions for $ξ$ and $N$. Finally, we illustrate the numerical performance of the algorithm on simulated examples.
△ Less
Submitted 8 May, 2024;
originally announced May 2024.
-
On nonparametric estimation of the interaction function in particle system models
Authors:
Denis Belomestny,
Mark Podolskij,
Shi-Yuan Zhou
Abstract:
This paper delves into a nonparametric estimation approach for the interaction function within diffusion-type particle system models. We introduce two estimation methods based upon an empirical risk minimization. Our study encompasses an analysis of the stochastic and approximation errors associated with both procedures, along with an examination of certain minimax lower bounds. In particular, we…
▽ More
This paper delves into a nonparametric estimation approach for the interaction function within diffusion-type particle system models. We introduce two estimation methods based upon an empirical risk minimization. Our study encompasses an analysis of the stochastic and approximation errors associated with both procedures, along with an examination of certain minimax lower bounds. In particular, we show that there is a natural metric under which the corresponding minimax estimation error of the interaction function converges to zero with parametric rate. This result is rather suprising given complexity of the underlying estimation problem and rather large classes of interaction functions for which the above parametric rate holds.
△ Less
Submitted 22 February, 2024;
originally announced February 2024.
-
Polynomial rates via deconvolution for nonparametric estimation in McKean-Vlasov SDEs
Authors:
Chiara Amorino,
Denis Belomestny,
Vytautė Pilipauskaitė,
Mark Podolskij,
Shi-Yuan Zhou
Abstract:
This paper investigates the estimation of the interaction function for a class of McKean-Vlasov stochastic differential equations. The estimation is based on observations of the associated particle system at time $T$, considering the scenario where both the time horizon $T$ and the number of particles $N$ tend to infinity. Our proposed method recovers polynomial rates of convergence for the result…
▽ More
This paper investigates the estimation of the interaction function for a class of McKean-Vlasov stochastic differential equations. The estimation is based on observations of the associated particle system at time $T$, considering the scenario where both the time horizon $T$ and the number of particles $N$ tend to infinity. Our proposed method recovers polynomial rates of convergence for the resulting estimator. This is achieved under the assumption of exponentially decaying tails for the interaction function. Additionally, we conduct a thorough analysis of the transform of the associated invariant density as a complex function, providing essential insights for our main results.
△ Less
Submitted 10 January, 2024; v1 submitted 9 January, 2024;
originally announced January 2024.
-
Model-free Posterior Sampling via Learning Rate Randomization
Authors:
Daniil Tiapkin,
Denis Belomestny,
Daniele Calandriello,
Eric Moulines,
Remi Munos,
Alexey Naumov,
Pierre Perrault,
Michal Valko,
Pierre Menard
Abstract:
In this paper, we introduce Randomized Q-learning (RandQL), a novel randomized model-free algorithm for regret minimization in episodic Markov Decision Processes (MDPs). To the best of our knowledge, RandQL is the first tractable model-free posterior sampling-based algorithm. We analyze the performance of RandQL in both tabular and non-tabular metric space settings. In tabular MDPs, RandQL achieve…
▽ More
In this paper, we introduce Randomized Q-learning (RandQL), a novel randomized model-free algorithm for regret minimization in episodic Markov Decision Processes (MDPs). To the best of our knowledge, RandQL is the first tractable model-free posterior sampling-based algorithm. We analyze the performance of RandQL in both tabular and non-tabular metric space settings. In tabular MDPs, RandQL achieves a regret bound of order $\widetilde{\mathcal{O}}(\sqrt{H^{5}SAT})$, where $H$ is the planning horizon, $S$ is the number of states, $A$ is the number of actions, and $T$ is the number of episodes. For a metric state-action space, RandQL enjoys a regret bound of order $\widetilde{\mathcal{O}}(H^{5/2} T^{(d_z+1)/(d_z+2)})$, where $d_z$ denotes the zooming dimension. Notably, RandQL achieves optimistic exploration without using bonuses, relying instead on a novel idea of learning rate randomization. Our empirical study shows that RandQL outperforms existing approaches on baseline exploration environments.
△ Less
Submitted 27 October, 2023;
originally announced October 2023.
-
Demonstration-Regularized RL
Authors:
Daniil Tiapkin,
Denis Belomestny,
Daniele Calandriello,
Eric Moulines,
Alexey Naumov,
Pierre Perrault,
Michal Valko,
Pierre Menard
Abstract:
Incorporating expert demonstrations has empirically helped to improve the sample efficiency of reinforcement learning (RL). This paper quantifies theoretically to what extent this extra information reduces RL's sample complexity. In particular, we study the demonstration-regularized reinforcement learning that leverages the expert demonstrations by KL-regularization for a policy learned by behavio…
▽ More
Incorporating expert demonstrations has empirically helped to improve the sample efficiency of reinforcement learning (RL). This paper quantifies theoretically to what extent this extra information reduces RL's sample complexity. In particular, we study the demonstration-regularized reinforcement learning that leverages the expert demonstrations by KL-regularization for a policy learned by behavior cloning. Our findings reveal that using $N^{\mathrm{E}}$ expert demonstrations enables the identification of an optimal policy at a sample complexity of order $\widetilde{O}(\mathrm{Poly}(S,A,H)/(\varepsilon^2 N^{\mathrm{E}}))$ in finite and $\widetilde{O}(\mathrm{Poly}(d,H)/(\varepsilon^2 N^{\mathrm{E}}))$ in linear Markov decision processes, where $\varepsilon$ is the target precision, $H$ the horizon, $A$ the number of action, $S$ the number of states in the finite case and $d$ the dimension of the feature space in the linear case. As a by-product, we provide tight convergence guarantees for the behaviour cloning procedure under general assumptions on the policy classes. Additionally, we establish that demonstration-regularized methods are provably efficient for reinforcement learning from human feedback (RLHF). In this respect, we provide theoretical evidence showing the benefits of KL-regularization for RLHF in tabular and linear MDPs. Interestingly, we avoid pessimism injection by employing computationally feasible regularization to handle reward estimation uncertainty, thus setting our approach apart from the prior works.
△ Less
Submitted 10 June, 2024; v1 submitted 26 October, 2023;
originally announced October 2023.
-
Nonparametric Bayesian inference for stochastic processes with piecewise constant priors
Authors:
Denis Belomestny,
Frank van der Meulen,
Peter Spreij
Abstract:
We present a survey of some of our recent results on Bayesian nonparametric inference for a multitude of stochastic processes. The common feature is that the prior distribution in the cases considered is on suitable sets of piecewise constant or piecewise linear functions, that differ for the specific situations at hand. Posterior consistency and in most cases contraction rates for the estimators…
▽ More
We present a survey of some of our recent results on Bayesian nonparametric inference for a multitude of stochastic processes. The common feature is that the prior distribution in the cases considered is on suitable sets of piecewise constant or piecewise linear functions, that differ for the specific situations at hand. Posterior consistency and in most cases contraction rates for the estimators are presented. Numerical studies on simulated and real data accompany the theoretical results.
△ Less
Submitted 17 May, 2023; v1 submitted 12 May, 2023;
originally announced May 2023.
-
Sharp Deviations Bounds for Dirichlet Weighted Sums with Application to analysis of Bayesian algorithms
Authors:
Denis Belomestny,
Pierre Menard,
Alexey Naumov,
Daniil Tiapkin,
Michal Valko
Abstract:
In this work, we derive sharp non-asymptotic deviation bounds for weighted sums of Dirichlet random variables. These bounds are based on a novel integral representation of the density of a weighted Dirichlet sum. This representation allows us to obtain a Gaussian-like approximation for the sum distribution using geometry and complex analysis methods. Our results generalize similar bounds for the B…
▽ More
In this work, we derive sharp non-asymptotic deviation bounds for weighted sums of Dirichlet random variables. These bounds are based on a novel integral representation of the density of a weighted Dirichlet sum. This representation allows us to obtain a Gaussian-like approximation for the sum distribution using geometry and complex analysis methods. Our results generalize similar bounds for the Beta distribution obtained in the seminal paper Alfers and Dinges [1984]. Additionally, our results can be considered a sharp non-asymptotic version of the inverse of Sanov's theorem studied by Ganesh and O'Connell [1999] in the Bayesian setting. Based on these results, we derive new deviation bounds for the Dirichlet process posterior means with application to Bayesian bootstrap. Finally, we apply our estimates to the analysis of the Multinomial Thompson Sampling (TS) algorithm in multi-armed bandits and significantly sharpen the existing regret bounds by making them independent of the size of the arms distribution support.
△ Less
Submitted 6 April, 2023;
originally announced April 2023.
-
Theoretical guarantees for neural control variates in MCMC
Authors:
Denis Belomestny,
Artur Goldman,
Alexey Naumov,
Sergey Samsonov
Abstract:
In this paper, we propose a variance reduction approach for Markov chains based on additive control variates and the minimization of an appropriate estimate for the asymptotic variance. We focus on the particular case when control variates are represented as deep neural networks. We derive the optimal convergence rate of the asymptotic variance under various ergodicity assumptions on the underlyin…
▽ More
In this paper, we propose a variance reduction approach for Markov chains based on additive control variates and the minimization of an appropriate estimate for the asymptotic variance. We focus on the particular case when control variates are represented as deep neural networks. We derive the optimal convergence rate of the asymptotic variance under various ergodicity assumptions on the underlying Markov chain. The proposed approach relies upon recent results on the stochastic errors of variance reduction algorithms and function approximation theory.
△ Less
Submitted 3 April, 2023;
originally announced April 2023.
-
Fast Rates for Maximum Entropy Exploration
Authors:
Daniil Tiapkin,
Denis Belomestny,
Daniele Calandriello,
Eric Moulines,
Remi Munos,
Alexey Naumov,
Pierre Perrault,
Yunhao Tang,
Michal Valko,
Pierre Menard
Abstract:
We address the challenge of exploration in reinforcement learning (RL) when the agent operates in an unknown environment with sparse or no rewards. In this work, we study the maximum entropy exploration problem of two different types. The first type is visitation entropy maximization previously considered by Hazan et al.(2019) in the discounted setting. For this type of exploration, we propose a g…
▽ More
We address the challenge of exploration in reinforcement learning (RL) when the agent operates in an unknown environment with sparse or no rewards. In this work, we study the maximum entropy exploration problem of two different types. The first type is visitation entropy maximization previously considered by Hazan et al.(2019) in the discounted setting. For this type of exploration, we propose a game-theoretic algorithm that has $\widetilde{\mathcal{O}}(H^3S^2A/\varepsilon^2)$ sample complexity thus improving the $\varepsilon$-dependence upon existing results, where $S$ is a number of states, $A$ is a number of actions, $H$ is an episode length, and $\varepsilon$ is a desired accuracy. The second type of entropy we study is the trajectory entropy. This objective function is closely related to the entropy-regularized MDPs, and we propose a simple algorithm that has a sample complexity of order $\widetilde{\mathcal{O}}(\mathrm{poly}(S,A,H)/\varepsilon)$. Interestingly, it is the first theoretical result in RL literature that establishes the potential statistical advantage of regularized MDPs for exploration. Finally, we apply developed regularization techniques to reduce sample complexity of visitation entropy maximization to $\widetilde{\mathcal{O}}(H^2SA/\varepsilon^2)$, yielding a statistical separation between maximum entropy exploration and reward-free exploration.
△ Less
Submitted 6 June, 2023; v1 submitted 14 March, 2023;
originally announced March 2023.
-
Spectral bootstrap confidence bands for Lévy-driven moving average processes
Authors:
D. Belomestny,
E. Ivanova,
T. Orlova
Abstract:
In this paper we study the problem of constructing bootstrap confidence intervals for the Lévy density of the driving Lévy process based on high-frequency observations of a Lévy-driven moving average processes. Using a spectral estimator of the Lévy density, we propose a novel implementations of multiplier and empirical bootstraps to construct confidence bands on a compact set away from the origin…
▽ More
In this paper we study the problem of constructing bootstrap confidence intervals for the Lévy density of the driving Lévy process based on high-frequency observations of a Lévy-driven moving average processes. Using a spectral estimator of the Lévy density, we propose a novel implementations of multiplier and empirical bootstraps to construct confidence bands on a compact set away from the origin. We also provide conditions under which the confidence bands are asymptotically valid.
△ Less
Submitted 12 November, 2022;
originally announced November 2022.
-
Statistical Inference for Scale Mixture Models via Mellin Transform Approach
Authors:
Denis Belomestny,
Ekaterina Morozova,
Vladimir Panov
Abstract:
This paper deals with statistical inference for the scale mixture models. We study an estimation approach based on the Mellin -- Stieltjes transform that can be applied to both discrete and absolute continuous mixing distributions. The accuracy of the corresponding estimate is analysed in terms of its expected pointwise error. As an important technical result, we prove the analogue of the Berry --…
▽ More
This paper deals with statistical inference for the scale mixture models. We study an estimation approach based on the Mellin -- Stieltjes transform that can be applied to both discrete and absolute continuous mixing distributions. The accuracy of the corresponding estimate is analysed in terms of its expected pointwise error. As an important technical result, we prove the analogue of the Berry -- Esseen inequality for the Mellin transforms. The proposed statistical approach is illustrated by numerical examples.
△ Less
Submitted 22 January, 2024; v1 submitted 3 November, 2022;
originally announced November 2022.
-
Primal-dual regression approach for Markov decision processes with general state and action space
Authors:
Denis Belomestny,
John Schoenmakers
Abstract:
We develop a regression based primal-dual martingale approach for solving finite time horizon MDPs with general state and action space. As a result, our method allows for the construction of tight upper and lower biased approximations of the value functions, and, provides tight approximations to the optimal policy. In particular, we prove tight error bounds for the estimated duality gap featuring…
▽ More
We develop a regression based primal-dual martingale approach for solving finite time horizon MDPs with general state and action space. As a result, our method allows for the construction of tight upper and lower biased approximations of the value functions, and, provides tight approximations to the optimal policy. In particular, we prove tight error bounds for the estimated duality gap featuring polynomial dependence on the time horizon, and sublinear dependence on the cardinality/dimension of the possibly infinite state and action space.From a computational point of view the proposed method is efficient since, in contrast to usual duality-based methods for optimal control problems in the literature, the Monte Carlo procedures here involved do not require nested simulations.
△ Less
Submitted 4 October, 2022; v1 submitted 1 October, 2022;
originally announced October 2022.
-
Optimistic Posterior Sampling for Reinforcement Learning with Few Samples and Tight Guarantees
Authors:
Daniil Tiapkin,
Denis Belomestny,
Daniele Calandriello,
Eric Moulines,
Remi Munos,
Alexey Naumov,
Mark Rowland,
Michal Valko,
Pierre Menard
Abstract:
We consider reinforcement learning in an environment modeled by an episodic, finite, stage-dependent Markov decision process of horizon $H$ with $S$ states, and $A$ actions. The performance of an agent is measured by the regret after interacting with the environment for $T$ episodes. We propose an optimistic posterior sampling algorithm for reinforcement learning (OPSRL), a simple variant of poste…
▽ More
We consider reinforcement learning in an environment modeled by an episodic, finite, stage-dependent Markov decision process of horizon $H$ with $S$ states, and $A$ actions. The performance of an agent is measured by the regret after interacting with the environment for $T$ episodes. We propose an optimistic posterior sampling algorithm for reinforcement learning (OPSRL), a simple variant of posterior sampling that only needs a number of posterior samples logarithmic in $H$, $S$, $A$, and $T$ per state-action pair. For OPSRL we guarantee a high-probability regret bound of order at most $\widetilde{\mathcal{O}}(\sqrt{H^3SAT})$ ignoring $\text{poly}\log(HSAT)$ terms. The key novel technical ingredient is a new sharp anti-concentration inequality for linear forms which may be of independent interest. Specifically, we extend the normal approximation-based lower bound for Beta distributions by Alfers and Dinges [1984] to Dirichlet distributions. Our bound matches the lower bound of order $Ω(\sqrt{H^3SAT})$, thereby answering the open problems raised by Agrawal and Jia [2017b] for the episodic setting.
△ Less
Submitted 28 September, 2022;
originally announced September 2022.
-
Simultaneous approximation of a smooth function and its derivatives by deep neural networks with piecewise-polynomial activations
Authors:
Denis Belomestny,
Alexey Naumov,
Nikita Puchkin,
Sergey Samsonov
Abstract:
This paper investigates the approximation properties of deep neural networks with piecewise-polynomial activation functions. We derive the required depth, width, and sparsity of a deep neural network to approximate any Hölder smooth function up to a given approximation error in Hölder norms in such a way that all weights of this neural network are bounded by $1$. The latter feature is essential to…
▽ More
This paper investigates the approximation properties of deep neural networks with piecewise-polynomial activation functions. We derive the required depth, width, and sparsity of a deep neural network to approximate any Hölder smooth function up to a given approximation error in Hölder norms in such a way that all weights of this neural network are bounded by $1$. The latter feature is essential to control generalization errors in many statistical and machine learning applications.
△ Less
Submitted 2 December, 2022; v1 submitted 19 June, 2022;
originally announced June 2022.
-
Variance Reduction for Policy-Gradient Methods via Empirical Variance Minimization
Authors:
Maxim Kaledin,
Alexander Golubev,
Denis Belomestny
Abstract:
Policy-gradient methods in Reinforcement Learning(RL) are very universal and widely applied in practice but their performance suffers from the high variance of the gradient estimate. Several procedures were proposed to reduce it including actor-critic(AC) and advantage actor-critic(A2C) methods. Recently the approaches have got new perspective due to the introduction of Deep RL: both new control v…
▽ More
Policy-gradient methods in Reinforcement Learning(RL) are very universal and widely applied in practice but their performance suffers from the high variance of the gradient estimate. Several procedures were proposed to reduce it including actor-critic(AC) and advantage actor-critic(A2C) methods. Recently the approaches have got new perspective due to the introduction of Deep RL: both new control variates(CV) and new sub-sampling procedures became available in the setting of complex models like neural networks. The vital part of CV-based methods is the goal functional for the training of the CV, the most popular one is the least-squares criterion of A2C. Despite its practical success, the criterion is not the only one possible. In this paper we for the first time investigate the performance of the one called Empirical Variance(EV). We observe in the experiments that not only EV-criterion performs not worse than A2C but sometimes can be considerably better. Apart from that, we also prove some theoretical guarantees of the actual variance reduction under very general assumptions and show that A2C least-squares goal functional is an upper bound for EV goal. Our experiments indicate that in terms of variance reduction EV-based methods are much better than A2C and allow stronger variance reduction.
△ Less
Submitted 15 June, 2022; v1 submitted 14 June, 2022;
originally announced June 2022.
-
From Dirichlet to Rubin: Optimistic Exploration in RL without Bonuses
Authors:
Daniil Tiapkin,
Denis Belomestny,
Eric Moulines,
Alexey Naumov,
Sergey Samsonov,
Yunhao Tang,
Michal Valko,
Pierre Menard
Abstract:
We propose the Bayes-UCBVI algorithm for reinforcement learning in tabular, stage-dependent, episodic Markov decision process: a natural extension of the Bayes-UCB algorithm by Kaufmann et al. (2012) for multi-armed bandits. Our method uses the quantile of a Q-value function posterior as upper confidence bound on the optimal Q-value function. For Bayes-UCBVI, we prove a regret bound of order…
▽ More
We propose the Bayes-UCBVI algorithm for reinforcement learning in tabular, stage-dependent, episodic Markov decision process: a natural extension of the Bayes-UCB algorithm by Kaufmann et al. (2012) for multi-armed bandits. Our method uses the quantile of a Q-value function posterior as upper confidence bound on the optimal Q-value function. For Bayes-UCBVI, we prove a regret bound of order $\widetilde{O}(\sqrt{H^3SAT})$ where $H$ is the length of one episode, $S$ is the number of states, $A$ the number of actions, $T$ the number of episodes, that matches the lower-bound of $Ω(\sqrt{H^3SAT})$ up to poly-$\log$ terms in $H,S,A,T$ for a large enough $T$. To the best of our knowledge, this is the first algorithm that obtains an optimal dependence on the horizon $H$ (and $S$) without the need for an involved Bernstein-like bonus or noise. Crucial to our analysis is a new fine-grained anti-concentration bound for a weighted Dirichlet sum that can be of independent interest. We then explain how Bayes-UCBVI can be easily extended beyond the tabular setting, exhibiting a strong link between our algorithm and Bayesian bootstrap (Rubin, 1981).
△ Less
Submitted 22 June, 2022; v1 submitted 16 May, 2022;
originally announced May 2022.
-
A Reproducing Kernel Hilbert Space approach to singular local stochastic volatility McKean-Vlasov models
Authors:
Christian Bayer,
Denis Belomestny,
Oleg Butkovsky,
John Schoenmakers
Abstract:
Motivated by the challenges related to the calibration of financial models, we consider the problem of numerically solving a singular McKean-Vlasov equation $$ d X_t= σ(t,X_t) X_t \frac{\sqrt v_t}{\sqrt {E[v_t|X_t]}}dW_t, $$ where $W$ is a Brownian motion and $v$ is an adapted diffusion process. This equation can be considered as a singular local stochastic volatility model. Whilst such models are…
▽ More
Motivated by the challenges related to the calibration of financial models, we consider the problem of numerically solving a singular McKean-Vlasov equation $$ d X_t= σ(t,X_t) X_t \frac{\sqrt v_t}{\sqrt {E[v_t|X_t]}}dW_t, $$ where $W$ is a Brownian motion and $v$ is an adapted diffusion process. This equation can be considered as a singular local stochastic volatility model. Whilst such models are quite popular among practitioners, unfortunately, its well-posedness has not been fully understood yet and, in general, is possibly not guaranteed at all. We develop a novel regularization approach based on the reproducing kernel Hilbert space (RKHS) technique and show that the regularized model is well-posed. Furthermore, we prove propagation of chaos. We demonstrate numerically that a thus regularized model is able to perfectly replicate option prices due to typical local volatility models. Our results are also applicable to more general McKean--Vlasov equations.
△ Less
Submitted 12 January, 2024; v1 submitted 2 March, 2022;
originally announced March 2022.
-
Weak solutions to gamma-driven stochastic differential equations
Authors:
Denis Belomestny,
Shota Gugushvili,
Moritz Schauer,
Peter Spreij
Abstract:
We study a stochastic differential equation driven by a gamma process, for which we give results on the existence of weak solutions under conditions on the volatility function. To that end we provide results on the density process between the laws of solutions with different volatility functions.
We study a stochastic differential equation driven by a gamma process, for which we give results on the existence of weak solutions under conditions on the volatility function. To that end we provide results on the density process between the laws of solutions with different volatility functions.
△ Less
Submitted 26 August, 2021;
originally announced August 2021.
-
Semiparametric estimation of McKean-Vlasov SDEs
Authors:
Denis Belomestny,
Vytautė Pilipauskaitė,
Mark Podolskij
Abstract:
In this paper we study the problem of semiparametric estimation for a class of McKean-Vlasov stochastic differential equations. Our aim is to estimate the drift coefficient of a MV-SDE based on observations of the corresponding particle system. We propose a semiparametric estimation procedure and derive the rates of convergence for the resulting estimator. We further prove that the obtained rates…
▽ More
In this paper we study the problem of semiparametric estimation for a class of McKean-Vlasov stochastic differential equations. Our aim is to estimate the drift coefficient of a MV-SDE based on observations of the corresponding particle system. We propose a semiparametric estimation procedure and derive the rates of convergence for the resulting estimator. We further prove that the obtained rates are essentially optimal in the minimax sense.
△ Less
Submitted 1 July, 2021;
originally announced July 2021.
-
UVIP: Model-Free Approach to Evaluate Reinforcement Learning Algorithms
Authors:
D. Belomestny,
I. Levin,
E. Moulines,
A. Naumov,
S. Samsonov,
V. Zorina
Abstract:
Policy evaluation is an important instrument for the comparison of different algorithms in Reinforcement Learning (RL). Yet even a precise knowledge of the value function $V^π$ corresponding to a policy $π$ does not provide reliable information on how far is the policy $π$ from the optimal one. We present a novel model-free upper value iteration procedure $({\sf UVIP})$ that allows us to estimate…
▽ More
Policy evaluation is an important instrument for the comparison of different algorithms in Reinforcement Learning (RL). Yet even a precise knowledge of the value function $V^π$ corresponding to a policy $π$ does not provide reliable information on how far is the policy $π$ from the optimal one. We present a novel model-free upper value iteration procedure $({\sf UVIP})$ that allows us to estimate the suboptimality gap $V^{\star}(x) - V^π(x)$ from above and to construct confidence intervals for $V^\star$. Our approach relies on upper bounds to the solution of the Bellman optimality equation via martingale approach. We provide theoretical guarantees for ${\sf UVIP}$ under general assumptions and illustrate its performance on a number of benchmark RL problems.
△ Less
Submitted 3 June, 2021; v1 submitted 5 May, 2021;
originally announced May 2021.
-
From optimal martingales to randomized dual optimal stop**
Authors:
Denis Belomestny,
John Schoenmakers
Abstract:
In this article we study and classify optimal martingales in the dual formulation of optimal stop** problems. In this respect we distinguish between weakly optimal and surely optimal martingales. It is shown that the family of weakly optimal and surely optimal martingales may be quite large. On the other hand it is shown that the Doob-martingale, that is, the martingale part of the Snell envelop…
▽ More
In this article we study and classify optimal martingales in the dual formulation of optimal stop** problems. In this respect we distinguish between weakly optimal and surely optimal martingales. It is shown that the family of weakly optimal and surely optimal martingales may be quite large. On the other hand it is shown that the Doob-martingale, that is, the martingale part of the Snell envelope, is in a certain sense the most robust surely optimal martingale under random perturbations. This new insight leads to a novel randomized dual martingale minimization algorithm that doesn't require nested simulation. As a main feature, in a possibly large family of optimal martingales the algorithm efficiently selects a martingale that is as close as possible to the Doob martingale. As a result, one obtains the dual upper bound for the optimal stop** problem with low variance.
△ Less
Submitted 2 February, 2021;
originally announced February 2021.
-
Rates of convergence for density estimation with generative adversarial networks
Authors:
Nikita Puchkin,
Sergey Samsonov,
Denis Belomestny,
Eric Moulines,
Alexey Naumov
Abstract:
In this work we undertake a thorough study of the non-asymptotic properties of the vanilla generative adversarial networks (GANs). We prove an oracle inequality for the Jensen-Shannon (JS) divergence between the underlying density $\mathsf{p}^*$ and the GAN estimate with a significantly better statistical error term compared to the previously known results. The advantage of our bound becomes clear…
▽ More
In this work we undertake a thorough study of the non-asymptotic properties of the vanilla generative adversarial networks (GANs). We prove an oracle inequality for the Jensen-Shannon (JS) divergence between the underlying density $\mathsf{p}^*$ and the GAN estimate with a significantly better statistical error term compared to the previously known results. The advantage of our bound becomes clear in application to nonparametric density estimation. We show that the JS-divergence between the GAN estimate and $\mathsf{p}^*$ decays as fast as $(\log{n}/n)^{2β/(2β+ d)}$, where $n$ is the sample size and $β$ determines the smoothness of $\mathsf{p}^*$. This rate of convergence coincides (up to logarithmic factors) with minimax optimal for the considered class of densities.
△ Less
Submitted 25 January, 2024; v1 submitted 30 January, 2021;
originally announced February 2021.
-
Reinforced optimal control
Authors:
Christian Bayer,
Denis Belomestny,
Paul Hager,
Paolo Pigato,
John Schoenmakers,
Vladimir Spokoiny
Abstract:
Least squares Monte Carlo methods are a popular numerical approximation method for solving stochastic control problems. Based on dynamic programming, their key feature is the approximation of the conditional expectation of future rewards by linear least squares regression. Hence, the choice of basis functions is crucial for the accuracy of the method. Earlier work by some of us [Belomestny, Schoen…
▽ More
Least squares Monte Carlo methods are a popular numerical approximation method for solving stochastic control problems. Based on dynamic programming, their key feature is the approximation of the conditional expectation of future rewards by linear least squares regression. Hence, the choice of basis functions is crucial for the accuracy of the method. Earlier work by some of us [Belomestny, Schoenmakers, Spokoiny, Zharkynbay. Commun.~Math.~Sci., 18(1):109-121, 2020](ar** problems by already computed value functions for later times, thereby considerably improving the accuracy with limited additional computational cost. We extend the reinforced regression method to a general class of stochastic control problems, while considerably improving the method's efficiency, as demonstrated by substantial numerical examples as well as theoretical analysis.
△ Less
Submitted 25 March, 2022; v1 submitted 24 November, 2020;
originally announced November 2020.
-
Nonparametric Bayesian volatility estimation for gamma-driven stochastic differential equations
Authors:
Denis Belomestny,
Shota Gugushvili,
Moritz Schauer,
Peter Spreij
Abstract:
We study a nonparametric Bayesian approach to estimation of the volatility function of a stochastic differential equation driven by a gamma process. The volatility function is modelled a priori as piecewise constant, and we specify a gamma prior on its values. This leads to a straightforward procedure for posterior inference via an MCMC procedure. We give theoretical performance guarantees (contra…
▽ More
We study a nonparametric Bayesian approach to estimation of the volatility function of a stochastic differential equation driven by a gamma process. The volatility function is modelled a priori as piecewise constant, and we specify a gamma prior on its values. This leads to a straightforward procedure for posterior inference via an MCMC procedure. We give theoretical performance guarantees (contraction rates for the posterior) for the Bayesian estimate in terms of the regularity of the unknown volatility function. We illustrate the method on synthetic and real data examples.
△ Less
Submitted 26 August, 2021; v1 submitted 16 November, 2020;
originally announced November 2020.
-
Variance reduction for dependent sequences with applications to Stochastic Gradient MCMC
Authors:
D. Belomestny,
L. Iosipoi,
E. Moulines,
A. Naumov,
S. Samsonov
Abstract:
In this paper we propose a novel and practical variance reduction approach for additive functionals of dependent sequences. Our approach combines the use of control variates with the minimisation of an empirical variance estimate. We analyse finite sample properties of the proposed method and derive finite-time bounds of the excess asymptotic variance to zero. We apply our methodology to Stochasti…
▽ More
In this paper we propose a novel and practical variance reduction approach for additive functionals of dependent sequences. Our approach combines the use of control variates with the minimisation of an empirical variance estimate. We analyse finite sample properties of the proposed method and derive finite-time bounds of the excess asymptotic variance to zero. We apply our methodology to Stochastic Gradient MCMC (SGMCMC) methods for Bayesian inference on large data sets and combine it with existing variance reduction methods for SGMCMC. We present empirical results carried out on a number of benchmark examples showing that our variance reduction method achieves significant improvement as compared to state-of-the-art methods at the expense of a moderate increase of computational overhead.
△ Less
Submitted 16 August, 2020;
originally announced August 2020.
-
Estimating TVP-VAR models with time invariant long-run multipliers
Authors:
Denis Belomestny,
Ekaterina Krymova,
Andrey Polbin
Abstract:
The main goal of this paper is to develop a methodology for estimating time varying parameter vector auto-regression (TVP-VAR) models with a timeinvariant long-run relationship between endogenous variables and changes in exogenous variables. We propose a Gibbs sampling scheme for estimation of model parameters as well as time-invariant long-run multiplier parameters. Further we demonstrate the app…
▽ More
The main goal of this paper is to develop a methodology for estimating time varying parameter vector auto-regression (TVP-VAR) models with a timeinvariant long-run relationship between endogenous variables and changes in exogenous variables. We propose a Gibbs sampling scheme for estimation of model parameters as well as time-invariant long-run multiplier parameters. Further we demonstrate the applicability of the proposed method by analyzing examples of the Norwegian and Russian economies based on the data on real GDP, real exchange rate and real oil prices. Our results show that incorporating the time invariance constraint on the long-run multipliers in TVP-VAR model helps to significantly improve the forecasting performance.
△ Less
Submitted 3 August, 2020;
originally announced August 2020.
-
Randomized optimal stop** algorithms and their convergence analysis
Authors:
Christian Bayer,
Denis Belomestny,
Paul Hager,
Paolo Pigato,
John Schoenmakers
Abstract:
In this paper we study randomized optimal stop** problems and consider corresponding forward and backward Monte Carlo based optimisation algorithms. In particular we prove the convergence of the proposed algorithms and derive the corresponding convergence rates.
In this paper we study randomized optimal stop** problems and consider corresponding forward and backward Monte Carlo based optimisation algorithms. In particular we prove the convergence of the proposed algorithms and derive the corresponding convergence rates.
△ Less
Submitted 3 February, 2020;
originally announced February 2020.
-
Variance reduction for Markov chains with application to MCMC
Authors:
D. Belomestny,
L. Iosipoi,
E. Moulines,
A. Naumov,
S. Samsonov
Abstract:
In this paper we propose a novel variance reduction approach for additive functionals of Markov chains based on minimization of an estimate for the asymptotic variance of these functionals over suitable classes of control variates. A distinctive feature of the proposed approach is its ability to significantly reduce the overall finite sample variance. This feature is theoretically demonstrated by…
▽ More
In this paper we propose a novel variance reduction approach for additive functionals of Markov chains based on minimization of an estimate for the asymptotic variance of these functionals over suitable classes of control variates. A distinctive feature of the proposed approach is its ability to significantly reduce the overall finite sample variance. This feature is theoretically demonstrated by means of a deep non asymptotic analysis of a variance reduced functional as well as by a thorough simulation study. In particular we apply our method to various MCMC Bayesian estimation problems where it favourably compares to the existing variance reduction approaches.
△ Less
Submitted 15 February, 2020; v1 submitted 8 October, 2019;
originally announced October 2019.
-
Iterative Multilevel density estimation for McKean-Vlasov SDEs via projections
Authors:
Denis Belomestny,
Lukasz Szpruch,
Shuren Tan
Abstract:
In this paper, we present a generic methodology for the efficient numerical approximation of the density function of the McKean-Vlasov SDEs. The weak error analysis for the projected process motivates us to combine the iterative Multilevel Monte Carlo method for McKean-Vlasov SDEs \cite{szpruch2019} with non-interacting kernels and projection estimation of particle densities \cite{belomestny2018pr…
▽ More
In this paper, we present a generic methodology for the efficient numerical approximation of the density function of the McKean-Vlasov SDEs. The weak error analysis for the projected process motivates us to combine the iterative Multilevel Monte Carlo method for McKean-Vlasov SDEs \cite{szpruch2019} with non-interacting kernels and projection estimation of particle densities \cite{belomestny2018projected}. By exploiting smoothness of the coefficients for McKean-Vlasov SDEs, in the best case scenario (i.e $C^{\infty}$ for the coefficients), we obtain the complexity of order $O(ε^{-2}|\logε|^4)$ for the approximation of expectations and $O(ε^{-2}|\logε|^5)$ for density estimation.
△ Less
Submitted 25 September, 2019;
originally announced September 2019.
-
Fourier transform MCMC, heavy tailed distributions and geometric ergodicity
Authors:
Denis Belomestny,
Leonid Iosipoi
Abstract:
Markov Chain Monte Carlo methods become increasingly popular in applied mathematics as a tool for numerical integration with respect to complex and high-dimensional distributions. However, application of MCMC methods to heavy tailed distributions and distributions with analytically intractable densities turns out to be rather problematic. In this paper, we propose a novel approach towards the use…
▽ More
Markov Chain Monte Carlo methods become increasingly popular in applied mathematics as a tool for numerical integration with respect to complex and high-dimensional distributions. However, application of MCMC methods to heavy tailed distributions and distributions with analytically intractable densities turns out to be rather problematic. In this paper, we propose a novel approach towards the use of MCMC algorithms for distributions with analytically known Fourier transforms and, in particular, heavy tailed distributions. The main idea of the proposed approach is to use MCMC methods in Fourier domain to sample from a density proportional to the absolute value of the underlying characteristic function. A subsequent application of the Parseval's formula leads to an efficient algorithm for the computation of integrals with respect to the underlying density. We show that the resulting Markov chain in Fourier domain may be geometrically ergodic even in the case of heavy tailed original distributions. We illustrate our approach by several numerical examples including multivariate elliptically contoured stable distributions.
△ Less
Submitted 31 December, 2019; v1 submitted 2 September, 2019;
originally announced September 2019.
-
Density deconvolution under general assumptions on the distribution of measurement errors
Authors:
Denis Belomestny,
Alexander Goldenshluger
Abstract:
In this paper we study the problem of density deconvolution under general assumptions on the measurement error distribution. Typically deconvolution estimators are constructed using Fourier transform techniques, and it is assumed that the characteristic function of the measurement errors does not have zeros on the real line. This assumption is rather strong and is not fulfilled in many cases of in…
▽ More
In this paper we study the problem of density deconvolution under general assumptions on the measurement error distribution. Typically deconvolution estimators are constructed using Fourier transform techniques, and it is assumed that the characteristic function of the measurement errors does not have zeros on the real line. This assumption is rather strong and is not fulfilled in many cases of interest. In this paper we develop a methodology for constructing optimal density deconvolution estimators in the general setting that covers vanishing and non--vanishing characteristic functions of the measurement errors. We derive upper bounds on the risk of the proposed estimators and provide sufficient conditions under which zeros of the corresponding characteristic function have no effect on estimation accuracy. Moreover, we show that the derived conditions are also necessary in some specific problem instances.
△ Less
Submitted 1 February, 2020; v1 submitted 25 July, 2019;
originally announced July 2019.
-
Semi-tractability of optimal stop** problems via a weighted stochastic mesh algorithm
Authors:
D. Belomestny,
M. Kaledin,
J. Schoenmakers
Abstract:
In this article we propose a Weighted Stochastic Mesh (WSM) Algorithm for approximating the value of a discrete and continuous time optimal stop** problem. We prove that in the discrete case the WSM algorithm leads to
semi-tractability of the corresponding optimal problems in the sense that its complexity is bounded in order by $\varepsilon^{-4}\log^{d+2}(1/\varepsilon)$ with $d$ being the dim…
▽ More
In this article we propose a Weighted Stochastic Mesh (WSM) Algorithm for approximating the value of a discrete and continuous time optimal stop** problem. We prove that in the discrete case the WSM algorithm leads to
semi-tractability of the corresponding optimal problems in the sense that its complexity is bounded in order by $\varepsilon^{-4}\log^{d+2}(1/\varepsilon)$ with $d$ being the dimension of the underlying Markov chain. Furthermore we study the WSM approach in the context of continuous time optimal stop** problems and derive the corresponding complexity bounds. Although we can not prove semi-tractability in this case, our bounds turn out to be the tightest ones among the bounds known for the existing algorithms in the literature. We illustrate our theoretical findings by a numerical example.
△ Less
Submitted 22 June, 2019;
originally announced June 2019.
-
Variance reduction for additive functional of Markov chains via martingale representations
Authors:
D. Belomestny,
E. Moulines,
S. Samsonov
Abstract:
In this paper we propose an efficient variance reduction approach for additive functionals of Markov chains relying on a novel discrete time martingale representation. Our approach is fully non-asymptotic and does not require the knowledge of the stationary distribution (and even any type of ergodicity) or specific structure of the underlying density. By rigorously analyzing the convergence proper…
▽ More
In this paper we propose an efficient variance reduction approach for additive functionals of Markov chains relying on a novel discrete time martingale representation. Our approach is fully non-asymptotic and does not require the knowledge of the stationary distribution (and even any type of ergodicity) or specific structure of the underlying density. By rigorously analyzing the convergence properties of the proposed algorithm, we show that its cost-to-variance product is indeed smaller than one of the naive algorithm. The numerical performance of the new method is illustrated for the Langevin-type Markov Chain Monte Carlo (MCMC) methods.
△ Less
Submitted 21 December, 2021; v1 submitted 18 March, 2019;
originally announced March 2019.
-
Sparse constrained projection approximation subspace tracking
Authors:
Denis Belomestny,
Ekaterina Krymova
Abstract:
In this paper we revisit the well-known constrained projection approximation subspace tracking algorithm (CPAST) and derive, for the first time, non-asymptotic error bounds. Furthermore, we introduce a novel sparse modification of CPAST which is able to exploit sparsity in the underlying covariance structure. We present a non-asymptotic analysis of the proposed algorithm and study its empirical pe…
▽ More
In this paper we revisit the well-known constrained projection approximation subspace tracking algorithm (CPAST) and derive, for the first time, non-asymptotic error bounds. Furthermore, we introduce a novel sparse modification of CPAST which is able to exploit sparsity in the underlying covariance structure. We present a non-asymptotic analysis of the proposed algorithm and study its empirical performance on simulated and real data.
△ Less
Submitted 23 November, 2018; v1 submitted 22 October, 2018;
originally announced October 2018.
-
Optimal stop** via reinforced regression
Authors:
Denis Belomestny,
John Schoenmakers,
Vladimir Spokoiny,
Bakhyt Zharkynbay
Abstract:
In this note we propose a new approach towards solving numerically optimal stop** problems via reinforced regression based Monte Carlo algorithms. The main idea of the method is to reinforce standard linear regression algorithms in each backward induction step by adding new basis functions based on previously estimated continuation values. The proposed methodology is illustrated by a numerical e…
▽ More
In this note we propose a new approach towards solving numerically optimal stop** problems via reinforced regression based Monte Carlo algorithms. The main idea of the method is to reinforce standard linear regression algorithms in each backward induction step by adding new basis functions based on previously estimated continuation values. The proposed methodology is illustrated by a numerical example from mathematical finance.
△ Less
Submitted 1 July, 2019; v1 submitted 7 August, 2018;
originally announced August 2018.
-
Optimal stop** of McKean-Vlasov diffusions via regression on particle systems
Authors:
Denis Belomestny,
John Schoenmakers
Abstract:
In this paper we study optimal stop** problems for nonlinear Markov processes driven by a McKean-Vlasov SDE and aim at solving them numerically by Monte Carlo. To this end we propose a novel regression algorithm based on the corresponding particle system and prove its convergence. The proof of convergence is based on perturbation analysis of a related linear regression problem. The performance o…
▽ More
In this paper we study optimal stop** problems for nonlinear Markov processes driven by a McKean-Vlasov SDE and aim at solving them numerically by Monte Carlo. To this end we propose a novel regression algorithm based on the corresponding particle system and prove its convergence. The proof of convergence is based on perturbation analysis of a related linear regression problem. The performance of the proposed algorithms is illustrated by a numerical example.
△ Less
Submitted 25 June, 2018;
originally announced June 2018.
-
Nonparametric Bayesian inference for Gamma-type Lévy subordinators
Authors:
Denis Belomestny,
Shota Gugushvili,
Moritz Schauer,
Peter Spreij
Abstract:
Given discrete time observations over a growing time interval, we consider a nonparametric Bayesian approach to estimation of the Lévy density of a Lévy process belonging to a flexible class of infinite activity subordinators. Posterior inference is performed via MCMC, and we circumvent the problem of the intractable likelihood via the data augmentation device, that in our case relies on bridge pr…
▽ More
Given discrete time observations over a growing time interval, we consider a nonparametric Bayesian approach to estimation of the Lévy density of a Lévy process belonging to a flexible class of infinite activity subordinators. Posterior inference is performed via MCMC, and we circumvent the problem of the intractable likelihood via the data augmentation device, that in our case relies on bridge process sampling via Gamma process bridges. Our approach also requires the use of a new infinite-dimensional form of a reversible jump MCMC algorithm. We show that our method leads to good practical results in challenging simulation examples. On the theoretical side, we establish that our nonparametric Bayesian procedure is consistent: in the low frequency data setting, with equispaced in time observations and intervals between successive observations remaining fixed, the posterior asymptotically, as the sample size $n\rightarrow\infty$, concentrates around the Lévy density under which the data have been generated. Finally, we test our method on a classical insurance dataset.
△ Less
Submitted 30 January, 2019; v1 submitted 30 April, 2018;
originally announced April 2018.
-
Solving linear parabolic rough partial differential equations
Authors:
Christian Bayer,
Denis Belomestny,
Martin Redmann,
Sebastian Riedel,
John Schoenmakers
Abstract:
We study linear rough partial differential equations in the setting of [Friz and Hairer, Springer, 2014, Chapter 12]. More precisely, we consider a linear parabolic partial differential equation driven by a deterministic rough path $\mathbf{W}$ of Hölder regularity $α$ with $1/3 < α
\le 1/2$. Based on a stochastic representation of the solution of the rough partial differential equation, we prop…
▽ More
We study linear rough partial differential equations in the setting of [Friz and Hairer, Springer, 2014, Chapter 12]. More precisely, we consider a linear parabolic partial differential equation driven by a deterministic rough path $\mathbf{W}$ of Hölder regularity $α$ with $1/3 < α
\le 1/2$. Based on a stochastic representation of the solution of the rough partial differential equation, we propose a regression Monte Carlo algorithm for spatio-temporal approximation of the solution. We provide a full convergence analysis of the proposed approximation method which essentially relies on the new bounds for the higher order derivatives of the solution in space. Finally, a comprehensive simulation study showing the applicability of the proposed algorithm is presented.
△ Less
Submitted 26 March, 2018;
originally announced March 2018.
-
Empirical Variance Minimization with Applications in Variance Reduction and Optimal Control
Authors:
D. Belomestny,
L. Iosipoi,
Q. Paris,
N. Zhivotovskiy
Abstract:
We study the problem of empirical minimization for variance-type functionals over functional classes. Sharp non-asymptotic bounds for the excess variance are derived under mild conditions. In particular, it is shown that under some restrictions imposed on the functional class fast convergence rates can be achieved including the optimal non-parametric rates for expressive classes in the non-Donsker…
▽ More
We study the problem of empirical minimization for variance-type functionals over functional classes. Sharp non-asymptotic bounds for the excess variance are derived under mild conditions. In particular, it is shown that under some restrictions imposed on the functional class fast convergence rates can be achieved including the optimal non-parametric rates for expressive classes in the non-Donsker regime under some additional assumptions. Our main applications include variance reduction and optimal control.
△ Less
Submitted 31 July, 2021; v1 submitted 13 December, 2017;
originally announced December 2017.
-
Sparse covariance matrix estimation in high-dimensional deconvolution
Authors:
Denis Belomestny,
Mathias Trabs,
Alexandre B. Tsybakov
Abstract:
We study the estimation of the covariance matrix $Σ$ of a $p$-dimensional normal random vector based on $n$ independent observations corrupted by additive noise. Only a general nonparametric assumption is imposed on the distribution of the noise without any sparsity constraint on its covariance matrix. In this high-dimensional semiparametric deconvolution problem, we propose spectral thresholding…
▽ More
We study the estimation of the covariance matrix $Σ$ of a $p$-dimensional normal random vector based on $n$ independent observations corrupted by additive noise. Only a general nonparametric assumption is imposed on the distribution of the noise without any sparsity constraint on its covariance matrix. In this high-dimensional semiparametric deconvolution problem, we propose spectral thresholding estimators that are adaptive to the sparsity of $Σ$. We establish an oracle inequality for these estimators under model miss-specification and derive non-asymptotic minimax convergence rates that are shown to be logarithmic in $n/\log p$. We also discuss the estimation of low-rank matrices based on indirect observations as well as the generalization to elliptical distributions. The finite sample performance of the threshold estimators is illustrated in a numerical example.
△ Less
Submitted 26 March, 2018; v1 submitted 30 October, 2017;
originally announced October 2017.
-
Nonparametric density estimation from observations with multiplicative measurement errors
Authors:
Denis Belomestny,
Alexander Goldenshluger
Abstract:
In this paper we study the problem of pointwise density estimation from observations with multiplicative measurement errors. We elucidate the main feature of this problem: the influence of the estimation point on the estimation accuracy. In particular, we show that, depending on whether this point is separated away from zero or not, there are two different regimes in terms of the rates of converge…
▽ More
In this paper we study the problem of pointwise density estimation from observations with multiplicative measurement errors. We elucidate the main feature of this problem: the influence of the estimation point on the estimation accuracy. In particular, we show that, depending on whether this point is separated away from zero or not, there are two different regimes in terms of the rates of convergence of the minimax risk. In both regimes we develop kernel--type density estimators and prove upper bounds on their maximal risk over suitable nonparametric classes of densities. We show that the proposed estimators are rate--optimal by establishing matching lower bounds on the minimax risk. Finally we test our estimation procedures on simulated data.
△ Less
Submitted 12 July, 2018; v1 submitted 2 September, 2017;
originally announced September 2017.
-
Minimax theorems for American options in incomplete markets without time-consistency
Authors:
Denis Belomestny,
Volker Kraetschmer
Abstract:
In this paper we give sufficient conditions guaranteeing the validity of the well-known minimax theorem for the lower Snell envelope with respect to a family of absolutely continuous probability measures. Such minimax results play an important role in the characterisation of arbitrage-free prices of American contingent claims in incomplete markets. Our conditions do not rely on the notions of stab…
▽ More
In this paper we give sufficient conditions guaranteeing the validity of the well-known minimax theorem for the lower Snell envelope with respect to a family of absolutely continuous probability measures. Such minimax results play an important role in the characterisation of arbitrage-free prices of American contingent claims in incomplete markets. Our conditions do not rely on the notions of stability under pasting or time-consistency and reveal some unexpected connection between the minimax result and the path properties of the corresponding density process.
△ Less
Submitted 29 August, 2017;
originally announced August 2017.
-
Projected particle methods for solving McKean-Vlasov stochastic differential equations
Authors:
Denis Belomestny,
John Schoenmakers
Abstract:
We propose a novel projection-based particle method for solving the McKean-Vlasov stochastic differential equations. Our approach is based on a projection-type estimation of the marginal density of the solution in each time step. The projection-based particle method leads in many situation to a significant reduction of numerical complexity compared to the widely used kernel density estimation algo…
▽ More
We propose a novel projection-based particle method for solving the McKean-Vlasov stochastic differential equations. Our approach is based on a projection-type estimation of the marginal density of the solution in each time step. The projection-based particle method leads in many situation to a significant reduction of numerical complexity compared to the widely used kernel density estimation algorithms. We derive strong convergence rates and rates of density estimation. The convergence analysis in the case of linearly growing coefficients turns out to be rather challenging and requires some new type of averaging technique. This case is exemplified by explicit solutions to a class of McKean-Vlasov equations with affine drift. The performance of the proposed algorithm is illustrated by several numerical examples.
△ Less
Submitted 4 August, 2018; v1 submitted 27 August, 2017;
originally announced August 2017.
-
Semiparametric estimation in the normal variance-mean mixture model
Authors:
Denis Belomestny,
Vladimir Panov
Abstract:
In this paper we study the problem of statistical inference on the parameters of the semiparametric variance-mean mixtures. This class of mixtures has recently become rather popular in statistical and financial modelling. We design a semiparametric estimation procedure that first estimates the mean of the underlying normal distribution and then recovers nonparametrically the density of the corresp…
▽ More
In this paper we study the problem of statistical inference on the parameters of the semiparametric variance-mean mixtures. This class of mixtures has recently become rather popular in statistical and financial modelling. We design a semiparametric estimation procedure that first estimates the mean of the underlying normal distribution and then recovers nonparametrically the density of the corresponding mixing distribution. We illustrate the performance of our procedure on simulated and real data.
△ Less
Submitted 22 May, 2017;
originally announced May 2017.
-
Statistical inference for moving-average Lévy-driven processes: Fourier-based approach
Authors:
Denis Belomestny,
Tatiana Orlova,
Vladimir Panov
Abstract:
We consider a new method of the semiparametric statistical estimation for the continuous-time moving average Lévy processes. We derive the convergence rates of the proposed estimators, and show that these rates are optimal in the minimax sense.
We consider a new method of the semiparametric statistical estimation for the continuous-time moving average Lévy processes. We derive the convergence rates of the proposed estimators, and show that these rates are optimal in the minimax sense.
△ Less
Submitted 9 February, 2017;
originally announced February 2017.
-
Truncated control variates for weak approximation schemes
Authors:
Denis Belomestny,
Stefan Häfner,
Mikhail Urusov
Abstract:
In this paper we present an enhancement of the regression-based variance reduction approaches recently proposed in Belomestny et al. This enhancement is based on a truncation of the control variate and allows for a significant reduction of the computing time, while the complexity stays of the same order. The performances of the proposed truncated algorithms are illustrated by a numerical example.
In this paper we present an enhancement of the regression-based variance reduction approaches recently proposed in Belomestny et al. This enhancement is based on a truncation of the control variate and allows for a significant reduction of the computing time, while the complexity stays of the same order. The performances of the proposed truncated algorithms are illustrated by a numerical example.
△ Less
Submitted 22 May, 2017; v1 submitted 1 January, 2017;
originally announced January 2017.
-
Stratified regression-based variance reduction approach for weak approximation schemes
Authors:
Denis Belomestny,
Stefan Häfner,
Mikhail Urusov
Abstract:
In this paper we suggest a modification of the regression-based variance reduction approach recently proposed in Belomestny et al. This modification is based on the stratification technique and allows for a further significant variance reduction. The performance of the proposed approach is illustrated by several numerical examples.
In this paper we suggest a modification of the regression-based variance reduction approach recently proposed in Belomestny et al. This modification is based on the stratification technique and allows for a further significant variance reduction. The performance of the proposed approach is illustrated by several numerical examples.
△ Less
Submitted 1 March, 2017; v1 submitted 16 December, 2016;
originally announced December 2016.
-
Regression-based variance reduction approach for strong approximation schemes
Authors:
Denis Belomestny,
Stefan Häfner,
Mikhail Urusov
Abstract:
In this paper we present a novel approach towards variance reduction for discretised diffusion processes. The proposed approach involves specially constructed control variates and allows for a significant reduction in the variance for the terminal functionals. In this way the complexity order of the standard Monte Carlo algorithm ($\varepsilon^{-3}$) can be reduced down to…
▽ More
In this paper we present a novel approach towards variance reduction for discretised diffusion processes. The proposed approach involves specially constructed control variates and allows for a significant reduction in the variance for the terminal functionals. In this way the complexity order of the standard Monte Carlo algorithm ($\varepsilon^{-3}$) can be reduced down to $\varepsilon^{-2}\sqrt{\left|\log(\varepsilon)\right|}$ in case of the Euler scheme with $\varepsilon$ being the precision to be achieved. These theoretical results are illustrated by several numerical examples.
△ Less
Submitted 1 March, 2017; v1 submitted 11 December, 2016;
originally announced December 2016.
-
Regression-based complexity reduction of the nested Monte Carlo methods
Authors:
Denis Belomestny,
Stefan Häfner,
Mikhail Urusov
Abstract:
In this paper we propose a novel dual regression-based approach for pricing American options. This approach reduces the complexity of the nested Monte Carlo method and has especially simple form for time discretised diffusion processes. We analyse the complexity of the proposed approach both in the case of fixed and increasing number of exercise dates. The method is illustrated by several numerica…
▽ More
In this paper we propose a novel dual regression-based approach for pricing American options. This approach reduces the complexity of the nested Monte Carlo method and has especially simple form for time discretised diffusion processes. We analyse the complexity of the proposed approach both in the case of fixed and increasing number of exercise dates. The method is illustrated by several numerical examples.
△ Less
Submitted 6 June, 2018; v1 submitted 19 November, 2016;
originally announced November 2016.