-
Decompounding Under General Mixing Distributions
Authors:
Denis Belomestny,
Ekaterina Morozova,
Vladimir Panov
Abstract:
This study focuses on statistical inference for compound models of the form $X=ξ_1+\ldots+ξ_N$, where $N$ is a random variable denoting the count of summands, which are independent and identically distributed (i.i.d.) random variables $ξ_1, ξ_2, \ldots$. The paper addresses the problem of reconstructing the distribution of $ξ$ from observed samples of $X$'s distribution, a process referred to as d…
▽ More
This study focuses on statistical inference for compound models of the form $X=ξ_1+\ldots+ξ_N$, where $N$ is a random variable denoting the count of summands, which are independent and identically distributed (i.i.d.) random variables $ξ_1, ξ_2, \ldots$. The paper addresses the problem of reconstructing the distribution of $ξ$ from observed samples of $X$'s distribution, a process referred to as decompounding, with the assumption that $N$'s distribution is known. This work diverges from the conventional scope by not limiting $N$'s distribution to the Poisson type, thus embracing a broader context. We propose a nonparametric estimate for the density of $ξ$, derive its rates of convergence and prove that these rates are minimax optimal for suitable classes of distributions for $ξ$ and $N$. Finally, we illustrate the numerical performance of the algorithm on simulated examples.
△ Less
Submitted 8 May, 2024;
originally announced May 2024.
-
Model-free Posterior Sampling via Learning Rate Randomization
Authors:
Daniil Tiapkin,
Denis Belomestny,
Daniele Calandriello,
Eric Moulines,
Remi Munos,
Alexey Naumov,
Pierre Perrault,
Michal Valko,
Pierre Menard
Abstract:
In this paper, we introduce Randomized Q-learning (RandQL), a novel randomized model-free algorithm for regret minimization in episodic Markov Decision Processes (MDPs). To the best of our knowledge, RandQL is the first tractable model-free posterior sampling-based algorithm. We analyze the performance of RandQL in both tabular and non-tabular metric space settings. In tabular MDPs, RandQL achieve…
▽ More
In this paper, we introduce Randomized Q-learning (RandQL), a novel randomized model-free algorithm for regret minimization in episodic Markov Decision Processes (MDPs). To the best of our knowledge, RandQL is the first tractable model-free posterior sampling-based algorithm. We analyze the performance of RandQL in both tabular and non-tabular metric space settings. In tabular MDPs, RandQL achieves a regret bound of order $\widetilde{\mathcal{O}}(\sqrt{H^{5}SAT})$, where $H$ is the planning horizon, $S$ is the number of states, $A$ is the number of actions, and $T$ is the number of episodes. For a metric state-action space, RandQL enjoys a regret bound of order $\widetilde{\mathcal{O}}(H^{5/2} T^{(d_z+1)/(d_z+2)})$, where $d_z$ denotes the zooming dimension. Notably, RandQL achieves optimistic exploration without using bonuses, relying instead on a novel idea of learning rate randomization. Our empirical study shows that RandQL outperforms existing approaches on baseline exploration environments.
△ Less
Submitted 27 October, 2023;
originally announced October 2023.
-
Demonstration-Regularized RL
Authors:
Daniil Tiapkin,
Denis Belomestny,
Daniele Calandriello,
Eric Moulines,
Alexey Naumov,
Pierre Perrault,
Michal Valko,
Pierre Menard
Abstract:
Incorporating expert demonstrations has empirically helped to improve the sample efficiency of reinforcement learning (RL). This paper quantifies theoretically to what extent this extra information reduces RL's sample complexity. In particular, we study the demonstration-regularized reinforcement learning that leverages the expert demonstrations by KL-regularization for a policy learned by behavio…
▽ More
Incorporating expert demonstrations has empirically helped to improve the sample efficiency of reinforcement learning (RL). This paper quantifies theoretically to what extent this extra information reduces RL's sample complexity. In particular, we study the demonstration-regularized reinforcement learning that leverages the expert demonstrations by KL-regularization for a policy learned by behavior cloning. Our findings reveal that using $N^{\mathrm{E}}$ expert demonstrations enables the identification of an optimal policy at a sample complexity of order $\widetilde{O}(\mathrm{Poly}(S,A,H)/(\varepsilon^2 N^{\mathrm{E}}))$ in finite and $\widetilde{O}(\mathrm{Poly}(d,H)/(\varepsilon^2 N^{\mathrm{E}}))$ in linear Markov decision processes, where $\varepsilon$ is the target precision, $H$ the horizon, $A$ the number of action, $S$ the number of states in the finite case and $d$ the dimension of the feature space in the linear case. As a by-product, we provide tight convergence guarantees for the behaviour cloning procedure under general assumptions on the policy classes. Additionally, we establish that demonstration-regularized methods are provably efficient for reinforcement learning from human feedback (RLHF). In this respect, we provide theoretical evidence showing the benefits of KL-regularization for RLHF in tabular and linear MDPs. Interestingly, we avoid pessimism injection by employing computationally feasible regularization to handle reward estimation uncertainty, thus setting our approach apart from the prior works.
△ Less
Submitted 10 June, 2024; v1 submitted 26 October, 2023;
originally announced October 2023.
-
Sharp Deviations Bounds for Dirichlet Weighted Sums with Application to analysis of Bayesian algorithms
Authors:
Denis Belomestny,
Pierre Menard,
Alexey Naumov,
Daniil Tiapkin,
Michal Valko
Abstract:
In this work, we derive sharp non-asymptotic deviation bounds for weighted sums of Dirichlet random variables. These bounds are based on a novel integral representation of the density of a weighted Dirichlet sum. This representation allows us to obtain a Gaussian-like approximation for the sum distribution using geometry and complex analysis methods. Our results generalize similar bounds for the B…
▽ More
In this work, we derive sharp non-asymptotic deviation bounds for weighted sums of Dirichlet random variables. These bounds are based on a novel integral representation of the density of a weighted Dirichlet sum. This representation allows us to obtain a Gaussian-like approximation for the sum distribution using geometry and complex analysis methods. Our results generalize similar bounds for the Beta distribution obtained in the seminal paper Alfers and Dinges [1984]. Additionally, our results can be considered a sharp non-asymptotic version of the inverse of Sanov's theorem studied by Ganesh and O'Connell [1999] in the Bayesian setting. Based on these results, we derive new deviation bounds for the Dirichlet process posterior means with application to Bayesian bootstrap. Finally, we apply our estimates to the analysis of the Multinomial Thompson Sampling (TS) algorithm in multi-armed bandits and significantly sharpen the existing regret bounds by making them independent of the size of the arms distribution support.
△ Less
Submitted 6 April, 2023;
originally announced April 2023.
-
Theoretical guarantees for neural control variates in MCMC
Authors:
Denis Belomestny,
Artur Goldman,
Alexey Naumov,
Sergey Samsonov
Abstract:
In this paper, we propose a variance reduction approach for Markov chains based on additive control variates and the minimization of an appropriate estimate for the asymptotic variance. We focus on the particular case when control variates are represented as deep neural networks. We derive the optimal convergence rate of the asymptotic variance under various ergodicity assumptions on the underlyin…
▽ More
In this paper, we propose a variance reduction approach for Markov chains based on additive control variates and the minimization of an appropriate estimate for the asymptotic variance. We focus on the particular case when control variates are represented as deep neural networks. We derive the optimal convergence rate of the asymptotic variance under various ergodicity assumptions on the underlying Markov chain. The proposed approach relies upon recent results on the stochastic errors of variance reduction algorithms and function approximation theory.
△ Less
Submitted 3 April, 2023;
originally announced April 2023.
-
Fast Rates for Maximum Entropy Exploration
Authors:
Daniil Tiapkin,
Denis Belomestny,
Daniele Calandriello,
Eric Moulines,
Remi Munos,
Alexey Naumov,
Pierre Perrault,
Yunhao Tang,
Michal Valko,
Pierre Menard
Abstract:
We address the challenge of exploration in reinforcement learning (RL) when the agent operates in an unknown environment with sparse or no rewards. In this work, we study the maximum entropy exploration problem of two different types. The first type is visitation entropy maximization previously considered by Hazan et al.(2019) in the discounted setting. For this type of exploration, we propose a g…
▽ More
We address the challenge of exploration in reinforcement learning (RL) when the agent operates in an unknown environment with sparse or no rewards. In this work, we study the maximum entropy exploration problem of two different types. The first type is visitation entropy maximization previously considered by Hazan et al.(2019) in the discounted setting. For this type of exploration, we propose a game-theoretic algorithm that has $\widetilde{\mathcal{O}}(H^3S^2A/\varepsilon^2)$ sample complexity thus improving the $\varepsilon$-dependence upon existing results, where $S$ is a number of states, $A$ is a number of actions, $H$ is an episode length, and $\varepsilon$ is a desired accuracy. The second type of entropy we study is the trajectory entropy. This objective function is closely related to the entropy-regularized MDPs, and we propose a simple algorithm that has a sample complexity of order $\widetilde{\mathcal{O}}(\mathrm{poly}(S,A,H)/\varepsilon)$. Interestingly, it is the first theoretical result in RL literature that establishes the potential statistical advantage of regularized MDPs for exploration. Finally, we apply developed regularization techniques to reduce sample complexity of visitation entropy maximization to $\widetilde{\mathcal{O}}(H^2SA/\varepsilon^2)$, yielding a statistical separation between maximum entropy exploration and reward-free exploration.
△ Less
Submitted 6 June, 2023; v1 submitted 14 March, 2023;
originally announced March 2023.
-
Spectral bootstrap confidence bands for Lévy-driven moving average processes
Authors:
D. Belomestny,
E. Ivanova,
T. Orlova
Abstract:
In this paper we study the problem of constructing bootstrap confidence intervals for the Lévy density of the driving Lévy process based on high-frequency observations of a Lévy-driven moving average processes. Using a spectral estimator of the Lévy density, we propose a novel implementations of multiplier and empirical bootstraps to construct confidence bands on a compact set away from the origin…
▽ More
In this paper we study the problem of constructing bootstrap confidence intervals for the Lévy density of the driving Lévy process based on high-frequency observations of a Lévy-driven moving average processes. Using a spectral estimator of the Lévy density, we propose a novel implementations of multiplier and empirical bootstraps to construct confidence bands on a compact set away from the origin. We also provide conditions under which the confidence bands are asymptotically valid.
△ Less
Submitted 12 November, 2022;
originally announced November 2022.
-
Statistical Inference for Scale Mixture Models via Mellin Transform Approach
Authors:
Denis Belomestny,
Ekaterina Morozova,
Vladimir Panov
Abstract:
This paper deals with statistical inference for the scale mixture models. We study an estimation approach based on the Mellin -- Stieltjes transform that can be applied to both discrete and absolute continuous mixing distributions. The accuracy of the corresponding estimate is analysed in terms of its expected pointwise error. As an important technical result, we prove the analogue of the Berry --…
▽ More
This paper deals with statistical inference for the scale mixture models. We study an estimation approach based on the Mellin -- Stieltjes transform that can be applied to both discrete and absolute continuous mixing distributions. The accuracy of the corresponding estimate is analysed in terms of its expected pointwise error. As an important technical result, we prove the analogue of the Berry -- Esseen inequality for the Mellin transforms. The proposed statistical approach is illustrated by numerical examples.
△ Less
Submitted 22 January, 2024; v1 submitted 3 November, 2022;
originally announced November 2022.
-
Primal-dual regression approach for Markov decision processes with general state and action space
Authors:
Denis Belomestny,
John Schoenmakers
Abstract:
We develop a regression based primal-dual martingale approach for solving finite time horizon MDPs with general state and action space. As a result, our method allows for the construction of tight upper and lower biased approximations of the value functions, and, provides tight approximations to the optimal policy. In particular, we prove tight error bounds for the estimated duality gap featuring…
▽ More
We develop a regression based primal-dual martingale approach for solving finite time horizon MDPs with general state and action space. As a result, our method allows for the construction of tight upper and lower biased approximations of the value functions, and, provides tight approximations to the optimal policy. In particular, we prove tight error bounds for the estimated duality gap featuring polynomial dependence on the time horizon, and sublinear dependence on the cardinality/dimension of the possibly infinite state and action space.From a computational point of view the proposed method is efficient since, in contrast to usual duality-based methods for optimal control problems in the literature, the Monte Carlo procedures here involved do not require nested simulations.
△ Less
Submitted 4 October, 2022; v1 submitted 1 October, 2022;
originally announced October 2022.
-
Optimistic Posterior Sampling for Reinforcement Learning with Few Samples and Tight Guarantees
Authors:
Daniil Tiapkin,
Denis Belomestny,
Daniele Calandriello,
Eric Moulines,
Remi Munos,
Alexey Naumov,
Mark Rowland,
Michal Valko,
Pierre Menard
Abstract:
We consider reinforcement learning in an environment modeled by an episodic, finite, stage-dependent Markov decision process of horizon $H$ with $S$ states, and $A$ actions. The performance of an agent is measured by the regret after interacting with the environment for $T$ episodes. We propose an optimistic posterior sampling algorithm for reinforcement learning (OPSRL), a simple variant of poste…
▽ More
We consider reinforcement learning in an environment modeled by an episodic, finite, stage-dependent Markov decision process of horizon $H$ with $S$ states, and $A$ actions. The performance of an agent is measured by the regret after interacting with the environment for $T$ episodes. We propose an optimistic posterior sampling algorithm for reinforcement learning (OPSRL), a simple variant of posterior sampling that only needs a number of posterior samples logarithmic in $H$, $S$, $A$, and $T$ per state-action pair. For OPSRL we guarantee a high-probability regret bound of order at most $\widetilde{\mathcal{O}}(\sqrt{H^3SAT})$ ignoring $\text{poly}\log(HSAT)$ terms. The key novel technical ingredient is a new sharp anti-concentration inequality for linear forms which may be of independent interest. Specifically, we extend the normal approximation-based lower bound for Beta distributions by Alfers and Dinges [1984] to Dirichlet distributions. Our bound matches the lower bound of order $Ω(\sqrt{H^3SAT})$, thereby answering the open problems raised by Agrawal and Jia [2017b] for the episodic setting.
△ Less
Submitted 28 September, 2022;
originally announced September 2022.
-
Simultaneous approximation of a smooth function and its derivatives by deep neural networks with piecewise-polynomial activations
Authors:
Denis Belomestny,
Alexey Naumov,
Nikita Puchkin,
Sergey Samsonov
Abstract:
This paper investigates the approximation properties of deep neural networks with piecewise-polynomial activation functions. We derive the required depth, width, and sparsity of a deep neural network to approximate any Hölder smooth function up to a given approximation error in Hölder norms in such a way that all weights of this neural network are bounded by $1$. The latter feature is essential to…
▽ More
This paper investigates the approximation properties of deep neural networks with piecewise-polynomial activation functions. We derive the required depth, width, and sparsity of a deep neural network to approximate any Hölder smooth function up to a given approximation error in Hölder norms in such a way that all weights of this neural network are bounded by $1$. The latter feature is essential to control generalization errors in many statistical and machine learning applications.
△ Less
Submitted 2 December, 2022; v1 submitted 19 June, 2022;
originally announced June 2022.
-
From Dirichlet to Rubin: Optimistic Exploration in RL without Bonuses
Authors:
Daniil Tiapkin,
Denis Belomestny,
Eric Moulines,
Alexey Naumov,
Sergey Samsonov,
Yunhao Tang,
Michal Valko,
Pierre Menard
Abstract:
We propose the Bayes-UCBVI algorithm for reinforcement learning in tabular, stage-dependent, episodic Markov decision process: a natural extension of the Bayes-UCB algorithm by Kaufmann et al. (2012) for multi-armed bandits. Our method uses the quantile of a Q-value function posterior as upper confidence bound on the optimal Q-value function. For Bayes-UCBVI, we prove a regret bound of order…
▽ More
We propose the Bayes-UCBVI algorithm for reinforcement learning in tabular, stage-dependent, episodic Markov decision process: a natural extension of the Bayes-UCB algorithm by Kaufmann et al. (2012) for multi-armed bandits. Our method uses the quantile of a Q-value function posterior as upper confidence bound on the optimal Q-value function. For Bayes-UCBVI, we prove a regret bound of order $\widetilde{O}(\sqrt{H^3SAT})$ where $H$ is the length of one episode, $S$ is the number of states, $A$ the number of actions, $T$ the number of episodes, that matches the lower-bound of $Ω(\sqrt{H^3SAT})$ up to poly-$\log$ terms in $H,S,A,T$ for a large enough $T$. To the best of our knowledge, this is the first algorithm that obtains an optimal dependence on the horizon $H$ (and $S$) without the need for an involved Bernstein-like bonus or noise. Crucial to our analysis is a new fine-grained anti-concentration bound for a weighted Dirichlet sum that can be of independent interest. We then explain how Bayes-UCBVI can be easily extended beyond the tabular setting, exhibiting a strong link between our algorithm and Bayesian bootstrap (Rubin, 1981).
△ Less
Submitted 22 June, 2022; v1 submitted 16 May, 2022;
originally announced May 2022.
-
Rates of convergence for density estimation with generative adversarial networks
Authors:
Nikita Puchkin,
Sergey Samsonov,
Denis Belomestny,
Eric Moulines,
Alexey Naumov
Abstract:
In this work we undertake a thorough study of the non-asymptotic properties of the vanilla generative adversarial networks (GANs). We prove an oracle inequality for the Jensen-Shannon (JS) divergence between the underlying density $\mathsf{p}^*$ and the GAN estimate with a significantly better statistical error term compared to the previously known results. The advantage of our bound becomes clear…
▽ More
In this work we undertake a thorough study of the non-asymptotic properties of the vanilla generative adversarial networks (GANs). We prove an oracle inequality for the Jensen-Shannon (JS) divergence between the underlying density $\mathsf{p}^*$ and the GAN estimate with a significantly better statistical error term compared to the previously known results. The advantage of our bound becomes clear in application to nonparametric density estimation. We show that the JS-divergence between the GAN estimate and $\mathsf{p}^*$ decays as fast as $(\log{n}/n)^{2β/(2β+ d)}$, where $n$ is the sample size and $β$ determines the smoothness of $\mathsf{p}^*$. This rate of convergence coincides (up to logarithmic factors) with minimax optimal for the considered class of densities.
△ Less
Submitted 25 January, 2024; v1 submitted 30 January, 2021;
originally announced February 2021.
-
Reinforced optimal control
Authors:
Christian Bayer,
Denis Belomestny,
Paul Hager,
Paolo Pigato,
John Schoenmakers,
Vladimir Spokoiny
Abstract:
Least squares Monte Carlo methods are a popular numerical approximation method for solving stochastic control problems. Based on dynamic programming, their key feature is the approximation of the conditional expectation of future rewards by linear least squares regression. Hence, the choice of basis functions is crucial for the accuracy of the method. Earlier work by some of us [Belomestny, Schoen…
▽ More
Least squares Monte Carlo methods are a popular numerical approximation method for solving stochastic control problems. Based on dynamic programming, their key feature is the approximation of the conditional expectation of future rewards by linear least squares regression. Hence, the choice of basis functions is crucial for the accuracy of the method. Earlier work by some of us [Belomestny, Schoenmakers, Spokoiny, Zharkynbay. Commun.~Math.~Sci., 18(1):109-121, 2020](ar** problems by already computed value functions for later times, thereby considerably improving the accuracy with limited additional computational cost. We extend the reinforced regression method to a general class of stochastic control problems, while considerably improving the method's efficiency, as demonstrated by substantial numerical examples as well as theoretical analysis.
△ Less
Submitted 25 March, 2022; v1 submitted 24 November, 2020;
originally announced November 2020.
-
Nonparametric Bayesian volatility estimation for gamma-driven stochastic differential equations
Authors:
Denis Belomestny,
Shota Gugushvili,
Moritz Schauer,
Peter Spreij
Abstract:
We study a nonparametric Bayesian approach to estimation of the volatility function of a stochastic differential equation driven by a gamma process. The volatility function is modelled a priori as piecewise constant, and we specify a gamma prior on its values. This leads to a straightforward procedure for posterior inference via an MCMC procedure. We give theoretical performance guarantees (contra…
▽ More
We study a nonparametric Bayesian approach to estimation of the volatility function of a stochastic differential equation driven by a gamma process. The volatility function is modelled a priori as piecewise constant, and we specify a gamma prior on its values. This leads to a straightforward procedure for posterior inference via an MCMC procedure. We give theoretical performance guarantees (contraction rates for the posterior) for the Bayesian estimate in terms of the regularity of the unknown volatility function. We illustrate the method on synthetic and real data examples.
△ Less
Submitted 26 August, 2021; v1 submitted 16 November, 2020;
originally announced November 2020.
-
Variance reduction for dependent sequences with applications to Stochastic Gradient MCMC
Authors:
D. Belomestny,
L. Iosipoi,
E. Moulines,
A. Naumov,
S. Samsonov
Abstract:
In this paper we propose a novel and practical variance reduction approach for additive functionals of dependent sequences. Our approach combines the use of control variates with the minimisation of an empirical variance estimate. We analyse finite sample properties of the proposed method and derive finite-time bounds of the excess asymptotic variance to zero. We apply our methodology to Stochasti…
▽ More
In this paper we propose a novel and practical variance reduction approach for additive functionals of dependent sequences. Our approach combines the use of control variates with the minimisation of an empirical variance estimate. We analyse finite sample properties of the proposed method and derive finite-time bounds of the excess asymptotic variance to zero. We apply our methodology to Stochastic Gradient MCMC (SGMCMC) methods for Bayesian inference on large data sets and combine it with existing variance reduction methods for SGMCMC. We present empirical results carried out on a number of benchmark examples showing that our variance reduction method achieves significant improvement as compared to state-of-the-art methods at the expense of a moderate increase of computational overhead.
△ Less
Submitted 16 August, 2020;
originally announced August 2020.
-
Estimating TVP-VAR models with time invariant long-run multipliers
Authors:
Denis Belomestny,
Ekaterina Krymova,
Andrey Polbin
Abstract:
The main goal of this paper is to develop a methodology for estimating time varying parameter vector auto-regression (TVP-VAR) models with a timeinvariant long-run relationship between endogenous variables and changes in exogenous variables. We propose a Gibbs sampling scheme for estimation of model parameters as well as time-invariant long-run multiplier parameters. Further we demonstrate the app…
▽ More
The main goal of this paper is to develop a methodology for estimating time varying parameter vector auto-regression (TVP-VAR) models with a timeinvariant long-run relationship between endogenous variables and changes in exogenous variables. We propose a Gibbs sampling scheme for estimation of model parameters as well as time-invariant long-run multiplier parameters. Further we demonstrate the applicability of the proposed method by analyzing examples of the Norwegian and Russian economies based on the data on real GDP, real exchange rate and real oil prices. Our results show that incorporating the time invariance constraint on the long-run multipliers in TVP-VAR model helps to significantly improve the forecasting performance.
△ Less
Submitted 3 August, 2020;
originally announced August 2020.
-
Variance reduction for Markov chains with application to MCMC
Authors:
D. Belomestny,
L. Iosipoi,
E. Moulines,
A. Naumov,
S. Samsonov
Abstract:
In this paper we propose a novel variance reduction approach for additive functionals of Markov chains based on minimization of an estimate for the asymptotic variance of these functionals over suitable classes of control variates. A distinctive feature of the proposed approach is its ability to significantly reduce the overall finite sample variance. This feature is theoretically demonstrated by…
▽ More
In this paper we propose a novel variance reduction approach for additive functionals of Markov chains based on minimization of an estimate for the asymptotic variance of these functionals over suitable classes of control variates. A distinctive feature of the proposed approach is its ability to significantly reduce the overall finite sample variance. This feature is theoretically demonstrated by means of a deep non asymptotic analysis of a variance reduced functional as well as by a thorough simulation study. In particular we apply our method to various MCMC Bayesian estimation problems where it favourably compares to the existing variance reduction approaches.
△ Less
Submitted 15 February, 2020; v1 submitted 8 October, 2019;
originally announced October 2019.
-
Fourier transform MCMC, heavy tailed distributions and geometric ergodicity
Authors:
Denis Belomestny,
Leonid Iosipoi
Abstract:
Markov Chain Monte Carlo methods become increasingly popular in applied mathematics as a tool for numerical integration with respect to complex and high-dimensional distributions. However, application of MCMC methods to heavy tailed distributions and distributions with analytically intractable densities turns out to be rather problematic. In this paper, we propose a novel approach towards the use…
▽ More
Markov Chain Monte Carlo methods become increasingly popular in applied mathematics as a tool for numerical integration with respect to complex and high-dimensional distributions. However, application of MCMC methods to heavy tailed distributions and distributions with analytically intractable densities turns out to be rather problematic. In this paper, we propose a novel approach towards the use of MCMC algorithms for distributions with analytically known Fourier transforms and, in particular, heavy tailed distributions. The main idea of the proposed approach is to use MCMC methods in Fourier domain to sample from a density proportional to the absolute value of the underlying characteristic function. A subsequent application of the Parseval's formula leads to an efficient algorithm for the computation of integrals with respect to the underlying density. We show that the resulting Markov chain in Fourier domain may be geometrically ergodic even in the case of heavy tailed original distributions. We illustrate our approach by several numerical examples including multivariate elliptically contoured stable distributions.
△ Less
Submitted 31 December, 2019; v1 submitted 2 September, 2019;
originally announced September 2019.
-
Density deconvolution under general assumptions on the distribution of measurement errors
Authors:
Denis Belomestny,
Alexander Goldenshluger
Abstract:
In this paper we study the problem of density deconvolution under general assumptions on the measurement error distribution. Typically deconvolution estimators are constructed using Fourier transform techniques, and it is assumed that the characteristic function of the measurement errors does not have zeros on the real line. This assumption is rather strong and is not fulfilled in many cases of in…
▽ More
In this paper we study the problem of density deconvolution under general assumptions on the measurement error distribution. Typically deconvolution estimators are constructed using Fourier transform techniques, and it is assumed that the characteristic function of the measurement errors does not have zeros on the real line. This assumption is rather strong and is not fulfilled in many cases of interest. In this paper we develop a methodology for constructing optimal density deconvolution estimators in the general setting that covers vanishing and non--vanishing characteristic functions of the measurement errors. We derive upper bounds on the risk of the proposed estimators and provide sufficient conditions under which zeros of the corresponding characteristic function have no effect on estimation accuracy. Moreover, we show that the derived conditions are also necessary in some specific problem instances.
△ Less
Submitted 1 February, 2020; v1 submitted 25 July, 2019;
originally announced July 2019.
-
Variance reduction for additive functional of Markov chains via martingale representations
Authors:
D. Belomestny,
E. Moulines,
S. Samsonov
Abstract:
In this paper we propose an efficient variance reduction approach for additive functionals of Markov chains relying on a novel discrete time martingale representation. Our approach is fully non-asymptotic and does not require the knowledge of the stationary distribution (and even any type of ergodicity) or specific structure of the underlying density. By rigorously analyzing the convergence proper…
▽ More
In this paper we propose an efficient variance reduction approach for additive functionals of Markov chains relying on a novel discrete time martingale representation. Our approach is fully non-asymptotic and does not require the knowledge of the stationary distribution (and even any type of ergodicity) or specific structure of the underlying density. By rigorously analyzing the convergence properties of the proposed algorithm, we show that its cost-to-variance product is indeed smaller than one of the naive algorithm. The numerical performance of the new method is illustrated for the Langevin-type Markov Chain Monte Carlo (MCMC) methods.
△ Less
Submitted 21 December, 2021; v1 submitted 18 March, 2019;
originally announced March 2019.
-
Sparse constrained projection approximation subspace tracking
Authors:
Denis Belomestny,
Ekaterina Krymova
Abstract:
In this paper we revisit the well-known constrained projection approximation subspace tracking algorithm (CPAST) and derive, for the first time, non-asymptotic error bounds. Furthermore, we introduce a novel sparse modification of CPAST which is able to exploit sparsity in the underlying covariance structure. We present a non-asymptotic analysis of the proposed algorithm and study its empirical pe…
▽ More
In this paper we revisit the well-known constrained projection approximation subspace tracking algorithm (CPAST) and derive, for the first time, non-asymptotic error bounds. Furthermore, we introduce a novel sparse modification of CPAST which is able to exploit sparsity in the underlying covariance structure. We present a non-asymptotic analysis of the proposed algorithm and study its empirical performance on simulated and real data.
△ Less
Submitted 23 November, 2018; v1 submitted 22 October, 2018;
originally announced October 2018.
-
Optimal stop** via reinforced regression
Authors:
Denis Belomestny,
John Schoenmakers,
Vladimir Spokoiny,
Bakhyt Zharkynbay
Abstract:
In this note we propose a new approach towards solving numerically optimal stop** problems via reinforced regression based Monte Carlo algorithms. The main idea of the method is to reinforce standard linear regression algorithms in each backward induction step by adding new basis functions based on previously estimated continuation values. The proposed methodology is illustrated by a numerical e…
▽ More
In this note we propose a new approach towards solving numerically optimal stop** problems via reinforced regression based Monte Carlo algorithms. The main idea of the method is to reinforce standard linear regression algorithms in each backward induction step by adding new basis functions based on previously estimated continuation values. The proposed methodology is illustrated by a numerical example from mathematical finance.
△ Less
Submitted 1 July, 2019; v1 submitted 7 August, 2018;
originally announced August 2018.
-
Nonparametric Bayesian inference for Gamma-type Lévy subordinators
Authors:
Denis Belomestny,
Shota Gugushvili,
Moritz Schauer,
Peter Spreij
Abstract:
Given discrete time observations over a growing time interval, we consider a nonparametric Bayesian approach to estimation of the Lévy density of a Lévy process belonging to a flexible class of infinite activity subordinators. Posterior inference is performed via MCMC, and we circumvent the problem of the intractable likelihood via the data augmentation device, that in our case relies on bridge pr…
▽ More
Given discrete time observations over a growing time interval, we consider a nonparametric Bayesian approach to estimation of the Lévy density of a Lévy process belonging to a flexible class of infinite activity subordinators. Posterior inference is performed via MCMC, and we circumvent the problem of the intractable likelihood via the data augmentation device, that in our case relies on bridge process sampling via Gamma process bridges. Our approach also requires the use of a new infinite-dimensional form of a reversible jump MCMC algorithm. We show that our method leads to good practical results in challenging simulation examples. On the theoretical side, we establish that our nonparametric Bayesian procedure is consistent: in the low frequency data setting, with equispaced in time observations and intervals between successive observations remaining fixed, the posterior asymptotically, as the sample size $n\rightarrow\infty$, concentrates around the Lévy density under which the data have been generated. Finally, we test our method on a classical insurance dataset.
△ Less
Submitted 30 January, 2019; v1 submitted 30 April, 2018;
originally announced April 2018.
-
Empirical Variance Minimization with Applications in Variance Reduction and Optimal Control
Authors:
D. Belomestny,
L. Iosipoi,
Q. Paris,
N. Zhivotovskiy
Abstract:
We study the problem of empirical minimization for variance-type functionals over functional classes. Sharp non-asymptotic bounds for the excess variance are derived under mild conditions. In particular, it is shown that under some restrictions imposed on the functional class fast convergence rates can be achieved including the optimal non-parametric rates for expressive classes in the non-Donsker…
▽ More
We study the problem of empirical minimization for variance-type functionals over functional classes. Sharp non-asymptotic bounds for the excess variance are derived under mild conditions. In particular, it is shown that under some restrictions imposed on the functional class fast convergence rates can be achieved including the optimal non-parametric rates for expressive classes in the non-Donsker regime under some additional assumptions. Our main applications include variance reduction and optimal control.
△ Less
Submitted 31 July, 2021; v1 submitted 13 December, 2017;
originally announced December 2017.
-
Sparse covariance matrix estimation in high-dimensional deconvolution
Authors:
Denis Belomestny,
Mathias Trabs,
Alexandre B. Tsybakov
Abstract:
We study the estimation of the covariance matrix $Σ$ of a $p$-dimensional normal random vector based on $n$ independent observations corrupted by additive noise. Only a general nonparametric assumption is imposed on the distribution of the noise without any sparsity constraint on its covariance matrix. In this high-dimensional semiparametric deconvolution problem, we propose spectral thresholding…
▽ More
We study the estimation of the covariance matrix $Σ$ of a $p$-dimensional normal random vector based on $n$ independent observations corrupted by additive noise. Only a general nonparametric assumption is imposed on the distribution of the noise without any sparsity constraint on its covariance matrix. In this high-dimensional semiparametric deconvolution problem, we propose spectral thresholding estimators that are adaptive to the sparsity of $Σ$. We establish an oracle inequality for these estimators under model miss-specification and derive non-asymptotic minimax convergence rates that are shown to be logarithmic in $n/\log p$. We also discuss the estimation of low-rank matrices based on indirect observations as well as the generalization to elliptical distributions. The finite sample performance of the threshold estimators is illustrated in a numerical example.
△ Less
Submitted 26 March, 2018; v1 submitted 30 October, 2017;
originally announced October 2017.
-
Nonparametric density estimation from observations with multiplicative measurement errors
Authors:
Denis Belomestny,
Alexander Goldenshluger
Abstract:
In this paper we study the problem of pointwise density estimation from observations with multiplicative measurement errors. We elucidate the main feature of this problem: the influence of the estimation point on the estimation accuracy. In particular, we show that, depending on whether this point is separated away from zero or not, there are two different regimes in terms of the rates of converge…
▽ More
In this paper we study the problem of pointwise density estimation from observations with multiplicative measurement errors. We elucidate the main feature of this problem: the influence of the estimation point on the estimation accuracy. In particular, we show that, depending on whether this point is separated away from zero or not, there are two different regimes in terms of the rates of convergence of the minimax risk. In both regimes we develop kernel--type density estimators and prove upper bounds on their maximal risk over suitable nonparametric classes of densities. We show that the proposed estimators are rate--optimal by establishing matching lower bounds on the minimax risk. Finally we test our estimation procedures on simulated data.
△ Less
Submitted 12 July, 2018; v1 submitted 2 September, 2017;
originally announced September 2017.
-
Semiparametric estimation in the normal variance-mean mixture model
Authors:
Denis Belomestny,
Vladimir Panov
Abstract:
In this paper we study the problem of statistical inference on the parameters of the semiparametric variance-mean mixtures. This class of mixtures has recently become rather popular in statistical and financial modelling. We design a semiparametric estimation procedure that first estimates the mean of the underlying normal distribution and then recovers nonparametrically the density of the corresp…
▽ More
In this paper we study the problem of statistical inference on the parameters of the semiparametric variance-mean mixtures. This class of mixtures has recently become rather popular in statistical and financial modelling. We design a semiparametric estimation procedure that first estimates the mean of the underlying normal distribution and then recovers nonparametrically the density of the corresponding mixing distribution. We illustrate the performance of our procedure on simulated and real data.
△ Less
Submitted 22 May, 2017;
originally announced May 2017.
-
Statistical inference for moving-average Lévy-driven processes: Fourier-based approach
Authors:
Denis Belomestny,
Tatiana Orlova,
Vladimir Panov
Abstract:
We consider a new method of the semiparametric statistical estimation for the continuous-time moving average Lévy processes. We derive the convergence rates of the proposed estimators, and show that these rates are optimal in the minimax sense.
We consider a new method of the semiparametric statistical estimation for the continuous-time moving average Lévy processes. We derive the convergence rates of the proposed estimators, and show that these rates are optimal in the minimax sense.
△ Less
Submitted 9 February, 2017;
originally announced February 2017.
-
Low-rank diffusion matrix estimation for high-dimensional time-changed Lévy processes
Authors:
Denis Belomestny,
Mathias Trabs
Abstract:
The estimation of the diffusion matrix $Σ$ of a high-dimensional, possibly time-changed Lévy process is studied, based on discrete observations of the process with a fixed distance. A low-rank condition is imposed on $Σ$. Applying a spectral approach, we construct a weighted least-squares estimator with nuclear-norm-penalisation. We prove oracle inequalities and derive convergence rates for the di…
▽ More
The estimation of the diffusion matrix $Σ$ of a high-dimensional, possibly time-changed Lévy process is studied, based on discrete observations of the process with a fixed distance. A low-rank condition is imposed on $Σ$. Applying a spectral approach, we construct a weighted least-squares estimator with nuclear-norm-penalisation. We prove oracle inequalities and derive convergence rates for the diffusion matrix estimator. The convergence rates show a surprising dependency on the rank of $Σ$ and are optimal in the minimax sense for fixed dimensions. Theoretical results are illustrated by a simulation study.
△ Less
Submitted 3 April, 2017; v1 submitted 15 October, 2015;
originally announced October 2015.
-
Statistical inference for generalized Ornstein-Uhlenbeck processes
Authors:
Denis Belomestny,
Vladimir Panov
Abstract:
In this paper, we consider the problem of statistical inference for generalized Ornstein-Uhlenbeck processes of the type \[ X_{t} = e^{-ξ_{t}} \left( X_{0} + \int_{0}^{t} e^{ξ_{u-}} d u \right), \] where \(ξ_s\) is a L{é}vy process. Our primal goal is to estimate the characteristics of the Lévy process \(ξ\) from the low-frequency observations of the process \(X\). We present a novel approach towa…
▽ More
In this paper, we consider the problem of statistical inference for generalized Ornstein-Uhlenbeck processes of the type \[ X_{t} = e^{-ξ_{t}} \left( X_{0} + \int_{0}^{t} e^{ξ_{u-}} d u \right), \] where \(ξ_s\) is a L{é}vy process. Our primal goal is to estimate the characteristics of the Lévy process \(ξ\) from the low-frequency observations of the process \(X\). We present a novel approach towards estimating the L{é}vy triplet of \(ξ,\) which is based on the Mellin transform technique. It is shown that the resulting estimates attain optimal minimax convergence rates. The suggested algorithms are illustrated by numerical simulations.
△ Less
Submitted 11 March, 2015;
originally announced March 2015.
-
Statistical Skorohod embedding problem and its generalizations
Authors:
Denis Belomestny,
John Schoenmakers
Abstract:
Given a Lévy process $L$, we consider the so-called statistical Skorohod embedding problem of recovering the distribution of an independent random time $T$ based on i.i.d. sample from $L_{T}.$ Our approach is based on the genuine use of the Mellin and Laplace transforms. We propose a consistent estimator for the density of $T,$ derive its convergence rates and prove their optimality. It turns out…
▽ More
Given a Lévy process $L$, we consider the so-called statistical Skorohod embedding problem of recovering the distribution of an independent random time $T$ based on i.i.d. sample from $L_{T}.$ Our approach is based on the genuine use of the Mellin and Laplace transforms. We propose a consistent estimator for the density of $T,$ derive its convergence rates and prove their optimality. It turns out that the convergence rates heavily depend on the decay of the Mellin transform of $T.$ We also consider the application of our results to the problem of statistical inference for variance-mean mixture models and for time-changed Lévy processes.
△ Less
Submitted 3 July, 2014;
originally announced July 2014.
-
Statistical inference for exponential functionals of Lévy processes
Authors:
Denis Belomestny,
Vladimir Panov
Abstract:
In this paper, we consider the exponential functional \(A_{\infty}=\int_0^\infty e^{-ξ_s}ds\) of a L{é}vy process \(ξ_s\) and aim to estimate the characteristics of \(ξ_{s}\) from the distribution of \(A_{\infty}\). We present a new approach, which allows to statistically infer on the L{é}vy triplet of \(ξ_{t}\), and study the theoretical properties of the proposed estimators. The suggested algori…
▽ More
In this paper, we consider the exponential functional \(A_{\infty}=\int_0^\infty e^{-ξ_s}ds\) of a L{é}vy process \(ξ_s\) and aim to estimate the characteristics of \(ξ_{s}\) from the distribution of \(A_{\infty}\). We present a new approach, which allows to statistically infer on the L{é}vy triplet of \(ξ_{t}\), and study the theoretical properties of the proposed estimators. The suggested algorithms are illustrated with numerical simulations.
△ Less
Submitted 17 December, 2013;
originally announced December 2013.
-
Statistical inference for time-changed Lévy processes via composite characteristic function estimation
Authors:
Denis Belomestny
Abstract:
In this article, the problem of semi-parametric inference on the parameters of a multidimensional Lévy process $L_t$ with independent components based on the low-frequency observations of the corresponding time-changed Lévy process $L_{\mathcal{T}(t)}$, where $\mathcal{T}$ is a nonnegative, nondecreasing real-valued process independent of $L_t$, is studied. We show that this problem is closely rel…
▽ More
In this article, the problem of semi-parametric inference on the parameters of a multidimensional Lévy process $L_t$ with independent components based on the low-frequency observations of the corresponding time-changed Lévy process $L_{\mathcal{T}(t)}$, where $\mathcal{T}$ is a nonnegative, nondecreasing real-valued process independent of $L_t$, is studied. We show that this problem is closely related to the problem of composite function estimation that has recently gotten much attention in statistical literature. Under suitable identifiability conditions, we propose a consistent estimate for the Lévy density of $L_t$ and derive the uniform as well as the pointwise convergence rates of the estimate proposed. Moreover, we prove that the rates obtained are optimal in a minimax sense over suitable classes of time-changed Lévy models. Finally, we present a simulation study showing the performance of our estimation algorithm in the case of time-changed Normal Inverse Gaussian (NIG) Lévy processes.
△ Less
Submitted 30 January, 2012; v1 submitted 1 March, 2010;
originally announced March 2010.
-
Spectral estimation of the Lévy density in partially observed affine models
Authors:
Denis Belomestny
Abstract:
The problem of estimating the Lévy density of a partially observed multidimensional affine process from low-frequency and mixed-frequency data is considered. The estimation methodology is based on the log-affine representation of the conditional characteristic function of an affine process and local linear smoothing in time. We derive almost sure uniform rates of convergence for the estimated Lévy…
▽ More
The problem of estimating the Lévy density of a partially observed multidimensional affine process from low-frequency and mixed-frequency data is considered. The estimation methodology is based on the log-affine representation of the conditional characteristic function of an affine process and local linear smoothing in time. We derive almost sure uniform rates of convergence for the estimated Lévy density both in mixed-frequency and low-frequency setups and prove that these rates are optimal in the minimax sense. Finally, the performance of the estimation algorithms is illustrated in the case of the Bates stochastic volatility model.
△ Less
Submitted 15 February, 2011; v1 submitted 28 July, 2009;
originally announced July 2009.
