-
A precise bare simulation approach to the minimization of some distances. II. Further Foundations
Authors:
Michel Broniatowski,
Wolfgang Stummer
Abstract:
The constrained minimization (respectively maximization) of directed distances and of related generalized entropies is a fundamental task in information theory as well as in the adjacent fields of statistics, machine learning, artificial intelligence, signal processing and pattern recognition. In our previous paper "A precise bare simulation approach to the minimization of some distances. I. Found…
▽ More
The constrained minimization (respectively maximization) of directed distances and of related generalized entropies is a fundamental task in information theory as well as in the adjacent fields of statistics, machine learning, artificial intelligence, signal processing and pattern recognition. In our previous paper "A precise bare simulation approach to the minimization of some distances. I. Foundations", we obtained such kind of constrained optima by a new dimension-free precise bare (pure) simulation method, provided basically that (i) the underlying directed distance is of f-divergence type, and that (ii) this can be connected to a light-tailed probability distribution in a certain manner. In the present paper, we extend this approach such that constrained optimization problems of a very huge amount of directed distances and generalized entropies -- and beyond -- can be tackled by a newly developed dimension-free extended bare simulation method, for obtaining both optima as well as optimizers. Almost no assumptions (like convexity) on the set of constraints are needed, within our discrete setup of arbitrary dimension, and our method is precise (i.e., converges in the limit). For instance, we cover constrained optimizations of arbitrary f-divergences, Bregman distances, scaled Bregman distances and weighted Euclidean distances. The potential for wide-spread applicability is indicated, too; in particular, we deliver many recent references for uses of the involved distances/divergences in various different research fields (which may also serve as an interdisciplinary interface).
△ Less
Submitted 13 February, 2024;
originally announced February 2024.
-
A Unifying Framework for Some Directed Distances in Statistics
Authors:
Michel Broniatowski,
Wolfgang Stummer
Abstract:
Density-based directed distances -- particularly known as divergences -- between probability distributions are widely used in statistics as well as in the adjacent research fields of information theory, artificial intelligence and machine learning. Prominent examples are the Kullback-Leibler information distance (relative entropy) which e.g. is closely connected to the omnipresent maximum likeliho…
▽ More
Density-based directed distances -- particularly known as divergences -- between probability distributions are widely used in statistics as well as in the adjacent research fields of information theory, artificial intelligence and machine learning. Prominent examples are the Kullback-Leibler information distance (relative entropy) which e.g. is closely connected to the omnipresent maximum likelihood estimation method, and Pearson's chisquare-distance which e.g. is used for the celebrated chisquare goodness-of-fit test. Another line of statistical inference is built upon distribution-function-based divergences such as e.g. the prominent (weighted versions of) Cramer-von Mises test statistics respectively Anderson-Darling test statistics which are frequently applied for goodness-of-fit investigations; some more recent methods deal with (other kinds of) cumulative paired divergences and closely related concepts. In this paper, we provide a general framework which covers in particular both the above-mentioned density-based and distribution-function-based divergence approaches; the dissimilarity of quantiles respectively of other statistical functionals will be included as well. From this framework, we structurally extract numerous classical and also state-of-the-art (including new) procedures. Furthermore, we deduce new concepts of dependence between random variables, as alternatives to the celebrated mutual information. Some variational representations are discussed, too.
△ Less
Submitted 1 March, 2022;
originally announced March 2022.
-
A precise bare simulation approach to the minimization of some distances. Foundations
Authors:
Michel Broniatowski,
Wolfgang Stummer
Abstract:
In information theory -- as well as in the adjacent fields of statistics, machine learning, artificial intelligence, signal processing and pattern recognition -- many flexibilizations of the omnipresent Kullback-Leibler information distance (relative entropy) and of the closely related Shannon entropy have become frequently used tools. To tackle corresponding constrained minimization (respectively…
▽ More
In information theory -- as well as in the adjacent fields of statistics, machine learning, artificial intelligence, signal processing and pattern recognition -- many flexibilizations of the omnipresent Kullback-Leibler information distance (relative entropy) and of the closely related Shannon entropy have become frequently used tools. To tackle corresponding constrained minimization (respectively maximization) problems by a newly developed dimension-free bare (pure) simulation method, is the main goal of this paper. Almost no assumptions (like convexity) on the set of constraints are needed, within our discrete setup of arbitrary dimension, and our method is precise (i.e., converges in the limit). As a side effect, we also derive an innovative way of constructing new useful distances/divergences. To illustrate the core of our approach, we present numerous solved cases. The potential for widespread applicability is indicated, too; in particular, we deliver many recent references for uses of the involved distances/divergences and entropies in various different research fields (which may also serve as an interdisciplinary interface).
△ Less
Submitted 15 November, 2022; v1 submitted 4 July, 2021;
originally announced July 2021.
-
Continuous indetermination and average likelihood minimization
Authors:
Pierre Bertrand,
Michel Broniatowski,
Jean-François Marcotorchino
Abstract:
The authors transpose a discrete notion of indetermination coupling in the case of continuous probabilities. They show that this coupling, expressed on densities, cannot be captured by a specific copula which acts on cumulative distribution functions without a high dependence on the margins. Furthermore, they define a notion of average likelihood which extends the discrete notion of couple matchin…
▽ More
The authors transpose a discrete notion of indetermination coupling in the case of continuous probabilities. They show that this coupling, expressed on densities, cannot be captured by a specific copula which acts on cumulative distribution functions without a high dependence on the margins. Furthermore, they define a notion of average likelihood which extends the discrete notion of couple matchings and demonstrate it is minimal under indetermination. Eventually, they leverage this property to build up a statistical test to distinguish indetermination and estimate its efficiency using the Bahadur's slope.
△ Less
Submitted 4 May, 2021;
originally announced May 2021.
-
A constructive method to minimize couple matchings
Authors:
Pierre Bertrand,
Michel Broniatowski,
Jean-François Marcotorchino
Abstract:
This paper provides constructive procedures for the indeterminacy coupling between two marginal distributions, an alternative to independence coupling. It also introduces a drawing under indeterminacy into a mixture of three independent couplings. Leveraging on this decomposition it states that indeterminacy optimally reduces couple matchings, minimizing the expected number of equal couples drawn…
▽ More
This paper provides constructive procedures for the indeterminacy coupling between two marginal distributions, an alternative to independence coupling. It also introduces a drawing under indeterminacy into a mixture of three independent couplings. Leveraging on this decomposition it states that indeterminacy optimally reduces couple matchings, minimizing the expected number of equal couples drawn in a row. Besides it is seen that the Janson Vegelius coefficient is nothing but a deviation to indeterminacy and it is shown that it tends to 0 when the number of modalities increases.
△ Less
Submitted 14 February, 2023; v1 submitted 29 December, 2020;
originally announced December 2020.
-
Minimum divergence estimators, Maximum Likelihood and the generalized bootstrap
Authors:
Michel Broniatowski
Abstract:
This paper is an attempt to set a justification for making use of some dicrepancy indexes, starting from the classical Maximum Likelihood definition, and adapting the corresponding basic principle of inference to situations where minimization of those indexes between a model and some extension of the empirical measure of the data appears as its natural extension. This leads to the so called genera…
▽ More
This paper is an attempt to set a justification for making use of some dicrepancy indexes, starting from the classical Maximum Likelihood definition, and adapting the corresponding basic principle of inference to situations where minimization of those indexes between a model and some extension of the empirical measure of the data appears as its natural extension. This leads to the so called generalized bootstrap setting for which minimum divergence inference seems to replace Maximum Likelihood one. 1 Motivation and context Divergences between probability measures are widely used in Statistics and Data Science in order to perform inference under models of various kinds, paramet-ric or semi parametric, or even in non parametric settings. The corresponding methods extend the likelihood paradigm and insert inference in some minimum "distance" framing, which provides a convenient description for the properties of the resulting estimators and tests, under the model or under misspecifica-tion. Furthermore they pave the way to a large number of competitive methods , which allows for trade-off between efficiency and robustness, among others. Many families of such divergences have been proposed, some of them stemming from classical statistics (such as the Chi-square), while others have their origin in other fields such as Information theory. Some measures of discrepancy involve regularity of the corresponding probability measures while others seem to be restricted to measures on finite or countable spaces, at least when using them as inferential tools, henceforth in situations when the elements of a model have to be confronted with a dataset. The choice of a specific discrepancy measure in specific context is somehow arbitrary in many cases, although the resulting conclusion of the inference might differ accordingly, above all under misspecification; however the need for such approaches is clear when aiming at robustness.
△ Less
Submitted 3 November, 2020;
originally announced November 2020.
-
Independence versus Indetermination: basis of two canonical clustering criteria
Authors:
Pierre Bertrand,
Michel Broniatowski,
Jean-François Marcotorchino
Abstract:
This paper aims at comparing two coupling approaches as basic layers for building clustering criteria, suited for modularizing and clustering very large networks. We briefly use "optimal transport theory" as a starting point, and a way as well, to derive two canonical couplings: "statistical independence" and "logical indetermination". A symmetric list of properties is provided and notably the so…
▽ More
This paper aims at comparing two coupling approaches as basic layers for building clustering criteria, suited for modularizing and clustering very large networks. We briefly use "optimal transport theory" as a starting point, and a way as well, to derive two canonical couplings: "statistical independence" and "logical indetermination". A symmetric list of properties is provided and notably the so called "Monge's properties", applied to contingency matrices, and justifying the $\otimes$ versus $\oplus$ notation. A study is proposed, highlighting "logical indetermination", because it is, by far, lesser known. Eventually we estimate the average difference between both couplings as the key explanation of their usually close results in network clustering.
△ Less
Submitted 18 March, 2021; v1 submitted 17 July, 2020;
originally announced July 2020.
-
A sequential design for extreme quantiles estimation under binary sampling
Authors:
Michel Broniatowski,
Emilie Miranda
Abstract:
We propose a sequential design method aiming at the estimation of an extreme quantile based on a sample of dichotomic data corresponding to peaks over a given threshold. This study is motivated by an industrial challenge in material reliability and consists in estimating a failure quantile from trials whose outcomes are reduced to indicators of whether the specimen have failed at the tested stress…
▽ More
We propose a sequential design method aiming at the estimation of an extreme quantile based on a sample of dichotomic data corresponding to peaks over a given threshold. This study is motivated by an industrial challenge in material reliability and consists in estimating a failure quantile from trials whose outcomes are reduced to indicators of whether the specimen have failed at the tested stress levels. The solution proposed is a sequential design making use of a splitting approach, decomposing the target probability level into a product of probabilities of conditional events of higher order. The method consists in gradually targeting the tail of the distribution and sampling under truncated distributions. The model is GEV or Weibull, and sequential estimation of its parameters involves an improved maximum likelihood procedure for binary data, due to the large uncertainty associated with such a restricted information.
△ Less
Submitted 3 April, 2020;
originally announced April 2020.
-
Uniform minimum risk equivariant estimates for moment condition models
Authors:
Jana Jureckova,
Amor Keziou,
Michel Broniatowski,
Jana CkovÂá,
CkovÂ\' CkovÂá,
Amor and
Abstract:
We consider semiparametric moment condition models invariant to transformation groups. The parameter of interest is estimated by minimum empirical divergence approach, introduced by Broniatowski and Keziou (2012). It is shown that the minimum empirical divergence estimates, including the empirical likelihood one, are equivariants. The minimum risk equivariant estimate is then identied to be any on…
▽ More
We consider semiparametric moment condition models invariant to transformation groups. The parameter of interest is estimated by minimum empirical divergence approach, introduced by Broniatowski and Keziou (2012). It is shown that the minimum empirical divergence estimates, including the empirical likelihood one, are equivariants. The minimum risk equivariant estimate is then identied to be any one of the minimum empirical divergence estimates minus its expectation conditionally to maximal invariant statistic of the considered group of transformations. An asymptotic approximation to the conditional expectation, is obtained, using the result of Jureckov{á} and Picek (2009).
△ Less
Submitted 25 April, 2019;
originally announced April 2019.
-
A Gibbs Conditional theorem under extreme deviation
Authors:
Maeva Biret,
Michel Broniatowski,
Zangsheng Cao
Abstract:
We explore some properties of the conditional distribution of an i.i.d. sample under large exceedances of its sum. Thresholds for the asymptotic independance of the summands are observed, in contrast with the classical case when the conditioning event is in the range of a large deviation. This paper is an extension to [7]. Tools include a new Edgeworth expansion adapted to specific triangular arra…
▽ More
We explore some properties of the conditional distribution of an i.i.d. sample under large exceedances of its sum. Thresholds for the asymptotic independance of the summands are observed, in contrast with the classical case when the conditioning event is in the range of a large deviation. This paper is an extension to [7]. Tools include a new Edgeworth expansion adapted to specific triangular arrays where the rows are generated by tilted distribution with diverging parameters, together with some Abelian type results.
△ Less
Submitted 13 October, 2016;
originally announced October 2016.
-
A recursive algorithm for a pipeline maintenance scheduling problem
Authors:
Assia Boumahdaf,
Michel Broniatowski
Abstract:
This paper deals with the problem of preventive maintenance (PM) scheduling of pipelines subject to external corrosion defects. The preventive maintenance strategy involves an inspection step at some epoch, together with a repair schedule. This paper proposes to determine the repair schedule as well as an inspection time minimizing the maintenance cost. This problem is formulated as a binary integ…
▽ More
This paper deals with the problem of preventive maintenance (PM) scheduling of pipelines subject to external corrosion defects. The preventive maintenance strategy involves an inspection step at some epoch, together with a repair schedule. This paper proposes to determine the repair schedule as well as an inspection time minimizing the maintenance cost. This problem is formulated as a binary integer non-linear programming model and we approach it under a decision support framework. We derive a polynomial-time algorithm that computes the optimum PM schedule and suggests different PM strategies in order to assist practitioners in making decision.
△ Less
Submitted 4 October, 2016;
originally announced October 2016.
-
SAFIP: a streaming algorithm for inverse problems
Authors:
Maeva Biret,
Michel Broniatowski
Abstract:
This paper presents a new algorithm which aims at the resolution of inverse problems of the form f(x) = 0, for x a vector of dimension d and f an arbitrary function with mild regularity condition. The set of solutions S may be infinite. This algorithm produces a good coverage of S, with a limited number of evaluations of the function f. It is therefore appropriate for complex problems where those…
▽ More
This paper presents a new algorithm which aims at the resolution of inverse problems of the form f(x) = 0, for x a vector of dimension d and f an arbitrary function with mild regularity condition. The set of solutions S may be infinite. This algorithm produces a good coverage of S, with a limited number of evaluations of the function f. It is therefore appropriate for complex problems where those evaluations are costly. Various examples are presented, with d varying from 2 to 10. Proofs of convergence and of coverage of S are presented.
△ Less
Submitted 27 September, 2016;
originally announced September 2016.
-
Two Iterative Proximal-Point Algorithms for the Calculus of Divergence-based Estimators with Application to Mixture Models
Authors:
Diaa Al Mohamad,
Michel Broniatowski
Abstract:
Estimators derived from an EM algorithm are not robust since they are based on the maximization of the likelihood function. We propose a proximal-point algorithm based on the EM algorithm which aim to minimize a divergence criterion. Resulting estimators are generally robust against outliers and misspecification. An EM-type proximal-point algorithm is also introduced in order to produce robust est…
▽ More
Estimators derived from an EM algorithm are not robust since they are based on the maximization of the likelihood function. We propose a proximal-point algorithm based on the EM algorithm which aim to minimize a divergence criterion. Resulting estimators are generally robust against outliers and misspecification. An EM-type proximal-point algorithm is also introduced in order to produce robust estimators for mixture models. Convergence properties of the two algorithms are treated. We relax an identifiability condition imposed on the proximal term in the literature; a condition which is generally not fulfilled by mixture models. The convergence of the introduced algorithms is discussed on a two-component Weibull mixture and a two-component Gaussian mixture entailing a condition on the initialization of the EM algorithm in order for the later to converge. Simulations on mixture models using different statistical divergences are provided to confirm the validity of our work and the robustness of the resulting estimators against outliers in comparison to the EM algorithm.
△ Less
Submitted 7 July, 2016;
originally announced July 2016.
-
A Proximal Point Algorithm for Minimum Divergence Estimators with Application to Mixture Models
Authors:
Diaa Al Mohamad,
Michel Broniatowski
Abstract:
Estimators derived from a divergence criterion such as $\varphi-$divergences are generally more robust than the maximum likelihood ones. We are interested in particular in the so-called MD$\varphi$DE, an estimator built using a dual representation of $\varphi$--divergences. We present in this paper an iterative proximal point algorithm which permits to calculate such estimator. This algorithm cont…
▽ More
Estimators derived from a divergence criterion such as $\varphi-$divergences are generally more robust than the maximum likelihood ones. We are interested in particular in the so-called MD$\varphi$DE, an estimator built using a dual representation of $\varphi$--divergences. We present in this paper an iterative proximal point algorithm which permits to calculate such estimator. This algorithm contains by its construction the well-known EM algorithm. Our work is based on the paper of \citep{Tseng} on the likelihood function. We provide several convergence properties of the sequence generated by the algorithm, and improve the existing results by relaxing the identifiability condition on the proximal term, a condition which is not verified for most mixture models and hard to be verified for non mixture ones. Since convergence analysis uses regularity conditions (continuity and differentiability) of the objective function, which has a supremal form, we find it useful to present some analytical approaches for studying such functions. Convergence of the EM algorithm is discussed here again in a Gaussian and Weibull mixtures in the spirit of our approach. Simulations are provided to confirm the validity of our work and the robustness of the resulting estimators against outliers.
△ Less
Submitted 12 June, 2016; v1 submitted 23 March, 2016;
originally announced March 2016.
-
Estimation for models defined by conditions on their L-moments
Authors:
Alexis Decurninge,
Michel Broniatowski
Abstract:
This paper extends the empirical minimum divergence approach for models which satisfy linear constraints with respect to the probability measure of the underlying variable (moment constraints) to the case where such constraints pertain to its quantile measure (called here semi parametric quantile models). The case when these constraints describe shape conditions as handled by the L-moments is cons…
▽ More
This paper extends the empirical minimum divergence approach for models which satisfy linear constraints with respect to the probability measure of the underlying variable (moment constraints) to the case where such constraints pertain to its quantile measure (called here semi parametric quantile models). The case when these constraints describe shape conditions as handled by the L-moments is considered and both the description of these models as well as the resulting non classical minimum divergence procedures are presented. These models describe neighborhoods of classical models used mainly for their tail behavior, for example neighborhoods of Pareto or Weibull distributions, with which they may share the same first L-moments. A parallel is drawn with similar problems held in elasticity theory and in optimal transport problems. The properties of the resulting estimators are illustrated by simulated examples comparing Maximum Likelihood estimators on Pareto and Weibull models to the minimum Chi-square empirical divergence approach on semi parametric quantile models, and others.
△ Less
Submitted 19 February, 2015; v1 submitted 20 September, 2014;
originally announced September 2014.
-
Some overview on unbiased interpolation and extrapolation designs
Authors:
Michel Broniatowski,
Giorgio Celant
Abstract:
This paper considers the construction of optimal designs due to Hoel and Levine and Guest. It focuses on the relation between the theory of the uniform approximation of functions and the optimality of the designs. Some application to accelerated tests is also presented. The multivariate case is also handled in some special situations.
This paper considers the construction of optimal designs due to Hoel and Levine and Guest. It focuses on the relation between the theory of the uniform approximation of functions and the optimality of the designs. Some application to accelerated tests is also presented. The multivariate case is also handled in some special situations.
△ Less
Submitted 20 March, 2014;
originally announced March 2014.
-
A sharp Abelian theorem for the Laplace transform
Authors:
Maeva Biret,
Michel Broniatowski,
Zhansheng Cao
Abstract:
This paper states asymptotic equivalents for the three first moments of the Eescher transform of a distribution on R with smooth density in the upper tail. As a by product if provides a tail approximation for its moment generating function, and shows that the Esscher transforms have a Gaussian behavior for large values of the parameter.
This paper states asymptotic equivalents for the three first moments of the Eescher transform of a distribution on R with smooth density in the upper tail. As a by product if provides a tail approximation for its moment generating function, and shows that the Esscher transforms have a Gaussian behavior for large values of the parameter.
△ Less
Submitted 20 March, 2014; v1 submitted 24 September, 2013;
originally announced September 2013.
-
Light tails: Gibbs conditional principle under extreme deviation
Authors:
Michel Broniatowski,
Zhansheng Cao
Abstract:
Let $X_{1},..,X_{n}$ denote an i.i.d. sample with light tail distribution and $S_{1}^{n}$ denote the sum of its terms; let $a_{n}$ be a real sequence\ going to infinity with $n.$\ In a previous paper (\cite{BoniaCao}) it is proved that as $n\rightarrow\infty$, given $\left(S_{1}^{n}/n>a_{n}\right) $ all terms $X_{i_{\text{}}}$ concentrate around $a_{n}$ with probability going to 1. This paper exp…
▽ More
Let $X_{1},..,X_{n}$ denote an i.i.d. sample with light tail distribution and $S_{1}^{n}$ denote the sum of its terms; let $a_{n}$ be a real sequence\ going to infinity with $n.$\ In a previous paper (\cite{BoniaCao}) it is proved that as $n\rightarrow\infty$, given $\left(S_{1}^{n}/n>a_{n}\right) $ all terms $X_{i_{\text{}}}$ concentrate around $a_{n}$ with probability going to 1. This paper explores the asymptotic distribution of $X_{1}$ under the conditioning events $\left(S_{1}^{n}/n=a_{n}\right) $ and $\left(S_{1}^{n}/n\geq a_{n}\right)$ . It is proved that under some regulatity property, the asymptotic conditional distribution of $X_{1}$ given $\left(S_{1}^{n}/n=a_{n}\right) $ can be approximated in variation norm by the tilted distribution at point $a_{n}$, extending therefore the classical LDP case developed in Diaconis and Freedman (1988) . Also under $\left(S_{1}^{n}/n\geq a_{n}\right) $ the dominating point property holds.
It also considers the case when the $X_{i}$'s are $\mathbb{R}^{d}-$valued, $f$ is a real valued function defined on $\mathbb{R}^{d}$ and the conditioning event writes $\left(U_{1}^{n}/n=a_{n}\right) $ or $\left(U_{1}^{n}/n\geq a_{n}\right)$ with $U_{1}^{n}:=\left(f(X_{1})+..+f(X_{n})\right) /n$ and $f(X_{1})$ has a light tail distribution$.$ As a by-product some attention is paid to the estimation of high level sets of functions.
△ Less
Submitted 15 May, 2013;
originally announced May 2013.
-
Weighted sampling, Maximum Likelihood and minimum divergence estimators
Authors:
Michel Broniatowski,
Zhansheng Cao
Abstract:
This paper explores Maximum Likelihood in parametric models in the context of Sanov type Large Deviation Probabilities. MLE in parametric models under weighted sampling is shown to be associated with the minimization of a specific divergence criterion defined with respect to the distribution of the weights. Some properties of the resulting inferential procedure are presented; Bahadur efficiency of…
▽ More
This paper explores Maximum Likelihood in parametric models in the context of Sanov type Large Deviation Probabilities. MLE in parametric models under weighted sampling is shown to be associated with the minimization of a specific divergence criterion defined with respect to the distribution of the weights. Some properties of the resulting inferential procedure are presented; Bahadur efficiency of tests are also considered in this context.
△ Less
Submitted 27 July, 2012;
originally announced July 2012.
-
A conditional limit theorem for random walks under extreme deviation
Authors:
Michel Broniatowski,
Zhansheng Cao
Abstract:
This paper explores a conditional Gibbs theorem for a random walkinduced by i.i.d. (X_{1},..,X_{n}) conditioned on an extreme deviation of its sum (S_{1}^{n}=na_{n}) or (S_{1}^{n}>na_{n}) where a_{n}\rightarrow\infty. It is proved that when the summands have light tails with some additional regulatity property, then the asymptotic conditional distribution of X_{1} can be approximated in variation…
▽ More
This paper explores a conditional Gibbs theorem for a random walkinduced by i.i.d. (X_{1},..,X_{n}) conditioned on an extreme deviation of its sum (S_{1}^{n}=na_{n}) or (S_{1}^{n}>na_{n}) where a_{n}\rightarrow\infty. It is proved that when the summands have light tails with some additional regulatity property, then the asymptotic conditional distribution of X_{1} can be approximated in variation norm by the tilted distribution at point a_{n}, extending therefore the classical LDP case.
△ Less
Submitted 29 June, 2012;
originally announced June 2012.
-
Stretched random walks and the behaviour of their summands
Authors:
Michel Broniatowski,
Zhansheng Cao
Abstract:
This paper explores the joint behaviour of the summands of a random walk when their mean value goes to infinity as its length increases. It is proved that all the summands must share the same value, which extends previous results in the context of large exceedances of finite sums of i.i.d. random variables. Some consequences are drawn pertaining to the local behaviour of a random walk conditioned…
▽ More
This paper explores the joint behaviour of the summands of a random walk when their mean value goes to infinity as its length increases. It is proved that all the summands must share the same value, which extends previous results in the context of large exceedances of finite sums of i.i.d. random variables. Some consequences are drawn pertaining to the local behaviour of a random walk conditioned on a large deviation constraint on its end value. It is shown that the sample paths exhibit local oblic segments with increasing size and slope as the length of the random walk increases.
△ Less
Submitted 27 May, 2012;
originally announced May 2012.
-
Conditional inference in parametric models
Authors:
Michel Broniatowski,
Virgile Caron
Abstract:
This paper presents a new approach to conditional inference, based on the simulation of samples conditioned by a statistics of the data. Also an explicit expression for the approximation of the conditional likelihood of long runs of the sample given the observed statistics is provided. It is shown that when the conditioning statistics is sufficient for a given parameter, the approximating density…
▽ More
This paper presents a new approach to conditional inference, based on the simulation of samples conditioned by a statistics of the data. Also an explicit expression for the approximation of the conditional likelihood of long runs of the sample given the observed statistics is provided. It is shown that when the conditioning statistics is sufficient for a given parameter, the approximating density is still invariant with respect to the parameter. A new Rao-Blackwellisation procedure is proposed and simulation shows that Lehmann Scheffé Theorem is valid for this approximation. Conditional inference for exponential families with nuisance parameter is also studied, leading to Monte carlo tests. Finally the estimation of the parameter of interest through conditional likelihood is considered. Comparison with the parametric bootstrap method is discussed.
△ Less
Submitted 5 February, 2012;
originally announced February 2012.
-
Long runs under a conditional limit distribution
Authors:
Michel Broniatowski,
Virgile Caron
Abstract:
This paper presents a sharp approximation of the density of long runs of a random walk conditioned on its end value or by an average of a function of its summands as their number tends to infinity. In the large deviation range of the conditioning event it extends the Gibbs conditional principle in the sense that it provides a description of the distribution of the random walk on long subsequences.…
▽ More
This paper presents a sharp approximation of the density of long runs of a random walk conditioned on its end value or by an average of a function of its summands as their number tends to infinity. In the large deviation range of the conditioning event it extends the Gibbs conditional principle in the sense that it provides a description of the distribution of the random walk on long subsequences. An approximation of the density of the runs is also obtained when the conditioning event states that the end value of the random walk belongs to a thin or a thick set with a nonempty interior. The approximations hold either in probability under the conditional distribution of the random walk, or in total variation norm between measures. An application of the approximation scheme to the evaluation of rare event probabilities through importance sampling is provided. When the conditioning event is in the range of the central limit theorem, it provides a tool for statistical inference in the sense that it produces an effective way to implement the Rao-Blackwell theorem for the improvement of estimators; it also leads to conditional inference procedures in models with nuisance parameters. An algorithm for the simulation of such long runs is presented, together with an algorithm determining the maximal length for which the approximation is valid up to a prescribed accuracy.
△ Less
Submitted 5 September, 2014; v1 submitted 3 February, 2012;
originally announced February 2012.
-
Minimum divergence estimators, maximum likelihood and exponential families
Authors:
Michel Broniatowski
Abstract:
In this note we prove the dual representation formula of the divergence between two distributions in a parametric model. Resulting estimators for the divergence as for the parameter are derived. These estimators do not make use of any grou** nor smoothing. It is proved that all differentiable divergences induce the same estimator of the parameter on any regular exponential family, which is nothi…
▽ More
In this note we prove the dual representation formula of the divergence between two distributions in a parametric model. Resulting estimators for the divergence as for the parameter are derived. These estimators do not make use of any grou** nor smoothing. It is proved that all differentiable divergences induce the same estimator of the parameter on any regular exponential family, which is nothing else but the MLE.
△ Less
Submitted 20 August, 2011; v1 submitted 3 August, 2011;
originally announced August 2011.
-
Decomposable Pseudodistances and Applications in Statistical Estimation
Authors:
Michel Broniatowski,
Aida Toma,
Igor Vajda
Abstract:
The aim of this paper is to introduce new statistical criterions for estimation, suitable for inference in models with common continuous support. This proposal is in the direct line of a renewed interest for divergence based inference tools imbedding the most classical ones, such as maximum likelihood, Chi-square or Kullback Leibler. General pseudodistances with decomposable structure are consider…
▽ More
The aim of this paper is to introduce new statistical criterions for estimation, suitable for inference in models with common continuous support. This proposal is in the direct line of a renewed interest for divergence based inference tools imbedding the most classical ones, such as maximum likelihood, Chi-square or Kullback Leibler. General pseudodistances with decomposable structure are considered, they allowing to define minimum pseudodistance estimators, without using nonparametric density estimators. A special class of pseudodistances indexed by α>0, leading for α\downarrow0 to the Kulback Leibler divergence, is presented in detail. Corresponding estimation criteria are developed and asymptotic properties are studied. The estimation method is then extended to regression models. Finally, some examples based on Monte Carlo simulations are discussed.
△ Less
Submitted 8 April, 2011;
originally announced April 2011.
-
Towards zero variance estimators for rare event probabilities
Authors:
Michel Broniatowski,
Virgile Caron
Abstract:
Improving Importance Sampling estimators for rare event probabilities requires sharp approximations of conditional densities. This is achieved for events E_{n}:=(f(X_{1})+...+f(X_{n}))\inA_{n} where the summands are i.i.d. and E_{n} is a large or moderate deviation event. The approximation of the conditional density of the real r.v's X_{i} 's, for 1\leqi\leqk_{n} with repect to E_{n} on long runs,…
▽ More
Improving Importance Sampling estimators for rare event probabilities requires sharp approximations of conditional densities. This is achieved for events E_{n}:=(f(X_{1})+...+f(X_{n}))\inA_{n} where the summands are i.i.d. and E_{n} is a large or moderate deviation event. The approximation of the conditional density of the real r.v's X_{i} 's, for 1\leqi\leqk_{n} with repect to E_{n} on long runs, when k_{n}/n\to1, is handled. The maximal value of k compatible with a given accuracy is discussed; algorithms and simulated results are presented.
△ Less
Submitted 6 February, 2012; v1 submitted 7 April, 2011;
originally announced April 2011.
-
An estimation method for the chi-square divergence with application to test of hypotheses
Authors:
Michel Broniatowski,
Samantha Leorato
Abstract:
We propose a new definition of the chi-square divergence between distributions. Based on convexity properties and duality, this version of the χ^2 is well suited both for the classical applications of the χ^2 for the analysis of contingency tables and for the statistical tests for parametric models, for which it has been advocated to be robust against inliers. We present two applications in testin…
▽ More
We propose a new definition of the chi-square divergence between distributions. Based on convexity properties and duality, this version of the χ^2 is well suited both for the classical applications of the χ^2 for the analysis of contingency tables and for the statistical tests for parametric models, for which it has been advocated to be robust against inliers. We present two applications in testing. In the first one we deal with tests for finite and infinite numbers of linear constraints, while, in the second one, we apply χ^2-methodology for parametric testing against contamination.
△ Less
Submitted 23 January, 2011;
originally announced January 2011.
-
Upper bounds for the error in some interpolation and extrapolation designs
Authors:
Michel Broniatowski,
Giorgio Celant,
Marco Di Battista,
Samuela Leoni-Aubin
Abstract:
This paper deals with probabilistic upper bounds for the error in functional estimation defined on some interpolation and extrapolation designs, when the function to estimate is supposed to be analytic. The error pertaining to the estimate may depend on various factors: the frequency of observations on the knots, the position and number of the knots, and also on the error committed when approximat…
▽ More
This paper deals with probabilistic upper bounds for the error in functional estimation defined on some interpolation and extrapolation designs, when the function to estimate is supposed to be analytic. The error pertaining to the estimate may depend on various factors: the frequency of observations on the knots, the position and number of the knots, and also on the error committed when approximating the function through its Taylor expansion. When the number of observations is fixed, then all these parameters are determined by the choice of the design and by the choice estimator of the unknown function. The scope of the paper is therefore to determine a rule for the minimal number of observation required to achieve an upper bound of the error on the estimate with a given maximal probability.
△ Less
Submitted 23 January, 2011;
originally announced January 2011.
-
Long runs under point conditioning. The real case
Authors:
Michel Broniatowski,
Virgile Caron
Abstract:
This paper presents a sharp approximation of the density of long runs of a random walk conditioned on its end value or by an average of a functions of its summands as their number tends to infinity. The conditioning event is of moderate or large deviation type. The result extends the Gibbs conditional principle in the sense that it provides a description of the distribution of the random walk on l…
▽ More
This paper presents a sharp approximation of the density of long runs of a random walk conditioned on its end value or by an average of a functions of its summands as their number tends to infinity. The conditioning event is of moderate or large deviation type. The result extends the Gibbs conditional principle in the sense that it provides a description of the distribution of the random walk on long subsequences. An algorithm for the simulation of such long runs is presented, together with an algorithm determining their maximal length for which the approximation is valid up to a prescribed accuracy.
△ Less
Submitted 12 June, 2011; v1 submitted 18 October, 2010;
originally announced October 2010.
-
Minimization of divergences on sets of signed measures
Authors:
Michel Broniatowski,
Amor Keziou
Abstract:
We consider the minimization problem of $φ$-divergences between a given probability measure $P$ and subsets $Ω$ of the vector space $\mathcal{M}_\mathcal{F}$ of all signed finite measures which integrate a given class $\mathcal{F}$ of bounded or unbounded measurable functions. The vector space $\mathcal{M}_\mathcal{F}$ is endowed with the weak topology induced by the class…
▽ More
We consider the minimization problem of $φ$-divergences between a given probability measure $P$ and subsets $Ω$ of the vector space $\mathcal{M}_\mathcal{F}$ of all signed finite measures which integrate a given class $\mathcal{F}$ of bounded or unbounded measurable functions. The vector space $\mathcal{M}_\mathcal{F}$ is endowed with the weak topology induced by the class $\mathcal{F}\cup \mathcal{B}_b$ where $\mathcal{B}_b$ is the class of all bounded measurable functions. We treat the problems of existence and characterization of the $φ$-projections of $P$ on $Ω$. We consider also the dual equality and the dual attainment problems when $Ω$ is defined by linear constraints.
△ Less
Submitted 29 March, 2010;
originally announced March 2010.
-
Divergences and Duality for Estimation and Test under Moment Condition Models
Authors:
Michel Broniatowski,
Amor Keziou
Abstract:
We introduce estimation and test procedures through divergence minimiza- tion for models satisfying linear constraints with unknown parameter. These procedures extend the empirical likelihood (EL) method and share common features with generalized empirical likelihood approach. We treat the problems of existence and characterization of the divergence projections of probability distributions on sets…
▽ More
We introduce estimation and test procedures through divergence minimiza- tion for models satisfying linear constraints with unknown parameter. These procedures extend the empirical likelihood (EL) method and share common features with generalized empirical likelihood approach. We treat the problems of existence and characterization of the divergence projections of probability distributions on sets of signed finite measures. We give a precise characterization of duality, for the proposed class of estimates and test statistics, which is used to derive their limiting distributions (including the EL estimate and the EL ratio statistic) both under the null hypotheses and under alterna- tives or misspecification. An approximation to the power function is deduced as well as the sample size which ensures a desired power for a given alternative.
△ Less
Submitted 11 November, 2011; v1 submitted 3 February, 2010;
originally announced February 2010.
-
Dual divergence estimators and tests: robustness results
Authors:
Aida Toma,
Michel Broniatowski
Abstract:
The class of dual $φ$-divergence estimators (introduced in Broniatowski and Keziou (2009) is explored with respect to robustness through the influence function approach. For scale and location models, this class is investigated in terms of robustness and asymptotic relative efficiency. Some hypothesis tests based on dual divergence criterions are proposed and their robustness properties are stud…
▽ More
The class of dual $φ$-divergence estimators (introduced in Broniatowski and Keziou (2009) is explored with respect to robustness through the influence function approach. For scale and location models, this class is investigated in terms of robustness and asymptotic relative efficiency. Some hypothesis tests based on dual divergence criterions are proposed and their robustness properties are studied. The empirical performances of these estimators and tests are illustrated by Monte Carlo simulation for both noncontaminated and contaminated data.
△ Less
Submitted 15 December, 2009; v1 submitted 14 December, 2009;
originally announced December 2009.
-
Bivariate Cox model and copulas
Authors:
Mohamed Achibi,
Michel Broniatowski
Abstract:
This paper introduces a new class of Cox models for dependent bivariate data. The impact of the covariate on the dependence of the variables is captured through the modification of their copula. Various classes of well known copulas are stable under the model (archimedean type and extreme value copulas), meaning that the role of the covariate acts in a simple and explicit way on the copula in the…
▽ More
This paper introduces a new class of Cox models for dependent bivariate data. The impact of the covariate on the dependence of the variables is captured through the modification of their copula. Various classes of well known copulas are stable under the model (archimedean type and extreme value copulas), meaning that the role of the covariate acts in a simple and explicit way on the copula in the class; specific parametric classes are considered.
△ Less
Submitted 30 June, 2010; v1 submitted 7 November, 2009;
originally announced November 2009.
-
Several Applications of Divergence Criteria in Continuous Families
Authors:
Michel Broniatowski,
Igor Vajda
Abstract:
This paper deals with four types of point estimators based on minimization of information-theoretic divergences between hypothetical and empirical distributions. These were introduced (i) by Liese & Vajda (2006) and independently Broniatowski & Keziou (2006), called here power superdivergence estimators, (ii) by Broniatowski & Keziou (2009), called here power subdivergence estimators, (iii) by B…
▽ More
This paper deals with four types of point estimators based on minimization of information-theoretic divergences between hypothetical and empirical distributions. These were introduced (i) by Liese & Vajda (2006) and independently Broniatowski & Keziou (2006), called here power superdivergence estimators, (ii) by Broniatowski & Keziou (2009), called here power subdivergence estimators, (iii) by Basu et al. (1998), called here power pseudodistance estimators, and (iv) by Vajda (2008) called here Renyi pseudodistance estimators. The paper studies and compares general properties of these estimators such as consistency and influence curves, and illustrates these properties by detailed analysis of the applications to the estimation of normal location and scale.
△ Less
Submitted 4 November, 2009;
originally announced November 2009.
-
Importance Sampling for rare events and conditioned random walks
Authors:
Michel Broniatowski,
Ya'Acov Ritov
Abstract:
This paper introduces a new Importance Sampling scheme, called Adaptive Twisted Importance Sampling, which is adequate for the improved estimation of rare event probabilities in he range of moderate deviations pertaining to the empirical mean of real i.i.d. summands. It is based on a sharp approximation of the density of long runs extracted from a random walk conditioned on its end value.
This paper introduces a new Importance Sampling scheme, called Adaptive Twisted Importance Sampling, which is adequate for the improved estimation of rare event probabilities in he range of moderate deviations pertaining to the empirical mean of real i.i.d. summands. It is based on a sharp approximation of the density of long runs extracted from a random walk conditioned on its end value.
△ Less
Submitted 9 October, 2009;
originally announced October 2009.
-
Parametric estimation and tests through divergences and duality technique
Authors:
Michel Broniatowski,
Amor Keziou
Abstract:
We introduce estimation and test procedures through divergence optimization for discrete or continuous parametric models. This approach is based on a new dual representation for divergences. We treat point estimation and tests for simple and composite hypotheses, extending maximum likelihood technique. An other view at the maximum likelihood approach, for estimation and test, is given. We prove…
▽ More
We introduce estimation and test procedures through divergence optimization for discrete or continuous parametric models. This approach is based on a new dual representation for divergences. We treat point estimation and tests for simple and composite hypotheses, extending maximum likelihood technique. An other view at the maximum likelihood approach, for estimation and test, is given. We prove existence and consistency of the proposed estimates. The limit laws of the estimates and test statistics (including the generalized likelihood ratio one) are given both under the null and the alternative hypotheses, and approximation of the power functions is deduced. A new procedure of construction of confidence regions, when the parameter may be a boundary value of the parameter space, is proposed. Also, a solution to the irregularity problem of the generalized likelihood ratio test pertaining to the number of components in a mixture is given, and a new test is proposed, based on $χ^{2}$-divergence on signed finite measures and duality technique.
△ Less
Submitted 22 November, 2008;
originally announced November 2008.
-
Estimation and tests for models satisfying linear constraints with unknown parameter
Authors:
Michel Broniatowski,
Amor Keziou
Abstract:
We introduce estimation and test procedures through divergence minimization for models satisfying linear constraints with unknown parameter. Several statistical examples and motivations are given. These procedures extend the empirical likelihood (EL) method and share common features with generalized empirical likelihood (GEL). We treat the problems of existence and characterization of the diverg…
▽ More
We introduce estimation and test procedures through divergence minimization for models satisfying linear constraints with unknown parameter. Several statistical examples and motivations are given. These procedures extend the empirical likelihood (EL) method and share common features with generalized empirical likelihood (GEL). We treat the problems of existence and characterization of the divergence projections of probability measures on sets of signed finite measures. Our approach allows for a study of the estimates under misspecification. The asymptotic behavior of the proposed estimates are studied using the dual representation of the divergences and the explicit forms of the divergence projections. We discuss the problem of the choice of the divergence under various respects. Also we handle efficiency and robustness properties of minimum divergence estimates. A simulation study shows that the Hellinger divergence enjoys good efficiency and robustness properties.
△ Less
Submitted 21 November, 2008;
originally announced November 2008.
