-
Loss-based prior for tree topologies in BART models
Authors:
F. Serafini,
F. Leisen,
C. Villa,
K. Wilson
Abstract:
We present a novel prior for tree topology within Bayesian Additive Regression Trees (BART) models. This approach quantifies the hypothetical loss in information and the loss due to complexity associated with choosing the wrong tree structure. The resulting prior distribution is compellingly geared toward sparsity, a critical feature considering BART models' tendency to overfit. Our method incorpo…
▽ More
We present a novel prior for tree topology within Bayesian Additive Regression Trees (BART) models. This approach quantifies the hypothetical loss in information and the loss due to complexity associated with choosing the wrong tree structure. The resulting prior distribution is compellingly geared toward sparsity, a critical feature considering BART models' tendency to overfit. Our method incorporates prior knowledge into the distribution via two parameters that govern the tree's depth and balance between its left and right branches. Additionally, we propose a default calibration for these parameters, offering an objective version of the prior. We demonstrate our method's efficacy on both simulated and real datasets.
△ Less
Submitted 30 March, 2024;
originally announced April 2024.
-
A multidimensional objective prior distribution from a scoring rule
Authors:
Isadora Antoniano-Villalobos,
Cristiano Villa,
Stephen G. Walker
Abstract:
The construction of objective priors is, at best, challenging for multidimensional parameter spaces. A common practice is to assume independence and set up the joint prior as the product of marginal distributions obtained via "standard" objective methods, such as Jeffreys or reference priors. However, the assumption of independence a priori is not always reasonable, and whether it can be viewed as…
▽ More
The construction of objective priors is, at best, challenging for multidimensional parameter spaces. A common practice is to assume independence and set up the joint prior as the product of marginal distributions obtained via "standard" objective methods, such as Jeffreys or reference priors. However, the assumption of independence a priori is not always reasonable, and whether it can be viewed as strictly objective is still open to discussion. In this paper, by extending a previously proposed objective approach based on scoring rules for the one dimensional case, we propose a novel objective prior for multidimensional parameter spaces which yields a dependence structure. The proposed prior has the appealing property of being proper and does not depend on the chosen model; only on the parameter space considered.
△ Less
Submitted 6 February, 2023;
originally announced February 2023.
-
Optimal Prior Pooling from Expert Opinions
Authors:
A. Kume,
C. Villa,
S. G. Walker
Abstract:
The pooling of prior opinions is an important area of research and has been for a number of decades. The idea is to obtain a single belief probability distribution from a set of expert opinion belief distributions. The paper proposes a new way to provide a resultant prior opinion based on a minimization of information principle. This is done in the square-root density space, which is identified wi…
▽ More
The pooling of prior opinions is an important area of research and has been for a number of decades. The idea is to obtain a single belief probability distribution from a set of expert opinion belief distributions. The paper proposes a new way to provide a resultant prior opinion based on a minimization of information principle. This is done in the square-root density space, which is identified with the positive orthant of Hilbert unit sphere of differentiable functions. It can be shown that the optimal prior is easily identified as an extrinsic mean in the sphere. For distributions belonging to the exponential family, the necessary calculations are exact, and so can be directly applied. The idea can also be adopted for any neighbourhood of a chosen base prior and spanned by a finite set of ``contaminating" directions.
△ Less
Submitted 25 July, 2022;
originally announced July 2022.
-
An Objective Prior from a Scoring Rule
Authors:
Stephen G. Walker,
Cristiano Villa
Abstract:
In this paper we introduce a novel objective prior distribution levering on the connections between information, divergence and scoring rules. In particular, we do so from the starting point of convex functions representing information in density functions. This provides a natural route to proper local scoring rules using Bregman divergence. Specifically, we determine the prior which solves settin…
▽ More
In this paper we introduce a novel objective prior distribution levering on the connections between information, divergence and scoring rules. In particular, we do so from the starting point of convex functions representing information in density functions. This provides a natural route to proper local scoring rules using Bregman divergence. Specifically, we determine the prior which solves setting the score function to be a constant. While in itself this provides motivation for an objective prior, the prior also minimizes a corresponding information criterion.
△ Less
Submitted 10 May, 2021; v1 submitted 7 May, 2021;
originally announced May 2021.
-
Loss based prior for the degrees of freedom of the Wishart distribution
Authors:
Luca Rossini,
Cristiano Villa,
Sotiris Prevenas,
Rachel McCrea
Abstract:
Motivated by the proliferation of extensive macroeconomic and health datasets necessitating accurate forecasts, a novel approach is introduced to address Vector Autoregressive (VAR) models. This approach employs the global-local shrinkage-Wishart prior. Unlike conventional VAR models, where degrees of freedom are predetermined to be equivalent to the size of the variable plus one or equal to zero,…
▽ More
Motivated by the proliferation of extensive macroeconomic and health datasets necessitating accurate forecasts, a novel approach is introduced to address Vector Autoregressive (VAR) models. This approach employs the global-local shrinkage-Wishart prior. Unlike conventional VAR models, where degrees of freedom are predetermined to be equivalent to the size of the variable plus one or equal to zero, the proposed method integrates a hyperprior for the degrees of freedom to account for the uncertainty about the parameter values. Specifically, a loss-based prior is derived to leverage information regarding the data-inherent degrees of freedom. The efficacy of the proposed prior is demonstrated in a multivariate setting for forecasting macroeconomic data, as well as Dengue infection data.
△ Less
Submitted 5 March, 2024; v1 submitted 23 March, 2021;
originally announced March 2021.
-
Beta-CoRM: A Bayesian Approach for $n$-gram Profiles Analysis
Authors:
José A. Perusquía,
Jim E. Griffin,
Cristiano Villa
Abstract:
$n…
▽ More
$n$-gram profiles have been successfully and widely used where long sequences of potentially differing lengths are analysed for clustering or classification. Mostly, machine learning algorithms have been used for this purpose but, despite their superb predictive performance, these methods cannot discover hidden structure or provide a full probabilistic representation of the data. That is why in this paper we centre our attention on a novel class of Bayesian generative models designed for $n$-gram profiles used as binary attributes. The flexibility of our modelling allows us to consider a straightforward approach to feature selection in this generative model. Furthermore, we derive a slice sampling algorithm for a fast inferential procedure which is applied to both synthetic and real data scenarios and shows that feature selection can improve classification accuracy.
△ Less
Submitted 11 October, 2022; v1 submitted 23 November, 2020;
originally announced November 2020.
-
A Loss-Based Prior for Gaussian Graphical Models
Authors:
Laurentiu Catalin Hinoveanu,
Fabrizio Leisen,
Cristiano Villa
Abstract:
Gaussian graphical models play an important role in various areas such as genetics, finance, statistical physics and others. They are a powerful modelling tool which allows one to describe the relationships among the variables of interest. From the Bayesian perspective, there are two sources of randomness: one is related to the multivariate distribution and the quantities that may parametrise the…
▽ More
Gaussian graphical models play an important role in various areas such as genetics, finance, statistical physics and others. They are a powerful modelling tool which allows one to describe the relationships among the variables of interest. From the Bayesian perspective, there are two sources of randomness: one is related to the multivariate distribution and the quantities that may parametrise the model, the other has to do with the underlying graph, $G$, equivalent to describing the conditional independence structure of the model under consideration. In this paper, we propose a prior on G based on two loss components. One considers the loss in information one would incur in selecting the wrong graph, while the second penalises for large number of edges, favouring sparsity. We illustrate the prior on simulated data and on real datasets, and compare the results with other priors on $G$ used in the literature. Moreover, we present a default choice of the prior as well as discuss how it can be calibrated so as to reflect available prior information.
△ Less
Submitted 18 April, 2020; v1 submitted 13 December, 2018;
originally announced December 2018.
-
On a Loss-based prior for the number of components in mixture models
Authors:
Clara Grazian,
Cristiano Villa,
Brunero Liseo
Abstract:
We propose a prior distribution for the number of components of a finite mixture model. The novelty is that the prior distribution is obtained by considering the loss one would incur if the true value representing the number of components were not considered. The prior has an elegant and easy to implement structure, which allows to naturally include any prior information one may have as well as to…
▽ More
We propose a prior distribution for the number of components of a finite mixture model. The novelty is that the prior distribution is obtained by considering the loss one would incur if the true value representing the number of components were not considered. The prior has an elegant and easy to implement structure, which allows to naturally include any prior information one may have as well as to opt for a default solution in cases where this information is not available. The performance of the prior, and comparison with existing alternatives, is studied through the analysis of both real and simulated data.
△ Less
Submitted 4 September, 2018; v1 submitted 20 July, 2018;
originally announced July 2018.
-
Loss-based approach to two-piece location-scale distributions with applications to dependent data
Authors:
Fabrizio Leisen,
Luca Rossini,
Cristiano Villa
Abstract:
Two-piece location-scale models are used for modeling data presenting departures from symmetry. In this paper, we propose an objective Bayesian methodology for the tail parameter of two particular distributions of the above family: the skewed exponential power distribution and the skewed generalised logistic distribution. We apply the proposed objective approach to time series models and linear re…
▽ More
Two-piece location-scale models are used for modeling data presenting departures from symmetry. In this paper, we propose an objective Bayesian methodology for the tail parameter of two particular distributions of the above family: the skewed exponential power distribution and the skewed generalised logistic distribution. We apply the proposed objective approach to time series models and linear regression models where the error terms follow the distributions object of study. The performance of the proposed approach is illustrated through simulation experiments and real data analysis. The methodology yields improvements in density forecasts, as shown by the analysis we carry out on the electricity prices in Nordpool markets.
△ Less
Submitted 28 November, 2018; v1 submitted 14 February, 2018;
originally announced February 2018.
-
On a Class of Objective Priors from Scoring Rules
Authors:
Fabrizio Leisen,
Cristiano Villa,
Stephen G. Walker
Abstract:
Objective prior distributions represent an important tool that allows one to have the advantages of using the Bayesian framework even when information about the parameters of a model is not available. The usual objective approaches work off the chosen statistical model and in the majority of cases the resulting prior is improper, which can pose limitations to a practical implementation, even when…
▽ More
Objective prior distributions represent an important tool that allows one to have the advantages of using the Bayesian framework even when information about the parameters of a model is not available. The usual objective approaches work off the chosen statistical model and in the majority of cases the resulting prior is improper, which can pose limitations to a practical implementation, even when the complexity of the model is moderate. In this paper we propose to take a novel look at the construction of objective prior distributions, where the connection with a chosen sampling distribution model is removed. We explore the notion of defining objective prior distributions which allow one to have some degree of flexibility, in particular in exhibiting some desirable features, such as being proper, or centered on specific values which would be of interest in nested model comparisons. The basic tool we use are proper scoring rules and the main result is a class of objective prior distributions that can be employed in scenarios where the usual model based priors fail, such as mixture models and model selection via Bayes factors. In addition, we show that the proposed class of priors is the result of minimising the information it contains, providing solid interpretation to the method.
△ Less
Submitted 23 September, 2018; v1 submitted 2 June, 2017;
originally announced June 2017.
-
Objective Bayesian Analysis for Change Point Problems
Authors:
Laurentiu Hinoveanu,
Fabrizio Leisen,
Cristiano Villa
Abstract:
In this paper we present a loss-based approach to change point analysis. In particular, we look at the problem from two perspectives. The first focuses on the definition of a prior when the number of change points is known a priori. The second contribution aims to estimate the number of change points by using a loss-based approach recently introduced in the literature. The latter considers change…
▽ More
In this paper we present a loss-based approach to change point analysis. In particular, we look at the problem from two perspectives. The first focuses on the definition of a prior when the number of change points is known a priori. The second contribution aims to estimate the number of change points by using a loss-based approach recently introduced in the literature. The latter considers change point estimation as a model selection exercise. We show the performance of the proposed approach on simulated data and real data sets.
△ Less
Submitted 7 January, 2018; v1 submitted 17 February, 2017;
originally announced February 2017.
-
Objective priors for the number of degrees of freedom of a multivariate t distribution and the t-copula
Authors:
Cristiano Villa,
Francisco J. Rubio
Abstract:
An objective Bayesian approach to estimate the number of degrees of freedom $(ν)$ for the multivariate $t$ distribution and for the $t$-copula, when the parameter is considered discrete, is proposed. Inference on this parameter has been problematic for the multivariate $t$ and, for the absence of any method, for the $t$-copula. An objective criterion based on loss functions which allows to overcom…
▽ More
An objective Bayesian approach to estimate the number of degrees of freedom $(ν)$ for the multivariate $t$ distribution and for the $t$-copula, when the parameter is considered discrete, is proposed. Inference on this parameter has been problematic for the multivariate $t$ and, for the absence of any method, for the $t$-copula. An objective criterion based on loss functions which allows to overcome the issue of defining objective probabilities directly is employed. The support of the prior for $ν$ is truncated, which derives from the property of both the multivariate $t$ and the $t$-copula of convergence to normality for a sufficiently large number of degrees of freedom. The performance of the priors is tested on simulated scenarios. The R codes and the replication material are available as a supplementary material of the electronic version of the paper and on real data: daily logarithmic returns of IBM and of the Center for Research in Security Prices Database.
△ Less
Submitted 13 March, 2018; v1 submitted 19 January, 2017;
originally announced January 2017.
-
Objective Bayesian modelling of insurance risks with the skewed Student-t distribution
Authors:
Fabrizio Leisen,
Juan Miguel Marin,
Cristiano Villa
Abstract:
Insurance risks data typically exhibit skewed behaviour. In this paper, we propose a Bayesian approach to capture the main features of these datasets. This work extends the methodology introduced in Villa and Walker (2014a) by considering an extra parameter which captures the skewness of the data. In particular, a skewed Student-t distribution is considered. Two datasets are analysed: the Danish f…
▽ More
Insurance risks data typically exhibit skewed behaviour. In this paper, we propose a Bayesian approach to capture the main features of these datasets. This work extends the methodology introduced in Villa and Walker (2014a) by considering an extra parameter which captures the skewness of the data. In particular, a skewed Student-t distribution is considered. Two datasets are analysed: the Danish fire losses and the US indemnity loss. The analysis is carried with an objective Bayesian approach. For the discrete parameter representing the number of the degrees of freedom, we adopt a novel prior recently introduced in Villa and Walker (2014b).
△ Less
Submitted 16 July, 2016;
originally announced July 2016.
-
A Note on the Posterior Inference for the Yule-Simon Distribution
Authors:
Fabrizio Leisen,
Luca Rossini,
Cristiano Villa
Abstract:
The Yule--Simon distribution has been out of the radar of the Bayesian community, so far. In this note, we propose an explicit Gibbs sampling scheme when a Gamma prior is chosen for the shape parameter. The performance of the algorithm is illustrated with simulation studies, including count data regression, and a real data application to text analysis. We compare our proposal to the frequentist co…
▽ More
The Yule--Simon distribution has been out of the radar of the Bayesian community, so far. In this note, we propose an explicit Gibbs sampling scheme when a Gamma prior is chosen for the shape parameter. The performance of the algorithm is illustrated with simulation studies, including count data regression, and a real data application to text analysis. We compare our proposal to the frequentist counterparts showing better performance of our algorithm when a small sample size is considered.
△ Less
Submitted 31 October, 2016; v1 submitted 25 April, 2016;
originally announced April 2016.
-
Objective Bayesian Analysis of the Yule-Simon Distribution with Applications
Authors:
Fabrizio Leisen,
Luca Rossini,
Cristiano Villa
Abstract:
The Yule-Simon distribution is usually employed in the analysis of frequency data. As the Bayesian literature, so far, ignored this distribution, here we show the derivation of two objective priors for the parameter of the Yule-Simon distribution. In particular, we discuss the Jeffreys prior and a loss-based prior, which has recently appeared in the literature. We illustrate the performance of the…
▽ More
The Yule-Simon distribution is usually employed in the analysis of frequency data. As the Bayesian literature, so far, ignored this distribution, here we show the derivation of two objective priors for the parameter of the Yule-Simon distribution. In particular, we discuss the Jeffreys prior and a loss-based prior, which has recently appeared in the literature. We illustrate the performance of the derived priors through a simulation study and the analysis of real datasets.
△ Less
Submitted 19 April, 2016;
originally announced April 2016.
-
A Property of the Kullback--Leibler Divergence for Location-scale Models
Authors:
Cristiano Villa
Abstract:
In this paper, we discuss a property of the Kullback--Leibler divergence measured between two models of the family of the location-scale distributions. We show that, if model $M_1$ and model $M_2$ are represented by location-scale distributions, then the minimum Kullback--Leibler divergence from $M_1$ to $M_2$, with respect to the parameters of $M_2$, is independent from the value of the parameter…
▽ More
In this paper, we discuss a property of the Kullback--Leibler divergence measured between two models of the family of the location-scale distributions. We show that, if model $M_1$ and model $M_2$ are represented by location-scale distributions, then the minimum Kullback--Leibler divergence from $M_1$ to $M_2$, with respect to the parameters of $M_2$, is independent from the value of the parameters of $M_1$. Furthermore, we show that the property holds for models that can be transformed into location-scale distributions. We illustrate a possible application of the property in objective Bayesian model selection.
△ Less
Submitted 7 April, 2016;
originally announced April 2016.
-
Bayesian Estimation of the Threshold of a Generalised Pareto Distribution for Heavy-Tailed Observations
Authors:
Cristiano Villa
Abstract:
In this paper, we discuss a method to define prior distributions for the threshold of a generalised Pareto distribution, in particular when its applications are directed to heavy-tailed data. We propose to assign prior probabilities to the order statistics of a given set of observations. In other words, we assume that the threshold coincides to one of the data points. We show two ways of defining…
▽ More
In this paper, we discuss a method to define prior distributions for the threshold of a generalised Pareto distribution, in particular when its applications are directed to heavy-tailed data. We propose to assign prior probabilities to the order statistics of a given set of observations. In other words, we assume that the threshold coincides to one of the data points. We show two ways of defining a prior: by assigning equal mass to each order statistic, that is a uniform prior, and by considering the worth that every order statistic has in representing the true threshold. Both proposed priors represent a scenario of minimal information, and we study their adequacy through simulation exercises and by analysing two applications from insurance and from finance.
△ Less
Submitted 5 April, 2016;
originally announced April 2016.
-
Model Prior Distribution for Variable Selection in Linear Regression Models
Authors:
Cristiano Villa,
Jeong Eun Lee
Abstract:
In this work we discuss a novel model prior probability for variable selection in linear regression. The idea is to determine the prior mass in an objective sense, by considering the worth of each of the possible regression models, given the number of covariates under consideration. Through a simulation study, we show that the proposed prior outperforms the uniform prior and the Scott \& Berger pr…
▽ More
In this work we discuss a novel model prior probability for variable selection in linear regression. The idea is to determine the prior mass in an objective sense, by considering the worth of each of the possible regression models, given the number of covariates under consideration. Through a simulation study, we show that the proposed prior outperforms the uniform prior and the Scott \& Berger prior in a scenario of no prior knowledge about the size of the true regression models. We illustrate the use of the prior using two well-known data sets with, respectively, 15 and 4 covariates.
△ Less
Submitted 26 December, 2015;
originally announced December 2015.
