Showing 1–50 of 102 results for author: Rue, H

inlabru: software for fitting latent Gaussian models with non-linear predictors

Authors: Finn Lindgren, Fabian Bachl, Janine Illian, Man Ho Suen, Håvard Rue, Andrew E. Seaton

Abstract: The integrated nested Laplace approximation (INLA) method has become a popular approach for computationally efficient approximate Bayesian computation. In particular, by leveraging sparsity in random effect precision matrices, INLA is commonly used in spatial and spatio-temporal applications. However, the speed of INLA comes at the cost of restricting the user to the family of latent Gaussian mode… ▽ More The integrated nested Laplace approximation (INLA) method has become a popular approach for computationally efficient approximate Bayesian computation. In particular, by leveraging sparsity in random effect precision matrices, INLA is commonly used in spatial and spatio-temporal applications. However, the speed of INLA comes at the cost of restricting the user to the family of latent Gaussian models and the likelihoods currently implemented in {INLA}, the main software implementation of the INLA methodology. {inlabru} is a software package that extends the types of models that can be fitted using INLA by allowing the latent predictor to be non-linear in its parameters, moving beyond the additive linear predictor framework to allow more complex functional relationships. For inference it uses an approximate iterative method based on the first-order Taylor expansion of the non-linear predictor, fitting the model using INLA for each linearised model configuration. {inlabru} automates much of the workflow required to fit models using {R-INLA}, simplifying the process for users to specify, fit and predict from models. There is additional support for fitting joint likelihood models by building each likelihood individually. {inlabru} also supports the direct use of spatial data structures, such as those implemented in the {sf} and {terra} packages. In this paper we outline the statistical theory, model structure and basic syntax required for users to understand and develop their own models using {inlabru}. We evaluate the approximate inference method using a Bayesian method checking approach. We provide three examples modelling simulated spatial data that demonstrate the benefits of the additional flexibility provided by {inlabru}. △ Less

Submitted 30 June, 2024; originally announced July 2024.

MSC Class: 62-04

Spatio-temporal Occupancy Models with INLA

Authors: Jafet Belmont, Sara Martino, Janine Illian, Håvard Rue

Abstract: Modern methods for quantifying and predicting species distribution play a crucial part in biodiversity conservation. Occupancy models are a popular choice for analyzing species occurrence data as they allow to separate the observational error induced by imperfect detection, and the sources of bias affecting the occupancy process. However, the spatial and temporal variation in occupancy not account… ▽ More Modern methods for quantifying and predicting species distribution play a crucial part in biodiversity conservation. Occupancy models are a popular choice for analyzing species occurrence data as they allow to separate the observational error induced by imperfect detection, and the sources of bias affecting the occupancy process. However, the spatial and temporal variation in occupancy not accounted for by environmental covariates is often ignored or modelled through simple spatial structures as the computational costs of fitting explicit spatio-temporal models is too high. In this work, we demonstrate how INLA may be used to fit complex occupancy models and how the R-INLA package can provide a user-friendly interface to make such complex models available to users. We show how occupancy models, provided some simplification on the detection process, can be framed as latent Gaussian models and benefit from the powerful INLA machinery. A large selection of complex modelling features, and random effect modelshave already been implemented in R-INLA. These become available for occupancy models, providing the user with an efficient and flexible toolbox. We illustrate how INLA provides a computationally efficient framework for develo** and fitting complex occupancy models using two case studies. Through these, we show how different spatio-temporal models that include spatial-varying trends, smooth terms, and spatio-temporal random effects can be fitted. At the cost of limiting the complexity of the detection model, INLA can incorporate a range of complex structures in the process. INLA-based occupancy models provide an alternative framework to fit complex spatiotemporal occupancy models. The need for new and more flexible computationally approaches to fit such models makes INLA an attractive option for addressing complex ecological problems, and a promising area of research. △ Less

Submitted 15 March, 2024; originally announced March 2024.

Joint Modeling of Multivariate Longitudinal and Survival Outcomes with the R package INLAjoint

Authors: Denis Rustand, Janet van Niekerk, Elias Teixeira Krainski, Håvard Rue

Abstract: This paper introduces the R package INLAjoint, designed as a toolbox for fitting a diverse range of regression models addressing both longitudinal and survival outcomes. INLAjoint relies on the computational efficiency of the integrated nested Laplace approximations methodology, an efficient alternative to Markov chain Monte Carlo for Bayesian inference, ensuring both speed and accuracy in paramet… ▽ More This paper introduces the R package INLAjoint, designed as a toolbox for fitting a diverse range of regression models addressing both longitudinal and survival outcomes. INLAjoint relies on the computational efficiency of the integrated nested Laplace approximations methodology, an efficient alternative to Markov chain Monte Carlo for Bayesian inference, ensuring both speed and accuracy in parameter estimation and uncertainty quantification. The package facilitates the construction of complex joint models by treating individual regression models as building blocks, which can be assembled to address specific research questions. Joint models are relevant in biomedical studies where the collection of longitudinal markers alongside censored survival times is common. They have gained significant interest in recent literature, demonstrating the ability to rectify biases present in separate modeling approaches such as informative censoring by a survival event or confusion bias due to population heterogeneity. We provide a comprehensive overview of the joint modeling framework embedded in INLAjoint with illustrative examples. Through these examples, we demonstrate the practical utility of INLAjoint in handling complex data scenarios encountered in biomedical research. △ Less

Submitted 3 April, 2024; v1 submitted 13 February, 2024; originally announced February 2024.

A graphical framework for interpretable correlation matrix models

Authors: Anna Freni Sterrantino, Denis Rustand, Janet van Niekerk, Elias Teixeira Krainski, Håvard Rue

Abstract: In this work, we present a new approach for constructing models for correlation matrices with a user-defined graphical structure. The graphical structure makes correlation matrices interpretable and avoids the quadratic increase of parameters as a function of the dimension. We suggest an automatic approach to define a prior using a natural sequence of simpler models within the Penalized Complexity… ▽ More In this work, we present a new approach for constructing models for correlation matrices with a user-defined graphical structure. The graphical structure makes correlation matrices interpretable and avoids the quadratic increase of parameters as a function of the dimension. We suggest an automatic approach to define a prior using a natural sequence of simpler models within the Penalized Complexity framework for the unknown parameters in these models. We illustrate this approach with three applications: a multivariate linear regression of four biomarkers, a multivariate disease map**, and a multivariate longitudinal joint modelling. Each application underscores our method's intuitive appeal, signifying a substantial advancement toward a more cohesive and enlightening model that facilitates a meaningful interpretation of correlation matrices. △ Less

Submitted 11 December, 2023; originally announced December 2023.

Enhanced spatial modeling on linear networks using Gaussian Whittle-Matérn fields

Authors: Somnath Chaudhuri, Maria A. Barceló, Pablo Juan, Diego Varga, David Bolin, Haavard Rue, Marc Saez

Abstract: Spatial statistics is traditionally based on stationary models on $\mathbb{R^d}$ like Matérn fields. The adaptation of traditional spatial statistical methods, originally designed for stationary models in Euclidean spaces, to effectively model phenomena on linear networks such as stream systems and urban road networks is challenging. The current study aims to analyze the incidence of traffic accid… ▽ More Spatial statistics is traditionally based on stationary models on $\mathbb{R^d}$ like Matérn fields. The adaptation of traditional spatial statistical methods, originally designed for stationary models in Euclidean spaces, to effectively model phenomena on linear networks such as stream systems and urban road networks is challenging. The current study aims to analyze the incidence of traffic accidents on road networks using three different methodologies and compare the model performance for each methodology. Initially, we analyzed the application of spatial triangulation precisely on road networks instead of traditional continuous regions. However, this approach posed challenges in areas with complex boundaries, leading to the emergence of artificial spatial dependencies. To address this, we applied an alternative computational method to construct nonstationary barrier models. Finally, we explored a recently proposed class of Gaussian processes on compact metric graphs, the Whittle-Matérn fields, defined by a fractional SPDE on the metric graph. The latter fields are a natural extension of Gaussian fields with Matérn covariance functions on Euclidean domains to non-Euclidean metric graph settings. A ten-year period (2010-2019) of daily traffic-accident records from Barcelona, Spain have been used to evaluate the three models referred above. While comparing model performance we observed that the Whittle-Matérn fields defined directly on the network outperformed the network triangulation and barrier models. Due to their flexibility, the Whittle-Matérn fields can be applied to a wide range of environmental problems on linear networks such as spatio-temporal modeling of water contamination in stream networks or modeling air quality or accidents on urban road networks. △ Less

Submitted 9 December, 2023; v1 submitted 2 December, 2023; originally announced December 2023.

Comments: 24 pages, 9 figures

Automatic cross-validation in structured models: Is it time to leave out leave-one-out?

Authors: A. Adin, E. Krainski, A. Lenzi, Z. Liu, J. Martínez-Minaya, H. Rue

Abstract: Standard techniques such as leave-one-out cross-validation (LOOCV) might not be suitable for evaluating the predictive performance of models incorporating structured random effects. In such cases, the correlation between the training and test sets could have a notable impact on the model's prediction error. To overcome this issue, an automatic group construction procedure for leave-group-out cross… ▽ More Standard techniques such as leave-one-out cross-validation (LOOCV) might not be suitable for evaluating the predictive performance of models incorporating structured random effects. In such cases, the correlation between the training and test sets could have a notable impact on the model's prediction error. To overcome this issue, an automatic group construction procedure for leave-group-out cross validation (LGOCV) has recently emerged as a valuable tool for enhancing predictive performance measurement in structured models. The purpose of this paper is (i) to compare LOOCV and LGOCV within structured models, emphasizing model selection and predictive performance, and (ii) to provide real data applications in spatial statistics using complex structured models fitted with INLA, showcasing the utility of the automatic LGOCV method. First, we briefly review the key aspects of the recently proposed LGOCV method for automatic group construction in latent Gaussian models. We also demonstrate the effectiveness of this method for selecting the model with the highest predictive performance by simulating extrapolation tasks in both temporal and spatial data analyses. Finally, we provide insights into the effectiveness of the LGOCV method in modelling complex structured data, encompassing spatio-temporal multivariate count data, spatial compositional data, and spatio-temporal geospatial data. △ Less

Submitted 7 March, 2024; v1 submitted 28 November, 2023; originally announced November 2023.

Journal ref: Spatial Statistics (2024)

INLA+ -- Approximate Bayesian inference for non-sparse models using HPC

Authors: Esmail Abdul-Fattah, Janet Van Niekerk, Haavard Rue

Abstract: The integrated nested Laplace approximations (INLA) method has become a widely utilized tool for researchers and practitioners seeking to perform approximate Bayesian inference across various fields of application. To address the growing demand for incorporating more complex models and enhancing the method's capabilities, this paper introduces a novel framework that leverages dense matrices for pe… ▽ More The integrated nested Laplace approximations (INLA) method has become a widely utilized tool for researchers and practitioners seeking to perform approximate Bayesian inference across various fields of application. To address the growing demand for incorporating more complex models and enhancing the method's capabilities, this paper introduces a novel framework that leverages dense matrices for performing approximate Bayesian inference based on INLA across multiple computing nodes using HPC. When dealing with non-sparse precision or covariance matrices, this new approach scales better compared to the current INLA method, capitalizing on the computational power offered by multiprocessors in shared and distributed memory architectures available in contemporary computing resources and specialized dense matrix algebra. To validate the efficacy of this approach, we conduct a simulation study then apply it to analyze cancer mortality data in Spain, employing a three-way spatio-temporal interaction model. △ Less

Submitted 14 November, 2023; originally announced November 2023.

Comments: 29 pages, 13 figures

Statistical inference for radially-stable generalized Pareto distributions and return level-sets in geometric extremes

Authors: Ioannis Papastathopoulos, Lambert de Monte, Ryan Campbell, Haavard Rue

Abstract: We use a functional analogue of the quantile function for probability measures admitting a continuous Lebesgue density on $\mathbb{R}^d$ to characterise the class of non-trivial limit distributions of radially recentered and rescaled multivariate exceedances. A new class of multivariate distributions is identified, termed radially-stable generalised Pareto distributions, and is shown to admit cert… ▽ More We use a functional analogue of the quantile function for probability measures admitting a continuous Lebesgue density on $\mathbb{R}^d$ to characterise the class of non-trivial limit distributions of radially recentered and rescaled multivariate exceedances. A new class of multivariate distributions is identified, termed radially-stable generalised Pareto distributions, and is shown to admit certain stability properties that permit extrapolation to extremal sets along any direction in cones such as $\mathbb{R}^d$ and $\mathbb{R}_+^d$. Leveraging the limit Poisson point process likelihood of the point process of radially renormalised exceedances, we develop parsimonious statistical models that exploit theoretical links between structural star-bodies and are amenable to Bayesian inference. Our framework sharpens statistical inference by suitably including additional information from the angular directions of the geometric exceedances and facilitates efficient computations in dimensions $d=2$ and $d=3$. Additionally, it naturally leads to the notion of return level-set, which is a canonical quantile set expressed in terms of its average recurrence interval, and a geometric analogue of the uni-dimensional return level. We illustrate our methods with a simulation study showing superior predictive performance of probabilities of rare events, and with two case studies, one associated with river flow extremes, and the other with oceanographic extremes. △ Less

Submitted 23 January, 2024; v1 submitted 9 October, 2023; originally announced October 2023.

Comments: 80 pages, 32 figures

Parallel Selected Inversion for Space-Time Gaussian Markov Random Fields

Authors: Abylay Zhumekenov, Elias T. Krainski, Håvard Rue

Abstract: Performing a Bayesian inference on large spatio-temporal models requires extracting inverse elements of large sparse precision matrices for marginal variances. Although direct matrix factorizations can be used for the inversion, such methods fail to scale well for distributed problems when run on large computing clusters. On the contrary, Krylov subspace methods for the selected inversion have bee… ▽ More Performing a Bayesian inference on large spatio-temporal models requires extracting inverse elements of large sparse precision matrices for marginal variances. Although direct matrix factorizations can be used for the inversion, such methods fail to scale well for distributed problems when run on large computing clusters. On the contrary, Krylov subspace methods for the selected inversion have been gaining traction. We propose a parallel hybrid approach based on domain decomposition, which extends the Rao-Blackwellized Monte Carlo estimator for distributed precision matrices. Our approach exploits the strength of Krylov subspace methods as global solvers and efficiency of direct factorizations as base case solvers to compute the marginal variances using a divide-and-conquer strategy. By introducing subdomain overlaps, one can achieve a greater accuracy at an increased computational effort with little to no additional communication. We demonstrate the speed improvements on both simulated models and a massive US daily temperature data. △ Less

Submitted 11 September, 2023; originally announced September 2023.

Comments: 17 pages, 7 figures

A flexible Bayesian tool for CoDa mixed models: logistic-normal distribution with Dirichlet covariance

Authors: Joaquín Martínez-Minaya, Haavard Rue

Abstract: Compositional Data Analysis (CoDa) has gained popularity in recent years. This type of data consists of values from disjoint categories that sum up to a constant. Both Dirichlet regression and logistic-normal regression have become popular as CoDa analysis methods. However, fitting this kind of multivariate models presents challenges, especially when structured random effects are included in the m… ▽ More Compositional Data Analysis (CoDa) has gained popularity in recent years. This type of data consists of values from disjoint categories that sum up to a constant. Both Dirichlet regression and logistic-normal regression have become popular as CoDa analysis methods. However, fitting this kind of multivariate models presents challenges, especially when structured random effects are included in the model, such as temporal or spatial effects. To overcome these challenges, we propose the logistic-normal Dirichlet Model (LNDM). We seamlessly incorporate this approach into the R-INLA package, facilitating model fitting and model prediction within the framework of Latent Gaussian Models (LGMs). Moreover, we explore metrics like Deviance Information Criteria (DIC), Watanabe Akaike information criterion (WAIC), and cross-validation measure conditional predictive ordinate (CPO) for model selection in R-INLA for CoDa. Illustrating LNDM through a simple simulated example and with an ecological case study on Arabidopsis thaliana in the Iberian Peninsula, we underscore its potential as an effective tool for managing CoDa and large CoDa databases. △ Less

Submitted 8 November, 2023; v1 submitted 26 August, 2023; originally announced August 2023.

Journal ref: Statistics and Compunting (2024)

Robustness, model checking and latent Gaussian models

Authors: Rafael Cabral, David Bolin, Håvard Rue

Abstract: Model checking is essential to evaluate the adequacy of statistical models and the validity of inferences drawn from them. Particularly, hierarchical models such as latent Gaussian models (LGMs) pose unique challenges as it is difficult to check assumptions about the distribution of the latent parameters. Discrepancy measures are often used to quantify the degree to which a model fit deviates from… ▽ More Model checking is essential to evaluate the adequacy of statistical models and the validity of inferences drawn from them. Particularly, hierarchical models such as latent Gaussian models (LGMs) pose unique challenges as it is difficult to check assumptions about the distribution of the latent parameters. Discrepancy measures are often used to quantify the degree to which a model fit deviates from the observed data. We construct discrepancy measures by (a) defining an alternative model with relaxed assumptions and (b) deriving the discrepancy measure most sensitive to discrepancies induced by this alternative model. We also promote a workflow for model criticism that combines model checking with subsequent robustness analysis. As a result, we obtain a general recipe to check assumptions in LGMs and the impact of these assumptions on the results. We demonstrate the ideas by assessing the latent Gaussianity assumption, a crucial but often overlooked assumption in LGMs. We illustrate the methods via examples utilising Stan and provide functions for easy usage of the methods for general models fitted through R-INLA. △ Less

Submitted 23 July, 2023; originally announced July 2023.

Comments: 40 pages, 21 figures

MSC Class: 62A01; 62C10; 62F03; 62F35

Non-stationary Bayesian Spatial Model for Disease Map** based on Sub-regions

Authors: Esmail Abdul Fattah, Elias Krainski, Janet van Niekerk, Håvard Rue

Abstract: This paper aims to extend the Besag model, a widely used Bayesian spatial model in disease map**, to a non-stationary spatial model for irregular lattice-type data. The goal is to improve the model's ability to capture complex spatial dependence patterns and increase interpretability. The proposed model uses multiple precision parameters, accounting for different intensities of spatial dependenc… ▽ More This paper aims to extend the Besag model, a widely used Bayesian spatial model in disease map**, to a non-stationary spatial model for irregular lattice-type data. The goal is to improve the model's ability to capture complex spatial dependence patterns and increase interpretability. The proposed model uses multiple precision parameters, accounting for different intensities of spatial dependence in different sub-regions. We derive a joint penalized complexity prior for the flexible local precision parameters to prevent overfitting and ensure contraction to the stationary model at a user-defined rate. The proposed methodology can be used as a basis for the development of various other non-stationary effects over other domains such as time. An accompanying R package 'fbesag' equips the reader with the necessary tools for immediate use and application. We illustrate the novelty of the proposal by modeling the risk of dengue in Brazil, where the stationary spatial assumption fails and interesting risk profiles are estimated when accounting for spatial non-stationary. △ Less

Submitted 29 June, 2023; originally announced June 2023.

Integrated Nested Laplace Approximations for Large-Scale Spatial-Temporal Bayesian Modeling

Authors: Lisa Gaedke-Merzhäuser, Elias Krainski, Radim Janalik, Håvard Rue, Olaf Schenk

Abstract: Bayesian inference tasks continue to pose a computational challenge. This especially holds for spatial-temporal modeling where high-dimensional latent parameter spaces are ubiquitous. The methodology of integrated nested Laplace approximations (INLA) provides a framework for performing Bayesian inference applicable to a large subclass of additive Bayesian hierarchical models. In combination with t… ▽ More Bayesian inference tasks continue to pose a computational challenge. This especially holds for spatial-temporal modeling where high-dimensional latent parameter spaces are ubiquitous. The methodology of integrated nested Laplace approximations (INLA) provides a framework for performing Bayesian inference applicable to a large subclass of additive Bayesian hierarchical models. In combination with the stochastic partial differential equations (SPDE) approach it gives rise to an efficient method for spatial-temporal modeling. In this work we build on the INLA-SPDE approach, by putting forward a performant distributed memory variant, INLA-DIST, for large-scale applications. To perform the arising computational kernel operations, consisting of Cholesky factorizations, solving linear systems, and selected matrix inversions, we present two numerical solver options, a sparse CPU-based library and a novel blocked GPU-accelerated approach which we propose. We leverage the recurring nonzero block structure in the arising precision (inverse covariance) matrices, which allows us to employ dense subroutines within a sparse setting. Both versions of INLA-DIST are highly scalable, capable of performing inference on models with millions of latent parameters. We demonstrate their accuracy and performance on synthetic as well as real-world climate dataset applications. △ Less

Submitted 27 March, 2023; originally announced March 2023.

Comments: 22 pages, 14 figures

Towards black-box parameter estimation

Authors: Amanda Lenzi, Haavard Rue

Abstract: Deep learning algorithms have recently shown to be a successful tool in estimating parameters of statistical models for which simulation is easy, but likelihood computation is challenging. But the success of these approaches depends on simulating parameters that sufficiently reproduce the observed data, and, at present, there is a lack of efficient methods to produce these simulations. We develop… ▽ More Deep learning algorithms have recently shown to be a successful tool in estimating parameters of statistical models for which simulation is easy, but likelihood computation is challenging. But the success of these approaches depends on simulating parameters that sufficiently reproduce the observed data, and, at present, there is a lack of efficient methods to produce these simulations. We develop new black-box procedures to estimate parameters of statistical models based only on weak parameter structure assumptions. For well-structured likelihoods with frequent occurrences, such as in time series, this is achieved by pre-training a deep neural network on an extensive simulated database that covers a wide range of data sizes. For other types of complex dependencies, an iterative algorithm guides simulations to the correct parameter region in multiple rounds. These approaches can successfully estimate and quantify the uncertainty of parameters from non-Gaussian models with complex spatial and temporal dependencies. The success of our methods is a first step towards a fully flexible automatic black-box estimation framework. △ Less

Submitted 19 February, 2024; v1 submitted 27 March, 2023; originally announced March 2023.

Bayesian Inference for Multivariate Spatial Models with R-INLA

Authors: Francisco Palmí-Perales, Virgilio Gómez-Rubio, Roger S Bivand, Michela Cameletti, Håvard Rue

Abstract: Bayesian methods and software for spatial data analysis are generally now well established in the scientific community. Despite the wide application of spatial models, the analysis of multivariate spatial data using R-INLA has not been widely described in the existing literature. Therefore, the main objective of this article is to demonstrate that R-INLA is a convenient toolbox to analyse differen… ▽ More Bayesian methods and software for spatial data analysis are generally now well established in the scientific community. Despite the wide application of spatial models, the analysis of multivariate spatial data using R-INLA has not been widely described in the existing literature. Therefore, the main objective of this article is to demonstrate that R-INLA is a convenient toolbox to analyse different types of multivariate spatial datasets. Additionally, this will be illustrated by analysing three datasets which are publicly available. Furthermore, the details and the R code of these analyses are provided to exemplify how to adjust multivariate spatial datasets with R-INLA. △ Less

Submitted 21 December, 2022; originally announced December 2022.

Comments: Submitted to the RJournal (19 pages and 6 figures)

Bayesian survival analysis with INLA

Authors: Danilo Alvares, Janet van Niekerk, Elias Teixeira Krainski, Håvard Rue, Denis Rustand

Abstract: This tutorial shows how various Bayesian survival models can be fitted using the integrated nested Laplace approximation in a clear, legible, and comprehensible manner using the INLA and INLAjoint R-packages. Such models include accelerated failure time, proportional hazards, mixture cure, competing risks, multi-state, frailty, and joint models of longitudinal and survival data, originally present… ▽ More This tutorial shows how various Bayesian survival models can be fitted using the integrated nested Laplace approximation in a clear, legible, and comprehensible manner using the INLA and INLAjoint R-packages. Such models include accelerated failure time, proportional hazards, mixture cure, competing risks, multi-state, frailty, and joint models of longitudinal and survival data, originally presented in the article "Bayesian survival analysis with BUGS" (Alvares et al., 2021). In addition, we illustrate the implementation of a new joint model for a longitudinal semicontinuous marker, recurrent events, and a terminal event. Our proposal aims to provide the reader with syntax examples for implementing survival models using a fast and accurate approximate Bayesian inferential approach. △ Less

Submitted 18 March, 2024; v1 submitted 4 December, 2022; originally announced December 2022.

Fitting latent non-Gaussian models using variational Bayes and Laplace approximations

Authors: Rafael Cabral, David Bolin, Håvard Rue

Abstract: Latent Gaussian models (LGMs) are perhaps the most commonly used class of models in statistical applications. Nevertheless, in areas ranging from longitudinal studies in biostatistics to geostatistics, it is easy to find datasets that contain inherently non-Gaussian features, such as sudden jumps or spikes, that adversely affect the inferences and predictions made from an LGM. These datasets requi… ▽ More Latent Gaussian models (LGMs) are perhaps the most commonly used class of models in statistical applications. Nevertheless, in areas ranging from longitudinal studies in biostatistics to geostatistics, it is easy to find datasets that contain inherently non-Gaussian features, such as sudden jumps or spikes, that adversely affect the inferences and predictions made from an LGM. These datasets require more general latent non-Gaussian models (LnGMs) that can handle these non-Gaussian features automatically. However, fast implementation and easy-to-use software are lacking, which prevent LnGMs from becoming widely applicable. In this paper, we derive variational Bayes algorithms for fast and scalable inference of LnGMs. The approximation leads to an LGM that downweights extreme events in the latent process, reducing their impact and leading to more robust inferences. It can be applied to a wide range of models, such as autoregressive processes for time series, simultaneous autoregressive models for areal data, and spatial Matérn models. To facilitate Bayesian inference, we introduce the ngvb package, where LGMs implemented in R-INLA can be easily extended to LnGMs by adding a single line of code. △ Less

Submitted 20 November, 2022; originally announced November 2022.

Comments: 30 pages, 10 figures

MSC Class: 62M20; 62F15; 62F35

Leave-group-out cross-validation for latent Gaussian models

Authors: Zhedong Liu, Haavard Rue

Abstract: Evaluating the predictive performance of a statistical model is commonly done using cross-validation. Although the leave-one-out method is frequently employed, its application is justified primarily for independent and identically distributed observations. However, this method tends to mimic interpolation rather than prediction when dealing with dependent observations. This paper proposes a modifi… ▽ More Evaluating the predictive performance of a statistical model is commonly done using cross-validation. Although the leave-one-out method is frequently employed, its application is justified primarily for independent and identically distributed observations. However, this method tends to mimic interpolation rather than prediction when dealing with dependent observations. This paper proposes a modified cross-validation for dependent observations. This is achieved by excluding an automatically determined set of observations from the training set to mimic a more reasonable prediction scenario. Also, within the framework of latent Gaussian models, we illustrate a method to adjust the joint posterior for this modified cross-validation to avoid model refitting. This new approach is accessible in the R-INLA package (www.r-inla.org). △ Less

Submitted 12 October, 2023; v1 submitted 10 October, 2022; originally announced October 2022.

Approximate Bayesian Inference for the Interaction Types 1, 2, 3 and 4 with Application in Disease Map**

Authors: Esmail Abdul Fattah, Haavard Rue

Abstract: We address in this paper a new approach for fitting spatiotemporal models with application in disease map** using the interaction types 1,2,3, and 4. When we account for the spatiotemporal interactions in disease-map** models, inference becomes more useful in revealing unknown patterns in the data. However, when the number of locations and/or the number of time points is large, the inference g… ▽ More We address in this paper a new approach for fitting spatiotemporal models with application in disease map** using the interaction types 1,2,3, and 4. When we account for the spatiotemporal interactions in disease-map** models, inference becomes more useful in revealing unknown patterns in the data. However, when the number of locations and/or the number of time points is large, the inference gets computationally challenging due to the high number of required constraints necessary for inference, and this holds for various inference architectures including Markov chain Monte Carlo (MCMC) and Integrated Nested Laplace Approximations (INLA). We re-formulate INLA approach based on dense matrices to fit the intrinsic spatiotemporal models with the four interaction types and account for the sum-to-zero constraints, and discuss how the new approach can be implemented in a high-performance computing framework. The computing time using the new approach does not depend on the number of constraints and can reach a 40-fold faster speed compared to INLA in realistic scenarios. This approach is verified by a simulation study and a real data application, and it is implemented in the R package INLAPLUS and the Python header function: inla1234(). △ Less

Submitted 18 June, 2022; originally announced June 2022.

MSC Class: 62H11 ACM Class: G.3.14

A new avenue for Bayesian inference with INLA

Authors: Janet van Niekerk, Elias Krainski, Denis Rustand, Haavard Rue

Abstract: Integrated Nested Laplace Approximations (INLA) has been a successful approximate Bayesian inference framework since its proposal by Rue et al. (2009). The increased computational efficiency and accuracy when compared with sampling-based methods for Bayesian inference like MCMC methods, are some contributors to its success. Ongoing research in the INLA methodology and implementation thereof in the… ▽ More Integrated Nested Laplace Approximations (INLA) has been a successful approximate Bayesian inference framework since its proposal by Rue et al. (2009). The increased computational efficiency and accuracy when compared with sampling-based methods for Bayesian inference like MCMC methods, are some contributors to its success. Ongoing research in the INLA methodology and implementation thereof in the R package R-INLA, ensures continued relevance for practitioners and improved performance and applicability of INLA. The era of big data and some recent research developments, presents an opportunity to reformulate some aspects of the classic INLA formulation, to achieve even faster inference, improved numerical stability and scalability. The improvement is especially noticeable for data-rich models. We demonstrate the efficiency gains with various examples of data-rich models, like Cox's proportional hazards model, an item-response theory model, a spatial model including prediction, and a 3-dimensional model for fMRI data. △ Less

Submitted 14 April, 2022; originally announced April 2022.

Parallelized integrated nested Laplace approximations for fast Bayesian inference

Authors: Lisa Gaedke-Merzhäuser, Janet van Niekerk, Olaf Schenk, Håvard Rue

Abstract: There is a growing demand for performing larger-scale Bayesian inference tasks, arising from greater data availability and higher-dimensional model parameter spaces. In this work we present parallelization strategies for the methodology of integrated nested Laplace approximations (INLA), a popular framework for performing approximate Bayesian inference on the class of Latent Gaussian models. Our a… ▽ More There is a growing demand for performing larger-scale Bayesian inference tasks, arising from greater data availability and higher-dimensional model parameter spaces. In this work we present parallelization strategies for the methodology of integrated nested Laplace approximations (INLA), a popular framework for performing approximate Bayesian inference on the class of Latent Gaussian models. Our approach makes use of nested OpenMP parallelism, a parallel line search procedure using robust regression in INLA's optimization phase and the state-of-the-art sparse linear solver PARDISO. We leverage mutually independent function evaluations in the algorithm as well as advanced sparse linear algebra techniques. This way we can flexibly utilize the power of today's multi-core architectures. We demonstrate the performance of our new parallelization scheme on a number of different real-world applications. The introduction of parallelism leads to speedups of a factor 10 and more for all larger models. Our work is already integrated in the current version of the open-source R-INLA package, making its improved performance conveniently available to all users. △ Less

Submitted 10 April, 2022; originally announced April 2022.

Comments: 18 pages, 7 figures

An Extended Simplified Laplace strategy for Approximate Bayesian inference of Latent Gaussian Models using R-INLA

Authors: Cristian Chiuchiolo, Janet van Niekerk, Håvard Rue

Abstract: Various computational challenges arise when applying Bayesian inference approaches to complex hierarchical models. Sampling-based inference methods, such as Markov Chain Monte Carlo strategies, are renowned for providing accurate results but with high computational costs and slow or questionable convergence. On the contrary, approximate methods like the Integrated Nested Laplace Approximation (INL… ▽ More Various computational challenges arise when applying Bayesian inference approaches to complex hierarchical models. Sampling-based inference methods, such as Markov Chain Monte Carlo strategies, are renowned for providing accurate results but with high computational costs and slow or questionable convergence. On the contrary, approximate methods like the Integrated Nested Laplace Approximation (INLA) construct a deterministic approximation to the univariate posteriors through nested Laplace Approximations. This method enables fast inference performance in Latent Gaussian Models, which encode a large class of hierarchical models. R-INLA software mainly consists of three strategies to compute all the required posterior approximations depending on the accuracy requirements. The Simplified Laplace approximation (SLA) is the most attractive because of its speed performance since it is based on a Taylor expansion up to order three of a full Laplace Approximation. Here we enhance the methodology by simplifying the computations necessary for the skewness and modal configuration. Then we propose an expansion up to order four and use the Extended Skew Normal distribution as a new parametric fit. The resulting approximations to the marginal posterior densities are more accurate than those calculated with the SLA, with essentially no additional cost. △ Less

Submitted 27 March, 2022; originally announced March 2022.

Comments: 22 pages, 11 figures

Fast and flexible inference for joint models of multivariate longitudinal and survival data using Integrated Nested Laplace Approximations

Authors: Denis Rustand, Janet van Niekerk, Elias Teixeira Krainski, Håvard Rue, Cécile Proust-Lima

Abstract: Modeling longitudinal and survival data jointly offers many advantages such as addressing measurement error and missing data in the longitudinal processes, understanding and quantifying the association between the longitudinal markers and the survival events and predicting the risk of events based on the longitudinal markers. A joint model involves multiple submodels (one for each longitudinal/sur… ▽ More Modeling longitudinal and survival data jointly offers many advantages such as addressing measurement error and missing data in the longitudinal processes, understanding and quantifying the association between the longitudinal markers and the survival events and predicting the risk of events based on the longitudinal markers. A joint model involves multiple submodels (one for each longitudinal/survival outcome) usually linked together through correlated or shared random effects. Their estimation is computationally expensive (particularly due to a multidimensional integration of the likelihood over the random effects distribution) so that inference methods become rapidly intractable, and restricts applications of joint models to a small number of longitudinal markers and/or random effects. We introduce a Bayesian approximation based on the Integrated Nested Laplace Approximation algorithm implemented in the R package R-INLA to alleviate the computational burden and allow the estimation of multivariate joint models with fewer restrictions. Our simulation studies show that R-INLA substantially reduces the computation time and the variability of the parameter estimates compared to alternative estimation strategies. We further apply the methodology to analyze 5 longitudinal markers (3 continuous, 1 count, 1 binary, and 16 random effects) and competing risks of death and transplantation in a clinical trial on primary biliary cholangitis. R-INLA provides a fast and reliable inference technique for applying joint models to the complex multivariate data encountered in health research. △ Less

Submitted 12 July, 2023; v1 submitted 11 March, 2022; originally announced March 2022.

Controlling the flexibility of non-Gaussian processes through shrinkage priors

Authors: Rafael Cabral, David Bolin, Håvard Rue

Abstract: The normal inverse Gaussian (NIG) and generalized asymmetric Laplace (GAL) distributions can be seen as skewed and semi-heavy-tailed extensions of the Gaussian distribution. Models driven by these more flexible noise distributions are then regarded as flexible extensions of simpler Gaussian models. Inferential procedures tend to overestimate the degree of non-Gaussianity in the data and therefore… ▽ More The normal inverse Gaussian (NIG) and generalized asymmetric Laplace (GAL) distributions can be seen as skewed and semi-heavy-tailed extensions of the Gaussian distribution. Models driven by these more flexible noise distributions are then regarded as flexible extensions of simpler Gaussian models. Inferential procedures tend to overestimate the degree of non-Gaussianity in the data and therefore we propose controlling the flexibility of these non-Gaussian models by adding sensible priors in the inferential framework that contract the model towards Gaussianity. In our venture to derive sensible priors, we also propose a new intuitive parameterization of the non-Gaussian models and discuss how to implement them efficiently in $Stan$. The methods are derived for a generic class of non-Gaussian models that include spatial Matérn fields, autoregressive models for time series, and simultaneous autoregressive models for aerial data. The results are illustrated with a simulation study and geostatistics application, where priors that penalize model complexity were shown to lead to more robust estimation and give preference to the Gaussian model, while at the same time allowing for non-Gaussianity if there is sufficient evidence in the data. △ Less

Submitted 29 October, 2022; v1 submitted 10 March, 2022; originally announced March 2022.

Comments: 34 pages, 12 figures; Added new section about Stan implementation and a cross-validation study in the application section, removed unecessary details

MSC Class: 62F15; 62M20 (Primary); 62M40 (Secondary)

Joint Modeling and Prediction of Massive Spatio-Temporal Wildfire Count and Burnt Area Data with the INLA-SPDE Approach

Authors: Zhongwei Zhang, Elias Krainski, Peng Zhong, Håvard Rue, Raphaël Huser

Abstract: This paper describes the methodology used by the team RedSea in the data competition organized for EVA 2021 conference. We develop a novel two-part model to jointly describe the wildfire count data and burnt area data provided by the competition organizers with covariates. Our proposed methodology relies on the integrated nested Laplace approximation combined with the stochastic partial differenti… ▽ More This paper describes the methodology used by the team RedSea in the data competition organized for EVA 2021 conference. We develop a novel two-part model to jointly describe the wildfire count data and burnt area data provided by the competition organizers with covariates. Our proposed methodology relies on the integrated nested Laplace approximation combined with the stochastic partial differential equation (INLA-SPDE) approach. In the first part, a binary non-stationary spatio-temporal model is used to describe the underlying process that determines whether or not there is wildfire at a specific time and location. In the second part, we consider a non-stationary model that is based on log-Gaussian Cox processes for positive wildfire count data, and a non-stationary log-Gaussian model for positive burnt area data. Dependence between the positive count data and positive burnt area data is captured by a shared spatio-temporal random effect. Our two-part modeling approach performs well in terms of the prediction score criterion chosen by the data competition organizers. Moreover, our model results show that surface pressure is the most influential driver for the occurrence of a wildfire, whilst surface net solar radiation and surface pressure are the key drivers for large numbers of wildfires, and temperature and evaporation are the key drivers of large burnt areas. △ Less

Submitted 14 February, 2022; originally announced February 2022.

Joint Quantile Disease Map** with Application to Malaria and G6PD Deficiency

Authors: Hanan Alahmadi, Håvard Rue, Janet van Niekerk

Abstract: Statistical analysis based on quantile regression methods is more comprehensive, flexible, and less sensitive to outliers when compared to mean regression methods. When the link between different diseases are of interest, joint disease map** is useful for measuring directional correlation between them. Most studies study this link through multiple correlated mean regressions. In this paper we pr… ▽ More Statistical analysis based on quantile regression methods is more comprehensive, flexible, and less sensitive to outliers when compared to mean regression methods. When the link between different diseases are of interest, joint disease map** is useful for measuring directional correlation between them. Most studies study this link through multiple correlated mean regressions. In this paper we propose a joint quantile regression framework for multiple diseases where different quantile levels can be considered. We are motivated by the theorized link between the presence of Malaria and the gene deficiency G6PD, where medical scientist have anecdotally discovered a possible link between high levels of G6PD and lower than expected levels of Malaria initially pointing towards the occurrence of G6PD inhibiting the occurrence of Malaria. This link cannot be investigated with mean regressions and thus the need for flexible joint quantile regression in a disease map** framework. Our joint quantile disease map** model can be used for linear and non-linear effects of covariates by stochastic splines, since we define it as a latent Gaussian model. We perform Bayesian inference of this model using the INLA framework embedded in the R software package INLA. Finally, we illustrate the applicability of model by analyzing the malaria and G6PD deficiency incidences in 21 African countries using linked quantiles of different levels. △ Less

Submitted 30 January, 2022; originally announced January 2022.

Joint Posterior Inference for Latent Gaussian Models with R-INLA

Authors: Cristian Chiuchiolo, Janet van Niekerk, Haavard Rue

Abstract: Efficient Bayesian inference remains a computational challenge in hierarchical models. Simulation-based approaches such as Markov Chain Monte Carlo methods are still popular but have a large computational cost. When dealing with the large class of Latent Gaussian Models, the INLA methodology embedded in the R-INLA software provides accurate Bayesian inference by computing deterministic mixture rep… ▽ More Efficient Bayesian inference remains a computational challenge in hierarchical models. Simulation-based approaches such as Markov Chain Monte Carlo methods are still popular but have a large computational cost. When dealing with the large class of Latent Gaussian Models, the INLA methodology embedded in the R-INLA software provides accurate Bayesian inference by computing deterministic mixture representation to approximate the joint posterior, from which marginals are computed. The INLA approach has from the beginning been targeting to approximate univariate posteriors. In this paper we lay out the development foundation of the tools for also providing joint approximations for subsets of the latent field. These approximations inherit Gaussian copula structure and additionally provide corrections for skewness. The same idea is carried forward also to sampling from the mixture representation, which we now can adjust for skewness. △ Less

Submitted 6 December, 2021; originally announced December 2021.

Comments: 33 pages, 11 figures

Low-rank variational Bayes correction to the Laplace method

Authors: Janet van Niekerk, Haavard Rue

Abstract: Approximate inference methods like the Laplace method, Laplace approximations and variational methods, amongst others, are popular methods when exact inference is not feasible due to the complexity of the model or the abundance of data. In this paper we propose a hybrid approximate method called Low-Rank Variational Bayes correction (VBC), that uses the Laplace method and subsequently a Variationa… ▽ More Approximate inference methods like the Laplace method, Laplace approximations and variational methods, amongst others, are popular methods when exact inference is not feasible due to the complexity of the model or the abundance of data. In this paper we propose a hybrid approximate method called Low-Rank Variational Bayes correction (VBC), that uses the Laplace method and subsequently a Variational Bayes correction in a lower dimension, to the joint posterior mean. The cost is essentially that of the Laplace method which ensures scalability of the method, in both model complexity and data size. Models with fixed and unknown hyperparameters are considered, for simulated and real examples, for small and large datasets. △ Less

Submitted 14 November, 2023; v1 submitted 25 November, 2021; originally announced November 2021.

The SPDE approach for Gaussian and non-Gaussian fields: 10 years and still running

Authors: Finn Lindgren, David Bolin, Håvard Rue

Abstract: Gaussian processes and random fields have a long history, covering multiple approaches to representing spatial and spatio-temporal dependence structures, such as covariance functions, spectral representations, reproducing kernel Hilbert spaces, and graph based models. This article describes how the stochastic partial differential equation approach to generalising Matérn covariance models via Hilbe… ▽ More Gaussian processes and random fields have a long history, covering multiple approaches to representing spatial and spatio-temporal dependence structures, such as covariance functions, spectral representations, reproducing kernel Hilbert spaces, and graph based models. This article describes how the stochastic partial differential equation approach to generalising Matérn covariance models via Hilbert space projections connects with several of these approaches, with each connection being useful in different situations. In addition to an overview of the main ideas, some important extensions, theory, applications, and other recent developments are discussed. The methods include both Markovian and non-Markovian models, non-Gaussian random fields, non-stationary fields and space-time fields on arbitrary manifolds, and practical computational considerations. △ Less

Submitted 4 January, 2022; v1 submitted 1 November, 2021; originally announced November 2021.

Comments: 33 pages, 1 figure

MSC Class: 60G60 (Primary); 60G60; 62M40; 62-08 (Secondary)

Variance partitioning in spatio-temporal disease map** models

Authors: Maria Franco-Villoria, Massimo Ventrucci, Håvard Rue

Abstract: Bayesian disease map**, yet if undeniably useful to describe variation in risk over time and space, comes with the hurdle of prior elicitation on hard-to-interpret random effect precision parameters. We introduce a reparametrized version of the popular spatio-temporal interaction models, based on Kronecker product intrinsic Gaussian Markov Random Fields, that we name the variance partitioning (V… ▽ More Bayesian disease map**, yet if undeniably useful to describe variation in risk over time and space, comes with the hurdle of prior elicitation on hard-to-interpret random effect precision parameters. We introduce a reparametrized version of the popular spatio-temporal interaction models, based on Kronecker product intrinsic Gaussian Markov Random Fields, that we name the variance partitioning (VP) model. The VP model includes a mixing parameter that balances the contribution of the main and interaction effects to the total (generalized) variance and enhances interpretability. The use of a penalized complexity prior on the mixing parameter aids in coding prior information in a intuitive way. We illustrate the advantages of the VP model using two case studies. △ Less

Submitted 17 May, 2022; v1 submitted 27 September, 2021; originally announced September 2021.

Quantification of empirical determinacy: the impact of likelihood weighting on posterior location and spread in Bayesian meta-analysis estimated with JAGS and INLA

Authors: Sona Hunanyan, Håvard Rue, Martyn Plummer, Małgorzata Roos

Abstract: The popular Bayesian meta-analysis expressed by Bayesian normal-normal hierarchical model (NNHM) synthesizes knowledge from several studies and is highly relevant in practice. Moreover, NNHM is the simplest Bayesian hierarchical model (BHM), which illustrates problems typical in more complex BHMs. Until now, it has been unclear to what extent the data determines the marginal posterior distribution… ▽ More The popular Bayesian meta-analysis expressed by Bayesian normal-normal hierarchical model (NNHM) synthesizes knowledge from several studies and is highly relevant in practice. Moreover, NNHM is the simplest Bayesian hierarchical model (BHM), which illustrates problems typical in more complex BHMs. Until now, it has been unclear to what extent the data determines the marginal posterior distributions of the parameters in NNHM. To address this issue we computed the second derivative of the Bhattacharyya coefficient with respect to the weighted likelihood, defined the total empirical determinacy (TED), the proportion of the empirical determinacy of location to TED (pEDL), and the proportion of the empirical determinacy of spread to TED (pEDS). We implemented this method in the R package \texttt{ed4bhm} and considered two case studies and one simulation study. We quantified TED, pEDL and pEDS under different modeling conditions such as model parametrization, the primary outcome, and the prior. This clarified to what extent the location and spread of the marginal posterior distributions of the parameters are determined by the data. Although these investigations focused on Bayesian NNHM, the method proposed is applicable more generally to complex BHMs. △ Less

Submitted 24 September, 2021; originally announced September 2021.

Comments: 22 pages, 1 figure

The Bayesian Learning Rule

Authors: Mohammad Emtiyaz Khan, Håvard Rue

Abstract: We show that many machine-learning algorithms are specific instances of a single algorithm called the \emph{Bayesian learning rule}. The rule, derived from Bayesian principles, yields a wide-range of algorithms from fields such as optimization, deep learning, and graphical models. This includes classical algorithms such as ridge regression, Newton's method, and Kalman filter, as well as modern dee… ▽ More We show that many machine-learning algorithms are specific instances of a single algorithm called the \emph{Bayesian learning rule}. The rule, derived from Bayesian principles, yields a wide-range of algorithms from fields such as optimization, deep learning, and graphical models. This includes classical algorithms such as ridge regression, Newton's method, and Kalman filter, as well as modern deep-learning algorithms such as stochastic-gradient descent, RMSprop, and Dropout. The key idea in deriving such algorithms is to approximate the posterior using candidate distributions estimated by using natural gradients. Different candidate distributions result in different algorithms and further approximations to natural gradients give rise to variants of those algorithms. Our work not only unifies, generalizes, and improves existing algorithms, but also helps us design new ones. △ Less

Submitted 8 June, 2024; v1 submitted 9 July, 2021; originally announced July 2021.

Journal ref: Journal of Machine Learning Research 24, no. 281 (2023): 1-46

Practical strategies for GEV-based regression models for extremes

Authors: Daniela Castro-Camilo, Raphaël Huser, Håvard Rue

Abstract: The generalised extreme value (GEV) distribution is a three parameter family that describes the asymptotic behaviour of properly renormalised maxima of a sequence of independent and identically distributed random variables. If the shape parameter $ξ$ is zero, the GEV distribution has unbounded support, whereas if $ξ$ is positive, the limiting distribution is heavy-tailed with infinite upper endpoi… ▽ More The generalised extreme value (GEV) distribution is a three parameter family that describes the asymptotic behaviour of properly renormalised maxima of a sequence of independent and identically distributed random variables. If the shape parameter $ξ$ is zero, the GEV distribution has unbounded support, whereas if $ξ$ is positive, the limiting distribution is heavy-tailed with infinite upper endpoint but finite lower endpoint. In practical applications, we assume that the GEV family is a reasonable approximation for the distribution of maxima over blocks, and we fit it accordingly. This implies that GEV properties, such as finite lower endpoint in the case $ξ>0$, are inherited by the finite-sample maxima, which might not have bounded support. This is particularly problematic when predicting extreme observations based on multiple and interacting covariates. To tackle this usually overlooked issue, we propose a blended GEV distribution, which smoothly combines the left tail of a Gumbel distribution (GEV with $ξ=0$) with the right tail of a Fréchet distribution (GEV with $ξ>0$) and, therefore, has unbounded support. Using a Bayesian framework, we reparametrise the GEV distribution to offer a more natural interpretation of the (possibly covariate-dependent) model parameters. Independent priors over the new location and spread parameters induce a joint prior distribution for the original location and scale parameters. We introduce the concept of property-preserving penalised complexity (P$^3$C) priors and apply it to the shape parameter to preserve first and second moments. We illustrate our methods with an application to NO$_2$ pollution levels in California, which reveals the robustness of the bGEV distribution, as well as the suitability of the new parametrisation and the P$^3$C prior framework. △ Less

Submitted 7 May, 2022; v1 submitted 24 June, 2021; originally announced June 2021.

Comments: 19 pages, 3 figures

Modelling sub-daily precipitation extremes with the blended generalised extreme value distribution

Authors: Silius M. Vandeskog, Sara Martino, Daniela Castro-Camilo, Håvard Rue

Abstract: A new method is proposed for modelling the yearly maxima of sub-daily precipitation, with the aim of producing spatial maps of return level estimates. Yearly precipitation maxima are modelled using a Bayesian hierarchical model with a latent Gaussian field, with the blended generalised extreme value (bGEV) distribution used as a substitute for the more standard generalised extreme value (GEV) dist… ▽ More A new method is proposed for modelling the yearly maxima of sub-daily precipitation, with the aim of producing spatial maps of return level estimates. Yearly precipitation maxima are modelled using a Bayesian hierarchical model with a latent Gaussian field, with the blended generalised extreme value (bGEV) distribution used as a substitute for the more standard generalised extreme value (GEV) distribution. Inference is made less wasteful with a novel two-step procedure that performs separate modelling of the scale parameter of the bGEV distribution using peaks over threshold data. Fast inference is performed using integrated nested Laplace approximations (INLA) together with the stochastic partial differential equation (SPDE) approach, both implemented in R-INLA. Heuristics for improving the numerical stability of R-INLA with the GEV and bGEV distributions are also presented. The model is fitted to yearly maxima of sub-daily precipitation from the south of Norway, and is able to quickly produce high-resolution return level maps with uncertainty. The proposed two-step procedure provides an improved model fit over standard inference techniques when modelling the yearly maxima of sub-daily precipitation with the bGEV distribution. △ Less

Submitted 21 May, 2022; v1 submitted 19 May, 2021; originally announced May 2021.

Importance Sampling with the Integrated Nested Laplace Approximation

Authors: Martin Outzen Berild, Sara Martino, Virgilio Gómez-Rubio, Håvard Rue

Abstract: The Integrated Nested Laplace Approximation (INLA) is a deterministic approach to Bayesian inference on latent Gaussian models (LGMs) and focuses on fast and accurate approximation of posterior marginals for the parameters in the models. Recently, methods have been developed to extend this class of models to those that can be expressed as conditional LGMs by fixing some of the parameters in the mo… ▽ More The Integrated Nested Laplace Approximation (INLA) is a deterministic approach to Bayesian inference on latent Gaussian models (LGMs) and focuses on fast and accurate approximation of posterior marginals for the parameters in the models. Recently, methods have been developed to extend this class of models to those that can be expressed as conditional LGMs by fixing some of the parameters in the models to descriptive values. These methods differ in the manner descriptive values are chosen. This paper proposes to combine importance sampling with INLA (IS-INLA), and extends this approach with the more robust adaptive multiple importance sampling algorithm combined with INLA (AMIS-INLA). This paper gives a comparison between these approaches and existing methods on a series of applications with simulated and observed datasets and evaluates their performance based on accuracy, efficiency, and robustness. The approaches are validated by exact posteriors in a simple bivariate linear model; then, they are applied to a Bayesian lasso model, a Bayesian imputation of missing covariate values, and lastly, in parametric Bayesian quantile regression. The applications show that the AMIS-INLA approach, in general, outperforms the other methods, but the IS-INLA algorithm could be considered for faster inference when good proposals are available. △ Less

Submitted 3 March, 2021; originally announced March 2021.

Bayesian Estimation of Two-Part Joint Models for a Longitudinal Semicontinuous Biomarker and a Terminal Event with R-INLA: Interests for Cancer Clinical Trial Evaluation

Authors: Denis Rustand, Janet van Niekerk, Håvard Rue, Christophe Tournigand, Virginie Rondeau, Laurent Briollais

Abstract: Two-part joint models for a longitudinal semicontinuous biomarker and a terminal event have been recently introduced based on frequentist estimation. The biomarker distribution is decomposed into a probability of positive value and the expected value among positive values. Shared random effects can represent the association structure between the biomarker and the terminal event. The computational… ▽ More Two-part joint models for a longitudinal semicontinuous biomarker and a terminal event have been recently introduced based on frequentist estimation. The biomarker distribution is decomposed into a probability of positive value and the expected value among positive values. Shared random effects can represent the association structure between the biomarker and the terminal event. The computational burden increases compared to standard joint models with a single regression model for the biomarker. In this context, the frequentist estimation implemented in the R package frailtypack can be challenging for complex models (i.e., large number of parameters and dimension of the random effects). As an alternative, we propose a Bayesian estimation of two-part joint models based on the Integrated Nested Laplace Approximation (INLA) algorithm to alleviate the computational burden and fit more complex models. Our simulation studies confirm that INLA provides accurate approximation of posterior estimates and to reduced computation time and variability of estimates compared to frailtypack in the situations considered. We contrast the Bayesian and frequentist approaches in the analysis of two randomized cancer clinical trials (GERCOR and PRIME studies), where INLA has a reduced variability for the association between the biomarker and the risk of event. Moreover, the Bayesian approach was able to characterize subgroups of patients associated with different responses to treatment in the PRIME study. Our study suggests that the Bayesian approach using INLA algorithm enables to fit complex joint models that might be of interest in a wide range of clinical applications. △ Less

Submitted 27 January, 2023; v1 submitted 26 October, 2020; originally announced October 2020.

Model-based bias correction for short AR(1) and AR(2) processes

Authors: Sigrunn H. Sørbye, Pedro G. Nicolau, Håvard Rue

Abstract: The class of autoregressive (AR) processes is extensively used to model temporal dependence in observed time series. Such models are easily available and routinely fitted using freely available statistical software like R. A potential caveat in analyzing short time series is that commonly applied estimators for the coefficients of AR processes are severely biased. This paper suggests a model-based… ▽ More The class of autoregressive (AR) processes is extensively used to model temporal dependence in observed time series. Such models are easily available and routinely fitted using freely available statistical software like R. A potential caveat in analyzing short time series is that commonly applied estimators for the coefficients of AR processes are severely biased. This paper suggests a model-based approach for bias correction of well-known estimators for the coefficients of first and second-order stationary AR processes, taking the sampling distribution of the original estimator into account. This is achieved by modeling the relationship between the true and estimated AR coefficients using weighted orthogonal polynomial regression, fitted to a huge number of simulations. The finite-sample distributions of the new estimators are approximated using transformations of skew-normal densities and their properties are demonstrated by simulations and in the analysis of a real ecological data set. The new estimators are easily available in our accompanying R-package ARbiascorrect for time series of length n = 10, 11, ... , 50, where original estimates are found using exact or conditional maximum likelihood, Burg's method or the Yule-Walker equations. △ Less

Submitted 12 October, 2020; originally announced October 2020.

Comments: 21 pages, 11 figures

MSC Class: 62M10; 62F10; 62E17

Skewed probit regression -- Identifiability, contraction and reformulation

Authors: Janet van Niekerk, Haavard Rue

Abstract: Skewed probit regression is but one example of a statistical model that generalizes a simpler model, like probit regression. All skew-symmetric distributions and link functions arise from symmetric distributions by incorporating a skewness parameter through some skewing mechanism. In this work we address some fundamental issues in skewed probit regression, and more genreally skew-symmetric distrib… ▽ More Skewed probit regression is but one example of a statistical model that generalizes a simpler model, like probit regression. All skew-symmetric distributions and link functions arise from symmetric distributions by incorporating a skewness parameter through some skewing mechanism. In this work we address some fundamental issues in skewed probit regression, and more genreally skew-symmetric distributions or skew-symmetric link functions. We address the issue of identifiability of the skewed probit model parameters by reformulating the intercept from first principles. A new standardization of the skew link function is given to provide and anchored interpretation of the inference. Possible skewness parameters are investigated and the penalizing complexity priors of these are derived. This prior is invariant under reparameterization of the skewness parameter and quantifies the contraction of the skewed probit model to the probit model. The proposed results are available in the R-INLA package and we illustrate the use and effects of this work using simulated data, and well-known datasets using the link as well as the likelihood. △ Less

Submitted 20 September, 2020; originally announced September 2020.

A diffusion-based spatio-temporal extension of Gaussian Matérn fields

Authors: Finn Lindgren, Haakon Bakka, David Bolin, Elias Krainski, Håvard Rue

Abstract: Gaussian random fields with Matérn covariance functions are popular models in spatial statistics and machine learning. In this work, we develop a spatio-temporal extension of the Gaussian Matérn fields formulated as solutions to a stochastic partial differential equation. The spatially stationary subset of the models have marginal spatial Matérn covariances, and the model also extends to Whittle-M… ▽ More Gaussian random fields with Matérn covariance functions are popular models in spatial statistics and machine learning. In this work, we develop a spatio-temporal extension of the Gaussian Matérn fields formulated as solutions to a stochastic partial differential equation. The spatially stationary subset of the models have marginal spatial Matérn covariances, and the model also extends to Whittle-Matérn fields on curved manifolds, and to more general non-stationary fields. In addition to the parameters of the spatial dependence (variance, smoothness, and practical correlation range) it additionally has parameters controlling the practical correlation range in time, the smoothness in time, and the type of non-separability of the spatio-temporal covariance. Through the separability parameter, the model also allows for separable covariance functions. We provide a sparse representation based on a finite element approximation, that is well suited for statistical inference and which is implemented in the R-INLA software. The flexibility of the model is illustrated in an application to spatio-temporal modeling of global temperature data. △ Less

Submitted 5 April, 2023; v1 submitted 8 June, 2020; originally announced June 2020.

Comments: 40 pages, 10 figures

MSC Class: 60G60 (Primary); 62M20; 62M30; 62M40; 62-08 (Secondary)

Efficient Quantile Tracking Using an Oracle

Authors: Hugo L. Hammer, Anis Yazidi, Michael A. Riegler, Håvard Rue

Abstract: For incremental quantile estimators the step size and possibly other tuning parameters must be carefully set. However, little attention has been given on how to set these values in an online manner. In this article we suggest two novel procedures that address this issue. The core part of the procedures is to estimate the current tracking mean squared error (MSE). The MSE is decomposed in trackin… ▽ More For incremental quantile estimators the step size and possibly other tuning parameters must be carefully set. However, little attention has been given on how to set these values in an online manner. In this article we suggest two novel procedures that address this issue. The core part of the procedures is to estimate the current tracking mean squared error (MSE). The MSE is decomposed in tracking variance and bias and novel and efficient procedures to estimate these quantities are presented. It is shown that estimation bias can be tracked by associating it with the portion of observations below the quantile estimates. The first procedure runs an ensemble of $L$ quantile estimators for wide range of values of the tuning parameters and typically around $L = 100$. In each iteration an oracle selects the best estimate by the guidance of the estimated MSEs. The second method only runs an ensemble of $L = 3$ estimators and thus the values of the tuning parameters need from time to time to be adjusted for the running estimators. The procedures have a low memory foot print of $8L$ and a computational complexity of $8L$ per iteration. The experiments show that the procedures are highly efficient and track quantiles with an error close to the theoretical optimum. The Oracle approach performs best, but comes with higher computational cost. The procedures were further applied to a massive real-life data stream of tweets and proofed real world applicability of them. △ Less

Submitted 27 April, 2020; originally announced April 2020.

A principled distance-based prior for the shape of the Weibull model

Authors: Janet van Niekerk, Haakon Bakka, Haavard Rue

Abstract: The use of flat or weakly informative priors is popular due to the objective a priori belief in the absence of strong prior information. In the case of the Weibull model the improper uniform, equal parameter gamma and joint Jeffrey's priors for the shape parameter are popular choices. The effects and behaviors of these priors have yet to be established from a modeling viewpoint, especially their a… ▽ More The use of flat or weakly informative priors is popular due to the objective a priori belief in the absence of strong prior information. In the case of the Weibull model the improper uniform, equal parameter gamma and joint Jeffrey's priors for the shape parameter are popular choices. The effects and behaviors of these priors have yet to be established from a modeling viewpoint, especially their ability to reduce to the simpler exponential model. In this work we propose a new principled prior for the shape parameter of the Weibull model, originating from a prior on the distance function, and advocate this new prior as a principled choice in the absence of strong prior information. This new prior can then be used in models with a Weibull modeling component, like competing risks, joint and spatial models, to mention a few. This prior is available in the R-INLA for use, and is applied in a joint longitudinal-survival model framework using the INLA method. △ Less

Submitted 16 February, 2020; originally announced February 2020.

Estimating Tukey Depth Using Incremental Quantile Estimators

Authors: Hugo Lewi Hammer, Anis Yazidi, Håvard Rue

Abstract: The concept of depth represents methods to measure how deep an arbitrary point is positioned in a dataset and can be seen as the opposite of outlyingness. It has proved very useful and a wide range of methods have been developed based on the concept. To address the well-known computational challenges associated with the depth concept, we suggest to estimate Tukey depth contours using recently de… ▽ More The concept of depth represents methods to measure how deep an arbitrary point is positioned in a dataset and can be seen as the opposite of outlyingness. It has proved very useful and a wide range of methods have been developed based on the concept. To address the well-known computational challenges associated with the depth concept, we suggest to estimate Tukey depth contours using recently developed incremental quantile estimators. The suggested algorithm can estimate depth contours when the dataset in known in advance, but also recursively update and even track Tukey depth contours for dynamically varying data stream distributions. Tracking was demonstrated in a real-life data example where changes in human activity was detected in real-time from accelerometer observations. △ Less

Submitted 8 January, 2020; originally announced January 2020.

A Novel Method of Marginalisation using Low Discrepancy Sequences for Integrated Nested Laplace Approximations

Authors: Paul T. Brown, Chaitanya Joshi, Stephen Joe, Haavard Rue

Abstract: Recently, it has been shown that approximations to marginal posterior distributions obtained using a low discrepancy sequence (LDS) can outperform standard grid-based methods with respect to both accuracy and computational efficiency. This recent method, which we will refer to as LDS-StM, can also produce good approximations to multimodal posteriors. However, implementation of LDS-StM into integra… ▽ More Recently, it has been shown that approximations to marginal posterior distributions obtained using a low discrepancy sequence (LDS) can outperform standard grid-based methods with respect to both accuracy and computational efficiency. This recent method, which we will refer to as LDS-StM, can also produce good approximations to multimodal posteriors. However, implementation of LDS-StM into integrated nested Laplace approximations (INLA), a methodology in which grid-based methods are used, is challenging. Motivated by this problem, we propose modifications to LDS-StM that improves the approximations and make it compatible with INLA, without sacrificing computational speed. We also present two examples to demonstrate that LDS-StM with modifications can outperform INLA's own grid approximation with respect to speed and accuracy. We also demonstrate the flexibility of the new approach for the approximation of multimodal marginals. △ Less

Submitted 3 December, 2019; v1 submitted 22 November, 2019; originally announced November 2019.

Bayesian model averaging with the integrated nested Laplace approximation

Authors: Virgilio Gómez-Rubio, Roger S. Bivand, Håvard Rue

Abstract: The integrated nested Laplace approximation (INLA) for Bayesian inference is an efficient approach to estimate the posterior marginal distributions of the parameters and latent effects of Bayesian hierarchical models that can be expressed as latent Gaussian Markov random fields (GMRF). The representation as a GMRF allows the associated software R-INLA to estimate the posterior marginals in a fract… ▽ More The integrated nested Laplace approximation (INLA) for Bayesian inference is an efficient approach to estimate the posterior marginal distributions of the parameters and latent effects of Bayesian hierarchical models that can be expressed as latent Gaussian Markov random fields (GMRF). The representation as a GMRF allows the associated software R-INLA to estimate the posterior marginals in a fraction of the time as typical Markov chain Monte Carlo algorithms. INLA can be extended by means of Bayesian model averaging (BMA) to increase the number of models that it can fit to conditional latent GMRF. In this paper we review the use of BMA with INLA and propose a new example on spatial econometrics models. △ Less

Submitted 2 November, 2019; originally announced November 2019.

Comments: Submitted to Econometrics (MDPI)

Competing risks joint models using R-INLA

Authors: Janet van Niekerk, Haakon Bakka, Haavard Rue

Abstract: The methodological advancements made in the field of joint models are numerous. None the less, the case of competing risks joint models have largely been neglected, especially from a practitioner's point of view. In the relevant works on competing risks joint models, the assumptions of Gaussian linear longitudinal series and proportional cause-specific hazard functions, amongst others, have remain… ▽ More The methodological advancements made in the field of joint models are numerous. None the less, the case of competing risks joint models have largely been neglected, especially from a practitioner's point of view. In the relevant works on competing risks joint models, the assumptions of Gaussian linear longitudinal series and proportional cause-specific hazard functions, amongst others, have remained unchallenged. In this paper, we provide a framework based on R-INLA to apply competing risks joint models in a unifying way such that non-Gaussian longitudinal data, spatial structures, time dependent splines and various latent association structures, to mention a few, are all embraced in our approach. Our motivation stems from the SANAD trial which exhibits non-linear longitudinal trajectories and competing risks for failure of treatment. We also present a discrete competing risks joint model for longitudinal count data as well as a spatial competing risks joint model, as specific examples. △ Less

Submitted 4 September, 2019; originally announced September 2019.

Fast Bayesian inference of Block Nearest Neighbor Gaussian process for large data

Authors: Zaida C. Quiroz, Marcos O. Prates, Dipak K. Dey, Håvard Rue

Abstract: This paper presents the development of a spatial block-Nearest Neighbor Gaussian process (block-NNGP) for location-referenced large spatial data. The key idea behind this approach is to divide the spatial domain into several blocks which are dependent under some constraints. The cross-blocks capture the large-scale spatial dependence, while each block captures the small-scale spatial dependence. T… ▽ More This paper presents the development of a spatial block-Nearest Neighbor Gaussian process (block-NNGP) for location-referenced large spatial data. The key idea behind this approach is to divide the spatial domain into several blocks which are dependent under some constraints. The cross-blocks capture the large-scale spatial dependence, while each block captures the small-scale spatial dependence. The resulting block-NNGP enjoys Markov properties reflected on its sparse precision matrix. It is embedded as a prior within the class of latent Gaussian models, thus Bayesian inference is obtained using the integrated nested Laplace approximation (INLA). The performance of the block-NNGP is illustrated on simulated examples and massive real data for locations in the order of $10^4$. △ Less

Submitted 4 February, 2021; v1 submitted 18 August, 2019; originally announced August 2019.

Comments: 60 pages, 20 figures (including the ones in the Supplementary Material), 4 tables

Statistical modeling of groundwater quality assessment in Iran using a flexible Poisson likelihood

Authors: Mahsa Nadifar, Hossein Baghishani, Afshin Fallah, Havard Rue

Abstract: Assessing water quality and recognizing its associated risks to human health and the broader environment is undoubtedly essential. Groundwater is widely used to supply water for drinking, industry, and agriculture purposes. The groundwater quality measurements vary for different climates and various human behaviors, and consequently, their spatial variability can be substantial. In this paper, we… ▽ More Assessing water quality and recognizing its associated risks to human health and the broader environment is undoubtedly essential. Groundwater is widely used to supply water for drinking, industry, and agriculture purposes. The groundwater quality measurements vary for different climates and various human behaviors, and consequently, their spatial variability can be substantial. In this paper, we aim to analyze a groundwater dataset from the Golestan province, Iran, for November 2003 to November 2013. Our target response variable to monitor the quality of groundwater is the number of counts that the quality of water is good for a drink. Hence, we are facing spatial count data. Due to the ubiquity of over or underdispersion in count data, we propose a Bayesian hierarchical modeling approach based on the renewal theory that relates nonexponential waiting times between events and the distribution of the counts, relaxing the assumption of equidispersion at the cost of an additional parameter. Particularly, we extend the methodology for the analysis of spatial count data based on the gamma distribution assumption for waiting times. The model can be formulated as a latent Gaussian model, and therefore, we can carry out the fast computation by using the integrated nested Laplace approximation method. The analysis of the groundwater dataset and a simulation study show a significant improvement over both Poisson and negative binomial models. △ Less

Submitted 6 August, 2019; originally announced August 2019.

Comments: 24 pages, 6 figures

New frontiers in Bayesian modeling using the INLA package in R

Authors: Janet van Niekerk, Haakon Bakka, Haavard Rue, Olaf Schenk

Abstract: The INLA package provides a tool for computationally efficient Bayesian modeling and inference for various widely used models, more formally the class of latent Gaussian models. It is a non-sampling based framework which provides approximate results for Bayesian inference, using sparse matrices. The swift uptake of this framework for Bayesian modeling is rooted in the computational efficiency of t… ▽ More The INLA package provides a tool for computationally efficient Bayesian modeling and inference for various widely used models, more formally the class of latent Gaussian models. It is a non-sampling based framework which provides approximate results for Bayesian inference, using sparse matrices. The swift uptake of this framework for Bayesian modeling is rooted in the computational efficiency of the approach and catalyzed by the demand presented by the big data era. In this paper, we present new developments within the INLA package with the aim to provide a computationally efficient mechanism for the Bayesian inference of relevant challenging situations. △ Less

Submitted 25 July, 2019; v1 submitted 24 July, 2019; originally announced July 2019.

Improving Bayesian Local Spatial Models in Large Data Sets

Authors: Amanda Lenzi, Stefano Castruccio, Haavard Rue, Marc G. Genton

Abstract: Environmental processes resolved at a sufficiently small scale in space and time will inevitably display non-stationary behavior. Such processes are both challenging to model and computationally expensive when the data size is large. Instead of modeling the global non-stationarity explicitly, local models can be applied to disjoint regions of the domain. The choice of the size of these regions is… ▽ More Environmental processes resolved at a sufficiently small scale in space and time will inevitably display non-stationary behavior. Such processes are both challenging to model and computationally expensive when the data size is large. Instead of modeling the global non-stationarity explicitly, local models can be applied to disjoint regions of the domain. The choice of the size of these regions is dictated by a bias-variance trade-off; large regions will have smaller variance and larger bias, whereas small regions will have higher variance and smaller bias. From both the modeling and computational point of view, small regions are preferable to better accommodate the non-stationarity. However, in practice, large regions are necessary to control the variance. We propose a novel Bayesian three-step approach that allows for smaller regions without compromising the increase of the variance that would follow. We are able to propagate the uncertainty from one step to the next without issues caused by reusing the data. The improvement in inference also results in improved prediction, as our simulated example shows. We illustrate this new approach on a data set of simulated high-resolution wind speed data over Saudi Arabia. △ Less

Submitted 20 August, 2020; v1 submitted 16 July, 2019; originally announced July 2019.

Joint Tracking of Multiple Quantiles Through Conditional Quantiles

Authors: Hugo Lewi Hammer, Anis Yazidi, Håvard Rue

Abstract: Estimation of quantiles is one of the most fundamental real-time analysis tasks. Most real-time data streams vary dynamically with time and incremental quantile estimators document state-of-the art performance to track quantiles of such data streams. However, most are not able to make joint estimates of multiple quantiles in a consistent manner, and estimates may violate the monotone property of q… ▽ More Estimation of quantiles is one of the most fundamental real-time analysis tasks. Most real-time data streams vary dynamically with time and incremental quantile estimators document state-of-the art performance to track quantiles of such data streams. However, most are not able to make joint estimates of multiple quantiles in a consistent manner, and estimates may violate the monotone property of quantiles. In this paper we propose the general concept of *conditional quantiles* that can extend incremental estimators to jointly track multiple quantiles. We apply the concept to propose two new estimators. Extensive experimental results, on both synthetic and real-life data, show that the new estimators clearly outperform legacy state-of-the-art joint quantile tracking algorithm and achieve faster adaptivity in dynamically varying data streams. △ Less

Submitted 13 February, 2019; originally announced February 2019.

Comments: arXiv admin note: text overlap with arXiv:1901.04681