inlabru: software for fitting latent Gaussian models with non-linear predictors

The following example demonstrates how to define a model, fit it, and generate posterior samples using inlabru. It contains all the basic building blocks of the inlabru workflow that will be present in later examples.

We use a simulated dataset with observations

$$z_{i}=\beta+f(\boldsymbol{s}_{i})+\epsilon_{i},\quad i=\dots,n,$$

(1)

where $\boldsymbol{s}_{i}$ is the location of observation $i$, $\beta$ is an intercept parameter, $f$ is a Gaussian random field with Matérn covariance and $\epsilon_{i}$ is unstructured Gaussian noise.

The data is stored as an sf spatial points object (Figure 1 A) which can be used directly with inlabru. The data is included with inlabru under the name toypoints. The following code shows how to load the data, fit a model, and generate model predictions.

R> library(inlabru)R> library(INLA)R> library(sf)R>R> # Load dataR> point_data <- toypoints$points # sf point dataR> mesh <- toypoints$mesh # SPDE meshR> pred_locs <- toypoints$pred_locs # sf prediction point locationsR>R> # Define model componentsR>R> # Spatially structure random effectR> matern <- inla.spde2.pcmatern(mesh,+    prior.range = c(2, 0.05),+    prior.sigma = c(2, 0.05)+  )R>R> cmp <- ~ Intercept(1) ++    grf(+      main = geometry,+      model = matern+    )R>R> # Construct likelihoodR> lik <- like(+    formula = z ~ .,+    data = point_data,+    family = "gaussian"+  )R>R> # Fit modelR> fit <- bru(lik,+    components = cmp+  )R>R> # Make predictionsR> predictions <- predict(fit,+    pred_locs,+    formula = ~ Intercept + grf+  )

This example contains the essential elements of the inlabru workflow. The user defines model components using a formula syntax that allows users to choose meaningful names for model components. Here the labels Intercept and grf (an abbreviation of Gaussian random field) were used to name the components of the model but they equally well could be something else. These are labels for components, not functions, despite the similarity to the function syntax.

The main argument describes the input data for each component. The spatial information is stored in the geometry column of the sf object and so this is used when defining the random field component. The software automatically detects the type of input, the type of component, and interally constructs the prior precision and model design matrix for that component.

An intercept parameter is associated with a column of ones in the model design matrix, and the shorthand notation Intercept(1) reflects this. Alternative methods to define an intercept are Intercept(ones, model = ’linear’) if ones is the name of a covariate in the data or Intercept(rep(1,n), model = ’linear’) if n is an object stored in the global R environment. The input data for a component is allowed to be a general R expression song as it returns an object that is appropriate for that type of model component.

The likelihood is constructed in a separate step to model fitting with the likelihood object created by the like() function. The sf point data object is used directly, requiring no pre-processing of data to a standard data.frame. This automates the process of ‘stack building’ familiar to INLA users.

After fitting the model, predictions are generated using a predict() method. The formula object for the prediction can be a generic R expression that references model components using the user-defined names. This greatly simplifies the process for generating predictions from INLA models. Figure 1 shows the data and the posterior mean prediction.

Figure 1B was produced using the predictions object. The predict() method returns an object in the same data format as was used in the predict call which, in this case, is an sf points object. Support for plotting sf data objects is available in the ggplot2 package (Wickham, 2009). The code to produce Figure 1B is as simple as

R> ggplot() ++    gg(+      data = predictions,+      aes(fill = mean),+      geom = "tile"+    ) ++    scale_colour_viridis_c() ++    theme_classic() where mean is one of the summary statistics returned by predict(). See ?predict.bru for a list of all summary statistics. In general, the prediction data can be in any format for which the component definitions make sense. This gives users flexibility to conceptually separate data that is used for model fitting from data that is used for prediction.

This example is not simple. It involves spatial data, a spatially structured random effect, and generating model predictions. Users already familiar with such models in INLA will recognise that the inlabru code required to run this is more concise and readable. This allows users to focus more on the model structure rather than wrangling data and design matrices into a format expected by INLA.

The rest of the paper is structured as follows: Section 2 reviews the standard INLA approach and describes the iterative INLA method for estimation with non-linear predictors, Section 3 covers basic package structure and syntax for specifying, fitting and predicting from models, and Section 4 presents a Bayesian method checking simulation to validate the iterative INLA inference method. Section 5 presents syntax examples for fitting spatial models. Examples include a standard LGM with a Besag-York-Mollié random effect, a non-linear predictor example, aggregating a continuously defined random field to discrete areal units, and a joint likelihood model. Finally, Section 6 draws links to other methods that extend the class of models that can be fitted using INLA, discusses existing published research that uses the inlabru non-linear predictors feature, existing software that depends on inlabru, and, finally, ideas for the future development of inlabru.

The INLA method is an approach to compute fast approximate posterior distributions for Bayesian generalised additive models. The class of models that INLA was developed to address are known as latent Gaussian models (LGMs).

The hierarchical structure of a LGM with latent Gaussian vector $\boldsymbol{u}$, covariance parameters $\boldsymbol{\theta}$, and response variable $\boldsymbol{y}$, can be written as

	$\displaystyle\boldsymbol{\theta}$	$\displaystyle\sim p(\boldsymbol{\theta})$
	$\displaystyle\boldsymbol{u}|\boldsymbol{\theta}$	$\displaystyle\sim\mathcal{N}\!\left(\boldsymbol{\mu}_{u},\boldsymbol{Q}(\boldsymbol{\theta})^{-1}\right)$
	$\displaystyle\boldsymbol{\eta}(\boldsymbol{u})$	$\displaystyle=\boldsymbol{A}\boldsymbol{u}$
	$\displaystyle\boldsymbol{y}|\boldsymbol{u},\boldsymbol{\theta}$	$\displaystyle\sim p(\boldsymbol{y}|\boldsymbol{\eta}(\boldsymbol{u}),\boldsymbol{\theta}).$

The latent Gaussian vector $\boldsymbol{u}$ has a covariance structure that depends on hyper parameters $\boldsymbol{\theta}$ which have prior distribution $p(\boldsymbol{\theta})$. Each linear predictor element, $\eta_{i}(\boldsymbol{u})$, is linked to the latent field via a linear map defined by the known, non-random matrix $\boldsymbol{A}$, i.e. the additive predictor for observation $y_{i}$ is $\eta_{i}(\boldsymbol{u})=\boldsymbol{A}_{i}\boldsymbol{u}$, where $\boldsymbol{A}_{i}$ is the $i$-th row of $\boldsymbol{A}$. This linear predictor defines the location parameter for the distribution of the data $\boldsymbol{y}$, via a (possibly non-linear) link function $g^{-1}(\cdot)$. Observations are assumed to be conditionally independent given $\boldsymbol{\eta}$ and $\boldsymbol{\theta}$, so that $p(\boldsymbol{y}|\boldsymbol{\eta}(\boldsymbol{u}),\boldsymbol{\theta})=\prod_{i}p(y_{i}|\eta_{i}(\boldsymbol{u}),\boldsymbol{\theta})$.

In the classic formulation of INLA (Rue et al., 2009), the linear predictors are included as part of an augmented latent Gaussian vector $\left[\boldsymbol{\eta},\boldsymbol{u}\right]^{\intercal}$ so that inference on the predictors is included as part of the overall estimation of the latent field. Since $\boldsymbol{\eta}$ is a linear combination of $\boldsymbol{u}$, a small amount of noise is added to avoid a singular precision. The INLA package has recently implemented an alternative approach which avoids the need to constructed the augmented latent Gaussian vector, which can be prohibitively large for certain classes of models (Van Niekerk et al., 2023; van Niekerk and Rue, 2024). We present the classic approach below, see Section 2.3 for a brief summary of the recent INLA developments.

INLA is a method to approximate the joint posterior density $p(\boldsymbol{\theta}|\boldsymbol{y})$, the marginal posterior distributions $(u_{j}|\boldsymbol{y})$ and allows for sampling from the joint posterior $(\boldsymbol{u}|\theta)$, where $\boldsymbol{u}$ now represents the augmented latent Gaussian vector. Rue et al. (2009) introduced the approach and Martins et al. (2013) includes additional details.

The joint posterior density of the hyperparameters is approximated as

$$\hat{p}(\boldsymbol{\theta}|\boldsymbol{y})\propto\left.\frac{p(\boldsymbol{u},\boldsymbol{\theta},\boldsymbol{y})}{p_{G}(\boldsymbol{u}|\boldsymbol{\theta},\boldsymbol{y})}\right|_{\boldsymbol{u}=\boldsymbol{u}^{*}(\boldsymbol{\theta})},$$

where $p_{G}(\boldsymbol{u}|\boldsymbol{\theta},\boldsymbol{y})$ is the Gaussian approximation to $p(\boldsymbol{u}|\boldsymbol{\theta},\boldsymbol{y})$ centred at the mode $\boldsymbol{u}^{*}(\boldsymbol{\theta})$ for a given $\boldsymbol{\theta}$. This is equivalent to the Laplace approximation of the marginal posterior (Tierney and Kadane, 1986). The marginal posterior $p(\theta_{k}|\boldsymbol{y})$ is approximated using an interpolation of $\hat{p}(\boldsymbol{\theta}|\boldsymbol{y})$ based on a grid search of the parameter space for $\boldsymbol{\theta}$. For this reason INLA is generally applied in contexts where $\boldsymbol{\theta}$ is low dimensional.

The marginal posterior density $p(u_{i}|\boldsymbol{y})$ is approximated as

$$\hat{p}(u_{i}|\boldsymbol{y})=\sum_{m}\hat{p}(u_{i}|\boldsymbol{\theta}^{(m)},\boldsymbol{y})\hat{p}(\boldsymbol{\theta}^{(m)}|\boldsymbol{y})\Delta\boldsymbol{\theta}^{(m)},$$

where $\hat{p}(u_{i}|\boldsymbol{\theta},\boldsymbol{y})$ is an approximate density given below and $\Delta\boldsymbol{\theta}^{(m)}$ is an integration weight associated with each grid search location for $\boldsymbol{\theta}$, indexed by $m$.

Rue and Martino (2007) originally proposed a Gaussian approximation for $\hat{p}(u_{i}|\boldsymbol{\theta},\boldsymbol{y})$ but found that this was not sufficiently accurate. Instead, Rue et al. (2009) improved on this by using another application of the Laplace approximation:

$$\hat{p}(u_{i}|\boldsymbol{\theta},\boldsymbol{y})\propto\left.\frac{p(\boldsymbol{u},\boldsymbol{\theta},\boldsymbol{y})}{p_{G}(\boldsymbol{u}_{-i}|u_{i},\boldsymbol{\theta},\boldsymbol{y})}\right|_{\boldsymbol{u}_{-i}=\boldsymbol{u}_{-i}^{*}(u_{i},\boldsymbol{\theta})},$$

where $p_{G}(\boldsymbol{u}_{-i}|u_{i},\boldsymbol{\theta},\boldsymbol{y})$ is the Gaussian approximation to $p(\boldsymbol{u}_{-i}|u_{i},\boldsymbol{\theta},\boldsymbol{y})$ centred at the mode $\boldsymbol{u}_{-i}^{*}(u_{i},\boldsymbol{\theta})$, given $u_{i}$ and $\boldsymbol{\theta}$. Until 2022, this was the standard method used by the INLA package. Although the joint posterior $p(\boldsymbol{u}|\boldsymbol{y})$ is not available directly, Chiuchiolo et al. (2023) propose a method generate samples from the joint posterior via a skew-Gaussian copula approach.

An alternative to the classic INLA approach was proposed by Van Niekerk et al. (2023) and van Niekerk and Rue (2024) that is more computationally efficient and avoids the need to include $\boldsymbol{\eta}$ as part of the latent field. In this case, the joint posterior is represented as $p(\boldsymbol{u},\boldsymbol{\theta}|\boldsymbol{y})\propto p(\boldsymbol{\theta})p(\boldsymbol{u}|\boldsymbol{\theta})\prod_{i}p(y_{i}|(\boldsymbol{A}\boldsymbol{u})_{i},\boldsymbol{\theta})$ and inference for $\boldsymbol{\eta}$ is dealt with separately to inference for $\boldsymbol{u}$ and $\boldsymbol{\theta}$. In this new approach, Van Niekerk et al. (2023) return to the idea of using a Gaussian approximation for $p(u_{i}|\boldsymbol{\theta},\boldsymbol{y})$. To correct for the inaccuracies identified in Rue and Martino (2007), van Niekerk and Rue (2024) use a variational Bayes correction to the mean of the Gaussian approximation. As shown by Van Niekerk et al. (2023), this approach is more computationally efficient and it was made the default mode of inference from version 22.11.22 of INLA.

For a fixed $\boldsymbol{\theta}$, the Gaussian approximation to $p(\boldsymbol{u}|\boldsymbol{\theta},\boldsymbol{y})$ is calculated from a second order Taylor expansion of the likelihood around the mode, with associated precision $\boldsymbol{Q}^{*}(\boldsymbol{\theta})$, which has a sparsity structure that depends on the sparsity of the prior precision $\boldsymbol{Q}(\boldsymbol{\theta})$ as well as the non-zero entries in $\boldsymbol{A}^{\intercal}\boldsymbol{A}$. The Gaussian approximation is then used to approximate the marginal posteriors as $p(u_{j}|\boldsymbol{\theta},\boldsymbol{y})\approx\mathsf{N}(\mu(\boldsymbol{\theta})_{j},\boldsymbol{Q}^{*}(\boldsymbol{\theta})^{-1}_{jj})$, where $\boldsymbol{\mu}(\boldsymbol{\theta})$ is the joint mode of $p(\boldsymbol{u}|\boldsymbol{\theta},\boldsymbol{y})$. The posterior density $p(u_{j}|\boldsymbol{y})$ is then approximated by numerical integration over $\boldsymbol{\theta}$, just as in the classic formulation of INLA. For inference on the predictors, the posterior $p(\boldsymbol{\eta}|\boldsymbol{\theta},\boldsymbol{y})$ is approximated as a Gaussian density with expected value $\boldsymbol{A}\boldsymbol{\mu}(\boldsymbol{\theta})$ and covariance $\boldsymbol{A}\boldsymbol{Q}^{*}(\boldsymbol{\theta})^{-1}\boldsymbol{A}^{\intercal}$.

Note that although $\boldsymbol{Q}^{*}(\boldsymbol{\theta})$ is sparse, its inverse is not. However, van Niekerk and Rue (2024) present a computationally efficient method to compute the $j$-th diagonal element of $\boldsymbol{A}\boldsymbol{Q}^{*}(\boldsymbol{\theta})^{-1}\boldsymbol{A}^{\intercal}$, which is all that is required for the approximation to $p(\eta_{j}|\boldsymbol{\theta},\boldsymbol{y})$ and avoids the need to compute the full inverse. The posterior density $p(\eta_{j}|\boldsymbol{y})$ is then approximated using numerical integration as above.

It is important to note that, although the approximate posteriors densities $p(u_{j}|\boldsymbol{\theta},\boldsymbol{y})$ and $p(\eta_{j}|\boldsymbol{\theta},\boldsymbol{y})$ are Gaussian, the full marginals can deviate from normality due to the numerical integration. This allows for, say, non-symmetric posteriors to be estimated, even when using these Gaussian approximations.

The structure of LGMs facilitates a modular view of the additive predictor $\boldsymbol{\eta}(\boldsymbol{u})=\boldsymbol{A}\boldsymbol{u}$, which can be decomposed by thinking of each fixed and random effect as individual model components. By a model component we mean anything that would usually be viewed as a separate additive component of the linear predictor, such as an intercept parameter, the linear effect of a known covariate or a ‘random effect’ component, such as the sparse Gaussian Markov Random Field (GMRF) effects (Rue and Held, 2005) that are implemented in INLA, examples of which include the Stochastic Partial Differential Equation (SPDE) approach to a GRF with Matérn covariance (Lindgren et al., 2011), and many other options including conditional auto-regressive and random walk models.

The matrix $\boldsymbol{A}$ is the model design matrix. In presentations of mixed effect models this is often decomposed as $\boldsymbol{A}=\left[\boldsymbol{X}\boldsymbol{Z}\right]$, where $\boldsymbol{X}$ contains the covariate information for fixed effects and $\boldsymbol{Z}$ contains the information to map the random effects to the observations. In the Bayesian LGM setting the distinction between fixed and random effects is somewhat arbitrary. We can think of the design matrix as decomposed into separate sub-matrices for each model component, i.e.

$$\boldsymbol{A}=\begin{bmatrix}\boldsymbol{A}^{(1)}&\dots&\boldsymbol{A}^{(d)}\end{bmatrix}$$

for a model with $d$ components, viewing each model component as having it’s own ‘component design matrix’, denoted here as $\boldsymbol{A}^{(j)}$ for $j=1,\dots,d$. The parameter vector can be partitioned into $\boldsymbol{u}=[\boldsymbol{u}^{(1)},\dots,\boldsymbol{u}^{(d)}]$, where each $\boldsymbol{u}^{(j)}$ is the parameter vector for component $j$. Given this decomposition the predictor expression for the $i$-th observation can be written as $\eta(\boldsymbol{u})_{i}=\delta_{i}+\sum_{j=1}^{d}\boldsymbol{A}^{(j)}_{i}\boldsymbol{u}^{(j)}$, where $\boldsymbol{A}^{(j)}_{i}$ is the $i$-th row of $\boldsymbol{A}^{(j)}$, and $\delta_{i}$ is an optional constant offset term.

For example, the component design matrix for an intercept parameter is a column vector of ones and the component design matrix for the effect of a known covariate is a column vector containing the covariate information for each observation. The component design matrix for an iid Gaussian effect is the identity matrix (or a block diagonal matrix consisting of identity sub-matrices of the correct dimension for each group). The component design matrix for an SPDE effect contains the finite element basis functions evaluated at each observation location. The key idea is that each component of the model predictor can be written as a known matrix multiplied by a parameter vector.

This view is the basis for the implementation in inlabru that automatically constructs the full model design matrix, given the component definitions and data. Each type of component has an associated method for constructing the component design matrix. These are known as bru_mapper() methods as they ‘map parameters to observations’. The user only needs to provide a model component definition and the data and the software does the rest. This component definition can be conceptualised in the mind of the user as ‘the linear effect of covariate x’ or ‘an SPDE effect on latitude and longitude’, for example, with an associated syntax for writing these as an R formula object. The software then parses these definitions and applies the relevant bru_mapper() methods to the data to construct the appropriate component design matrices and full model design matrix. In addition to linear effects, inlabru also supports non-linear component effects, e.g. via marginal distribution mappers, that are discussed in Supplement A.1.

In the existing implementation of INLA, for all but the most simple models, users must construct the design matrix with the use of helper functions, in a process that is known as ‘stack building’ (Lindgren and Rue, 2015, Section 2.5). For complex models this process can be quite involved. A key feature of inlabru is that this is now entirely automated, greatly simplifying the model fitting process.

For certain model components, such as the SPDE effect, the design matrix has become known among INLA users as the ‘projector’ matrix, as it is thought of as ‘projecting the GMRF parameters to the observation locations’. In this paper we use the term projector matrix and design matrix interchangeably.

inlabru extends the class of models that can be fitted using INLA by allowing the predictor to be a non-linear function of the parameter vector $\boldsymbol{u}$. The premise for the inlabru implementation for non-linear predictors is to build on the existing INLA implementation using an iterative model fitting scheme, fitting the model using INLA at each iteration. inlabru can therefore be viewed as both a wrapper for, and an extension of, the existing INLA implementation. This iterative approach is an approximate method of inference based on a linearisation step and the properties of the resulting approximation will depend on the nature of the non-linearity and the choice of convergence criteria.

Let $\widetilde{\boldsymbol{\eta}}(\boldsymbol{u})$ denote a non-linear predictor, i.e. a deterministic function of $\boldsymbol{u}$. Choosing some linearisation point $\boldsymbol{u}_{0}$, the 1st order Taylor approximation of $\widetilde{\boldsymbol{\eta}}(\boldsymbol{u})$ at $\boldsymbol{u}_{0}$ can be written as

	$\displaystyle\overline{\boldsymbol{\eta}}(\boldsymbol{u})$	$\displaystyle=\widetilde{\boldsymbol{\eta}}(\boldsymbol{u}_{0})+\sum_{j=1}^{d}\boldsymbol{B}^{(j)}(\boldsymbol{u}^{(j)}-\boldsymbol{u}_{0}^{(j)})=\left[\widetilde{\boldsymbol{\eta}}(\boldsymbol{u}_{0})-\sum_{j}\boldsymbol{B}^{(j)}\boldsymbol{u}_{0}^{(j)}\right]+\sum_{j}\boldsymbol{B}^{(j)}\boldsymbol{u}^{(j)}$
		$\displaystyle=\boldsymbol{\delta}+\sum_{j}\boldsymbol{B}^{(j)}\boldsymbol{u}^{(j)},$

where $\boldsymbol{B}^{(j)}$ are the derivative matrices of the non-linear predictor, with respect to each latent component state vector $\boldsymbol{u}^{(j)}$, evaluated at the linearisation point $\boldsymbol{u}_{0}$. In contrast to the linear predictor case, the offset term $\boldsymbol{\delta}$ is not optional, as it varies with the linearisation point and is therefore typically non-zero.

Generally we define the observation models via

$\displaystyle\boldsymbol{y}|\boldsymbol{u},{\boldsymbol{\theta}}$

$\displaystyle\sim p(\boldsymbol{y}|\widetilde{\boldsymbol{\eta}}(\boldsymbol{u}),{\boldsymbol{\theta}}),$

so that the observation distribution only depends on the latent vector $\boldsymbol{u}$ via the predictor $\widetilde{\eta}(\boldsymbol{u})$. Commonly, a link function transformation is applied to each element of the predictor vector, so that observation $y_{i}$ is linked to $g^{-1}\left[\widetilde{\eta}(\boldsymbol{u})_{i}\right]$. However, for some models, such as inhomogeneous Poisson point processes, further linear or non-linear transformations may also be applied internally, in order to setup the required likelihood construction in for INLA (Yuan et al., 2017).

The non-linear observation model $p(\boldsymbol{y}|\widetilde{\boldsymbol{\eta}}(\boldsymbol{u}),\boldsymbol{\theta})$ is approximated by replacing the non-linear predictor with its linearisation, so that the linearised model is defined by

$$\overline{p}(\boldsymbol{y}|\boldsymbol{u},\boldsymbol{\theta})=p(\boldsymbol{y}|\overline{\boldsymbol{\eta}}(\boldsymbol{u}),\boldsymbol{\theta})\approx p(\boldsymbol{y}|\widetilde{\boldsymbol{\eta}}(\boldsymbol{u}),\boldsymbol{\theta})=\widetilde{p}(\boldsymbol{y}|\boldsymbol{u},\boldsymbol{\theta}).$$

Each such linearised predictor model $\overline{\eta}(\boldsymbol{u})$ has design matrix

$\displaystyle\boldsymbol{B}$

$\displaystyle=\begin{bmatrix}\boldsymbol{B}^{(1)}&\dots&\boldsymbol{B}^{(d)}\end{bmatrix}$

and offset term $\boldsymbol{\delta}=\widetilde{\boldsymbol{\eta}}(\boldsymbol{u}_{0})-\boldsymbol{B}\boldsymbol{u}_{0}$. This defines a model that can be fitted using the existing INLA methodology for LGMs with linear predictors. The next step is to choose a suitable linearisation point. This is done using a fixed point iteration method, which locates a point $\boldsymbol{u}_{0}$ such that the resulting conditional posterior mode $\widehat{\boldsymbol{u}}$ of the linearised model is the same as the linearisation point. Starting at some initial point $\boldsymbol{u}_{0}^{(0)}$, for $k=1,2,\dots$ applying the INLA method, to obtain

	$\displaystyle\widehat{{\boldsymbol{\theta}}}^{(k)}$	$\displaystyle=\operatorname*{arg\,max}_{\boldsymbol{\theta}}\overline{p}({\boldsymbol{\theta}}|\boldsymbol{y};\boldsymbol{u}_{0}^{(k-1)}),$
	$\displaystyle\widehat{\boldsymbol{u}}^{(k)}$	$\displaystyle=\operatorname*{arg\,max}_{\boldsymbol{u}}\overline{p}(\boldsymbol{u}|\boldsymbol{y},\widehat{{\boldsymbol{\theta}}},\boldsymbol{u}_{0}^{(k-1)}),$

and choosing the next linearisation point as the point $\boldsymbol{u}_{0}^{(k)}=(1-\alpha)\boldsymbol{u}_{0}^{(k-1)}+\alpha\widehat{\boldsymbol{u}}^{(k)}$ for some $\alpha>0$ that minimises a norm of the difference between the linearised and non-linear predictors. The iteration continues until convergence is detected. The convergence stop** criteria are when

• •

  a maximum number of iterations have been carried out (the bru_max_iter option, default $10$), or

• •

  the relative maximum change in the linearisation point is below a threshold relative to the estimated posterior standard deviations (bru_method$rel_tol, default 10%), and

• •

  the line search for choosing $\alpha$ is inactive, i.e. $\alpha\approx 1$, indicating that the non-linear and linearised predictors coincide.

For more details on the iterative method, see Supplementary Material Section B.

Whereas the inlabru optimisation method leads to an estimate where $\|\widetilde{\boldsymbol{\eta}}(\boldsymbol{u}_{*})-\overline{\boldsymbol{\eta}}(\boldsymbol{u}_{*})\|=0$ for a specific $\boldsymbol{u}_{*}$, the overall posterior approximation accuracy depends on the degree of nonlinearity in the vicinity of $\boldsymbol{u}_{*}$. There are two main options for evaluating this nonlinearity, that can both be computed using sampling from the approximate posterior distribution. The first option is

$\displaystyle\sum_{i}\frac{E_{\boldsymbol{u}\sim\overline{p}(\boldsymbol{u}|\boldsymbol{y})}\left[|\overline{\boldsymbol{\eta}}_{i}(\boldsymbol{u})-\widetilde{\boldsymbol{\eta}}_{i}(\boldsymbol{u})|^{2}\right]}{\mathrm{Var}_{\boldsymbol{u}\sim\overline{p}(\boldsymbol{u}|\boldsymbol{y})}(\overline{\boldsymbol{\eta}}_{i}(\boldsymbol{u}))},$

which is the posterior expectation of the component-wise variance-normalised squared deviation between the non-linear and linearised predictor. Note that the normalising variance includes the variability induced by the posterior uncertainty for $\boldsymbol{\theta}$, whereas the $\|\cdot\|_{V}$ norm used for the line search used only the posterior mode.

The second option is to approximate the Kullback-Leibler divergences between the conditional posterior distributions under the linear and nonlinear models,

	$\displaystyle\mathsf{D}_{\mathsf{KL}}\left(\overline{p}\,\middle\|\,\widetilde{p}\right)$	$\displaystyle=E_{\boldsymbol{u}\sim\overline{p}(\boldsymbol{u}|\boldsymbol{y},\boldsymbol{\theta})}\left[\ln\frac{\overline{p}(\boldsymbol{u}|\boldsymbol{y},{\boldsymbol{\theta}})}{\widetilde{p}(\boldsymbol{u}|\boldsymbol{y},{\boldsymbol{\theta}})}\right],$		(2)
	$\displaystyle\mathsf{D}_{\mathsf{KL}}\left(\widetilde{p}\,\middle\|\,\overline{p}\right)$	$\displaystyle=E_{\boldsymbol{u}\sim\widetilde{p}(\boldsymbol{u}|\boldsymbol{y},\boldsymbol{\theta})}\left[\ln\frac{\widetilde{p}(\boldsymbol{u}|\boldsymbol{y},{\boldsymbol{\theta}})}{\overline{p}(\boldsymbol{u}|\boldsymbol{y},{\boldsymbol{\theta}})}\right],$		(3)

both evaluated at the posterior mode for $\boldsymbol{\theta}$. While it may be possible to also approximate the K-L divergence for the full posterior distributions, in practice it is easier to evaluate the conditional K-L divergences only at the posterior mode for $\boldsymbol{\theta}$. Implementing this requires access to specific aspects of the likelihood and prior distribution details, that are available from the INLA software package output, and we plan to include an implementation of these K-L divergence as a model diagnostic in a future version of inlabru. Supplementary Material Section F presents this in more detail and includes results to approximate Equations (2) and (3).

Users can also evaluate the accuracy of the method for a given model using general Bayesian method checking approaches, as we show in Section 4.

On a side note, one might be concerned about initialisation at, or convergence to, a saddle point. Although it is not implemented in inlabru, we want to talk about the technicality how we define the initial linearisation point $\boldsymbol{u}_{0}$.

Generally speaking, any values of $\boldsymbol{u}_{0}$ work except the case that the gradient evaluated at $\boldsymbol{u}_{0}$ is $\boldsymbol{0}$ because the linearisation point will never move away if the prior mean is also $\boldsymbol{0}$. In general, this tends to be a saddle point problem. In some cases the problem can be handled by changing the predictor parameterisation or just changing the initialisation point using the bru_initial option. However, for true saddle point problems, it indicates that the predictor parameterisation may lead to a multimodal posterior distribution or is ill-posed in some other way. This is a more fundamental problem that cannot be fixed by changing the initialisation point.

In these examples, where $\beta$ and $\boldsymbol{u}$ are latent Gaussian components, the predictors 1, 3, and 4 would typically be safe, but predictor 2 is fundamentally non-identifiable:

	$\displaystyle\boldsymbol{\eta}_{1}$	$\displaystyle=\boldsymbol{u},$
	$\displaystyle\boldsymbol{\eta}_{2}$	$\displaystyle=\beta\boldsymbol{u},$
	$\displaystyle\boldsymbol{\eta}_{3}$	$\displaystyle=e^{\beta}\boldsymbol{u},$
	$\displaystyle\boldsymbol{\eta}_{4}$	$\displaystyle=F_{\beta}^{-1}(\Phi(z_{\beta}))\boldsymbol{u},\quad z_{\beta}\sim\mathsf{N}(0,1).$

As mentioned above, inlabru provides support for sf (Pebesma, 2018) and terra (Hijmans, 2024) data structures. This means that sf and terra objects can be passed as data and covariates, respectively, directly to the like() and bru() functions. For example, a log-Gaussian Cox process can be fitted using an sf points object to describe the locations of the observed points and an sf polygon object to define the observation window. Spatial objects can also be used for prediction as data objects in predict() and generate() calls.

Some model components can work directly with the spatial information provided in the geometry column of the sf data object. For example, the bru_mapper() method for the SPDE random effect has methods for working with sf geometry objects. The mapper is system is the basis for extending support for spatial data to more model components in the future.

The eval_spatial() methods can be used to extract information from spatial data objects. This is useful for defining components where spatial data is an input. For example, a component specified as

R> ~ my_sp_effect(+    main = a_spatial_object,+    model = "linear"+  ) is equivalent to

R> ~ my_sp_effect(+    main = eval_spatial(a_spatial_object, .data.),+    model = "linear"+  ) where .data. is a keyword that refers to whatever data object is passed to the like() or bru() function. The eval_spatial() function extracts the value of the spatial covariate at the locations provided in the data. See ?eval_spatial for more information on what spatial data formats are supported.

There is also legacy support for the sp and raster packages. For these legacy packages there is support for the plotting of spatial data. The gg() function is a geometry class for ggplot2 (Wickham, 2009) plot objects. It automatically detects the type of sp object (points, lines, pixels, grid or polygons) and coerces this into a correct format for plotting with standard ggplot2 geometry classes. This plotting support is not required for the newer spatial packages which have already been supported in ggplot2 with the geom_sf function, the tidyterra package (Hernangómez, 2023), and the geom_fm function in the fmesher package (Lindgren, 2024).

The Besag-York-Mollié model is an example of an intrinsic random field defined on a graph that represents the neighbourhood structure of the areal units. The BYM component consists of the sum of a spatially structured intrinsic conditional autoregressive (ICAR) field $\boldsymbol{u}$ and a spatially unstructured Gaussian field $\boldsymbol{v}$. i.e. The log rate is $\ln\lambda_{i}=\beta+w_{i}$, where $w_{i}=u_{i}+v_{i}$.

The data object glasgow is an sf polygons object with the response variable named count and a variable named ID that is an integer identifier for each areal unit. The model components and formula are

R> cmp <- ~ 0 + beta(1) + w(ID,+    model = "bym",+    graph = g+  )R>R> fml <- count ~ beta + w

where g is the graph defining the neighbourhood structure. Note that we have chosen the component names beta and w to directly match the mathematical notation above. To match the formula conventions of other packages, inlabru will include an intercept by default unless +0 or -1 is included in the components definition, or there is a component name that begins with the string intercept or Intercept.

To fit and predict from the model:

R> lik <- like(+    formula = fml,+    data = glasgow,+    family = "poisson"+  )R>R> fit <- bru(+    components = cmp,+    lik+  )R>R> predict(+    fit,+    newdata = glasgow,+    formula = ~ exp(beta + w),+    n.samples = 500+  )

Figure 4 shows the mean posterior rate returned by the predict() call alongside the true rate used to simulate the data.

This example demonstrates that the extra flexibility provided by the iterated INLA method allows for models to be fitted that are not possible to fit using standard INLA.

In this example we specify a continuously indexed latent random field using the SPDE approach. We use the term continuously indexed to distinguish from the discrete area models like the BYM model. In this case the index is a spatial location $\bs$ which can be any location in a continuous spatial domain.

This model structure is useful for applications where the response variable is available at a different spatial scale to the assumed causes. For example, one might expect counts of asthma cases in an areal unit to depend on air pollution, which varies continuously in space and does not recognise areal unit boundaries.

The log-rate is approximated using a numerical integration scheme

	$\displaystyle\ln\lambda_{i}$	$\displaystyle=\ln\left(\int_{\Omega_{i}}\exp(\beta+\xi(\boldsymbol{s}))\,\mathrm{d}{}\boldsymbol{s}\right)$
		$\displaystyle\approx\ln\left[\sum_{j}w_{ij}\exp(\beta+\xi(\boldsymbol{s}_{ij}))\right]$

with weights $w_{ij}$ defined using projected integration weights to mesh nodes. In order to implement this, one could store these weights in a matrix $\boldsymbol{W}$ such that $\boldsymbol{\lambda}\approx\boldsymbol{W}\exp(\beta+\boldsymbol{\xi})$, using a predictor expression written as log(W %*% exp(Intercept + field)). However, this procedure is already encapsulated in the bru_mapper_logsumexp mapper, so that the $\boldsymbol{W}$ matrix is constructed automatically and doesn’t need to be visible in the user code. In addition, the mapper uses an internal weight shift to avoid numerical over/underflow when summing the terms. The code to construct the integration scheme can be found in Supplementary Material Section E.

Given this integration scheme, the code to specify and fit the model is

R> # Define Matern/SPDE modelR> matern <- inla.spde2.pcmatern(+    mesh,+    prior.range = c(2, 0.1),+    prior.sigma = c(3, 0.1)+  )R>R> # Define the intermediate zone integration schemeR> ips <- fm_int(mesh, samplers = glasgow)R> agg <- bru_mapper_logsumexp(+    rescale = FALSE,+    n_block = nrow(glasgow)+  )R>R> # Define model components and formulaR> cmp <- ~ 0 + beta(1) + xi(main = geometry, model = matern)R> fml <- count ~ ibm_eval(+    agg,+    input = list(weights = weight, block = .block),+    state = beta + xi+  )R>R> # Construct the likelihoodR> lik <- like(+    formula = fml,+    family = "poisson",+    data = ips,+    response_data = glasgow+  )R>R> # Fit the modelR> fit <- bru(+    components = cmp,+    lik+  ) In this example we make use of the response_data argument of the like() function, which allows users to specify response data of different length to what would usually be expected. This feature of inlabru allows covariates and latent parameters to be on a different scale to the observed data. The non-linear predictor function provides the link between these different scales. In this case the link is the numerical integration of the latent field within each intermediate zone where ibm_eval(agg, ...) results in a vector of length $n$ that matches the response data.

The xi component is evaluated at the locations provided in ips, which define the integration scheme. Note that this is an sf object in this example, and the xi component was defined using geometry as input. If geometry is not a named variable then the software assumes it is a general R expression to be applied to the data. In this case geometry is the geometry information column of the sf data object, which is supported by the bru_mapper() method for the matern model object, that constructs the model matrix for the component. For old SpatialPointsDataFrame objects, one would instead use xi(coordinates, ...), where coordinates is the sp function that extracts a 2-column matrix of coordinates from the data object. However, this would drop all coordinate system projection information, which is retained in the sf object, and so we recommend using sf objects where possible to avoid associating data with incorrect model locations.

The mapper bru_mapper_logsumexp here defines the link from the state vectors to the effects vector. In other words, it is the logarithms of the weighted sums of the element-wise exponential of the state vectors beta and xi, taking into account of the input vectors weight and .block. .block argument gives the block index vector from the fm_int function. By default, the rescale = FALSE argument gives the plain weighted sums. With rescale = TRUE, the sum is rescaled by the sum of the weights within each block, as

$$\ln\left[\frac{\sum_{j}w_{ij}\exp\big{(}\beta+\xi(\boldsymbol{s}_{ij})\big{)}}{\sum_{j}w_{ij}}\right].$$

The n_block argument specifies a predetermined number of output blocks. For more information, please see the theory and technical documentation on bru_mapper system (Lindgren, 2022).

This example uses the same aggregation scheme as in the above but also assumes that we have additional data observing the latent random field at a set of point locations. This data structure can be common where information on relevant covariates is only available at a finite set of locations in the region of interest, such as measures of air pollution at air pollution monitoring stations, for example. In addition to the count data at the areal unit level, we have data $z_{k}$ for $k=1,\ldots,K$, and we assume the following model:

	$\displaystyle z_{k}$	$\displaystyle=\alpha_{0}+\xi(\boldsymbol{s}_{k})+\epsilon_{k}$
	$\displaystyle\ln\lambda(\boldsymbol{s})$	$\displaystyle=\beta_{0}+\beta_{1}\xi(\boldsymbol{s})$
	$\displaystyle\lambda_{i}$	$\displaystyle\approx\sum_{j}\ln\left(w_{ij}\lambda(\boldsymbol{s}_{ij})\right),$

where $\alpha_{0}$ is an intercept parameter which could account for systematic measurement or calibration error involved in measuring the random field $\xi(\boldsymbol{s})$ and the $w_{ij}$ are defined as above. We do not include $\alpha_{0}$ to the second line to avoid identifiability issues. The likelihood for $\boldsymbol{z}$ is Gaussian which also incorporates iid measurement error. In this example we view the random field as though it were a fixed effect covariate that we have only observed in a finite set of locations. This partially observed covariate, which we model using the SPDE effect, affects the continuously indexed log-rate linearly through the parameter $\beta_{1}$.

The code to specify and fit the model is

R> # Define model componentsR> cmp <- ~ 0 + xi(main = geometry, model = matern) ++    alpha_0(1) ++    beta_0(1) ++    beta_1(1)R>R> # Gaussian likelihood for z observationsR> z_fml <- z ~ alpha_0 + xiR> z_lik <- like(+    "Gaussian",+    formula = z_fml,+    data = z_data+  )R>R> # Poisson likelihood for count observationsR> count_fml <- count ~ ibm_eval(+    agg,+    input = list(weights = weight, block = .block),+    state = beta_0 + beta_1 * xi+  )R> count_lik <- like(+    "Poisson",+    formula = count_fml,+    data = ips,+    response_data = glasgow+  )R>R> # Fit joint likelihood modelR> fit <- bru(+    components = cmp,+    z_lik,+    count_lik,+    options = list(+      bru_max_iter = 10,+      bru_verbose = 2,+  #    bru_initial = list(+  #      beta_1 = 1+  #    )+    )+  ) where z_data is an sf object with a column named z. Predictions can be generated from the model in a similar way to the previous examples. The methods will normally auto-detect which effects are involved in the predictor expressions, but if needed, the default auto-detection can be overridden using the used argument. We can optionally include the bru_initial option to set the initial values for the latent variables, to potentially speed up the method convergence, see Figure 7. We believe that the random field $\xi(\boldsymbol{s})$ would contribute positive effect on the intensity $\lambda(\boldsymbol{s})$. The bru_verbose option controls the amount of information about the iterative algorithm that is printed by bru(). The verbose information for this model is included in Supplementary Material Section G.

A seen by the convergence diagnostic plot in Figure 7, the iterated linearisation method converges in 6 steps, with the final 7th step only computing the posterior integration step of the INLA method.

The posterior mean for $\lambda(\boldsymbol{s})$ and for $\xi(\boldsymbol{s})$ are shown in Figure 8, alongside the truth and the latent field observations.

The iterative INLA approach extends the class of models that can be fitted using INLA. Other research has investigated this by embedding INLA within other inference frameworks such as Metropolis Hastings Markov chain Monte Carlo (MH-MCMC) (Gómez-Rubio and Rue, 2018) and importance sampling (Berild et al., 2022). These approaches allow users to fit conditional latent Gaussian models, where the model is, conditional on certain parameters, able to be fitted using standard INLA.

In the case of MH-MCMC, this comes at considerable extra computational cost. Typical chains can require fitting a model using INLA tens or even hundreds of thousands of times, which is prohibitively expensive for all but the most simple of models. This computational cost has hindered embedding INLA within MCMC approaches. In our experience so far, iterative INLA generally requires something between a couple and twenty iterations to converge, with simpler models often converging in 3 or 4 steps.

The class of models that can be fitted using inlabru is smaller than the class of conditional LGMs because only the latent predictor expression is allowed to be non-linear. However, the data likelihood must still be one of those that is currently implemented in INLA. In contrast, the MH-MCMC approach is, in theory at least, not restricted in this sense.

An alternative to MH-MCMC is to use importance sampling (IS) to marginalise over the conditioning parameters for the conditional LGM. Let $\theta_{c}$ denote the parameters conditional on which the other model parameters, $\theta_{-c}$, can be estimated using INLA. Then the unconditional marginal posterior distributions for elements of $\theta_{c}$ can be obtained by

$$\pi(\theta_{-c,j})|\boldsymbol{y})=\int\pi(\theta_{-c,j}|\boldsymbol{y},\theta_{c})\pi(\theta_{c}|\boldsymbol{y})\,\mathrm{d}{}\theta_{c}.$$

Berild et al. (2022) present estimating this integral using IS. The importance sampling weight can also be used to estimate the joint posterior $\pi(\theta_{c}|\boldsymbol{y})$. The performance of IS is sensitive to the choice of proposal distribution. The greater the difference from $\pi(\theta_{c}|\boldsymbol{y})$, the more samples are required. To reduce the risk of a poor choice, Berild et al. (2022) suggest using an adaptive IS algorithm. This approach appears more promising than MH-MCMC, with examples showing good convergence to true posterior in under 30 adaptations of the proposal distribution. However, the examples in their paper are more simple than those presented here.