Showing 1–32 of 32 results for author: Guinness, J

Implementation and Analysis of GPU Algorithms for Vecchia Approximation

Authors: Zachary James, Joseph Guinness

Abstract: Gaussian Processes have become an indispensable part of the spatial statistician's toolbox but are unsuitable for analyzing large dataset because of the significant time and memory needed to fit the associated model exactly. Vecchia Approximation is widely used to reduce the computational complexity and can be calculated with embarrassingly parallel algorithms. While multi-core software has been d… ▽ More Gaussian Processes have become an indispensable part of the spatial statistician's toolbox but are unsuitable for analyzing large dataset because of the significant time and memory needed to fit the associated model exactly. Vecchia Approximation is widely used to reduce the computational complexity and can be calculated with embarrassingly parallel algorithms. While multi-core software has been developed for Vecchia Approximation, such as the GpGp R package, software designed to run on graphics processing units (GPU) is lacking, despite the tremendous success GPUs have had in statistics and machine learning. We compare three different ways to implement Vecchia Approximation on a GPU: two of which are similar to methods used for other Gaussian Process approximations and one that is new. The impact of memory type on performance is investigated and the final method is optimized accordingly. We show that our new method outperforms the other two and then present it in the GpGpU R package. We compare GpGpU to existing multi-core and GPU-accelerated software by fitting Gaussian Process models on various datasets, including a large spatial-temporal dataset of $n>10^6$ points collected from an earth-observing satellite. Our results show that GpGpU achieves faster runtimes and better predictive accuracy. △ Less

Submitted 2 July, 2024; originally announced July 2024.

Asymptotic properties of Vecchia approximation for Gaussian processes

Authors: Myeongjong Kang, Florian Schäfer, Joseph Guinness, Matthias Katzfuss

Abstract: Vecchia approximation has been widely used to accurately scale Gaussian-process (GP) inference to large datasets, by expressing the joint density as a product of conditional densities with small conditioning sets. We study fixed-domain asymptotic properties of Vecchia-based GP inference for a large class of covariance functions (including Matérn covariances) with boundary conditioning. In this set… ▽ More Vecchia approximation has been widely used to accurately scale Gaussian-process (GP) inference to large datasets, by expressing the joint density as a product of conditional densities with small conditioning sets. We study fixed-domain asymptotic properties of Vecchia-based GP inference for a large class of covariance functions (including Matérn covariances) with boundary conditioning. In this setting, we establish that consistency and asymptotic normality of maximum exact-likelihood estimators imply those of maximum Vecchia-likelihood estimators, and that exact GP prediction can be approximated accurately by Vecchia GP prediction, given that the size of conditioning sets grows polylogarithmically with the data size. Hence, Vecchia-based inference with quasilinear complexity is asymptotically equivalent to exact GP inference with cubic complexity. This also provides a general new result on the screening effect. Our findings are illustrated by numerical experiments, which also show that Vecchia approximation can be more accurate than alternative approaches such as covariance tapering and reduced-rank approximations. △ Less

Submitted 5 February, 2024; v1 submitted 28 January, 2024; originally announced January 2024.

Comments: 24 pages, 6 figures

Sparse Cholesky factorization by greedy conditional selection

Authors: Stephen Huan, Joseph Guinness, Matthias Katzfuss, Houman Owhadi, Florian Schäfer

Abstract: Dense kernel matrices resulting from pairwise evaluations of a kernel function arise naturally in machine learning and statistics. Previous work in constructing sparse approximate inverse Cholesky factors of such matrices by minimizing Kullback-Leibler divergence recovers the Vecchia approximation for Gaussian processes. These methods rely only on the geometry of the evaluation points to construct… ▽ More Dense kernel matrices resulting from pairwise evaluations of a kernel function arise naturally in machine learning and statistics. Previous work in constructing sparse approximate inverse Cholesky factors of such matrices by minimizing Kullback-Leibler divergence recovers the Vecchia approximation for Gaussian processes. These methods rely only on the geometry of the evaluation points to construct the sparsity pattern. In this work, we instead construct the sparsity pattern by leveraging a greedy selection algorithm that maximizes mutual information with target points, conditional on all points previously selected. For selecting $k$ points out of $N$, the naive time complexity is $\mathcal{O}(N k^4)$, but by maintaining a partial Cholesky factor we reduce this to $\mathcal{O}(N k^2)$. Furthermore, for multiple ($m$) targets we achieve a time complexity of $\mathcal{O}(N k^2 + N m^2 + m^3)$, which is maintained in the setting of aggregated Cholesky factorization where a selected point need not condition every target. We apply the selection algorithm to image classification and recovery of sparse Cholesky factors. By minimizing Kullback-Leibler divergence, we apply the algorithm to Cholesky factorization, Gaussian process regression, and preconditioning with the conjugate gradient, improving over $k$-nearest neighbors selection. △ Less

Submitted 21 July, 2023; originally announced July 2023.

MSC Class: 65F08; 65F55; 62-08

Vecchia Approximations and Optimization for Multivariate Matérn Models

Authors: Youssef Fahmy, Joseph Guinness

Abstract: We describe our implementation of the multivariate Matérn model for multivariate spatial datasets, using Vecchia's approximation and a Fisher scoring optimization algorithm. We consider various pararameterizations for the multivariate Matérn that have been proposed in the literature for ensuring model validity, as well as an unconstrained model. A strength of our study is that the code is tested o… ▽ More We describe our implementation of the multivariate Matérn model for multivariate spatial datasets, using Vecchia's approximation and a Fisher scoring optimization algorithm. We consider various pararameterizations for the multivariate Matérn that have been proposed in the literature for ensuring model validity, as well as an unconstrained model. A strength of our study is that the code is tested on many real-world multivariate spatial datasets. We use it to study the effect of ordering and conditioning in Vecchia's approximation and the restrictions imposed by the various parameterizations. We also consider a model in which co-located nuggets are correlated across components and find that forcing this cross-component nugget correlation to be zero can have a serious impact on the other model parameters, so we suggest allowing cross-component correlation in co-located nugget terms. △ Less

Submitted 18 October, 2022; v1 submitted 17 October, 2022; originally announced October 2022.

A Gaussian-process approximation to a spatial SIR process using moment closures and emulators

Authors: Parker Trostle, Joseph Guinness, Brian J. Reich

Abstract: The dynamics that govern disease spread are hard to model because infections are functions of both the underlying pathogen as well as human or animal behavior. This challenge is increased when modeling how diseases spread between different spatial locations. Many proposed spatial epidemiological models require trade-offs to fit, either by abstracting away theoretical spread dynamics, fitting a det… ▽ More The dynamics that govern disease spread are hard to model because infections are functions of both the underlying pathogen as well as human or animal behavior. This challenge is increased when modeling how diseases spread between different spatial locations. Many proposed spatial epidemiological models require trade-offs to fit, either by abstracting away theoretical spread dynamics, fitting a deterministic model, or by requiring large computational resources for many simulations. We propose an approach that approximates the complex spatial spread dynamics with a Gaussian process. We first propose a flexible spatial extension to the well-known SIR stochastic process, and then we derive a moment-closure approximation to this stochastic process. This moment-closure approximation yields ordinary differential equations for the evolution of the means and covariances of the susceptibles and infectious through time. Because these ODEs are a bottleneck to fitting our model by MCMC, we approximate them using a low-rank emulator. This approximation serves as the basis for our hierarchical model for noisy, underreported counts of new infections by spatial location and time. We demonstrate using our model to conduct inference on simulated infections from the underlying, true spatial SIR jump process. We then apply our method to model counts of new Zika infections in Brazil from late 2015 through early 2016. △ Less

Submitted 5 August, 2022; originally announced August 2022.

Comparison of CYGNSS and Jason-3 Wind Speed Measurements via Gaussian Processes

Authors: William Bekerman, Joseph Guinness

Abstract: Wind is a critical component of the Earth system and has unmistakable impacts on everyday life. The CYGNSS satellite mission improves observational coverage of ocean winds via a fleet of eight micro-satellites that use reflected GNSS signals to infer surface wind speed. We present analyses characterizing variability in wind speed measurements among the eight CYGNSS satellites and between antennas.… ▽ More Wind is a critical component of the Earth system and has unmistakable impacts on everyday life. The CYGNSS satellite mission improves observational coverage of ocean winds via a fleet of eight micro-satellites that use reflected GNSS signals to infer surface wind speed. We present analyses characterizing variability in wind speed measurements among the eight CYGNSS satellites and between antennas. In particular, we use a carefully constructed Gaussian process model that leverages comparisons between CYGNSS and Jason-3 during a one-year period from September 2019 to September 2020. The CYGNSS sensors exhibit a range of biases, most of them between -1.0 m/s and +0.2 m/s with respect to Jason-3, indicating that some CYGNSS sensors are biased with respect to one another and with respect to Jason-3. The biases between the starboard and port antennas within a CYGNSS satellite are smaller. Our results are consistent with, yet sharper than, a more traditional paired comparison analysis. We also explore the possibility that the bias depends on wind speed, finding some evidence that CYGNSS satellites have positive biases with respect to Jason-3 at low wind speeds. However, we argue that there are subtle issues associated with estimating wind speed-dependent biases, so additional careful statistical modeling and analysis is warranted. △ Less

Submitted 7 March, 2022; originally announced March 2022.

Scalable Gaussian-process regression and variable selection using Vecchia approximations

Authors: Jian Cao, Joseph Guinness, Marc G. Genton, Matthias Katzfuss

Abstract: Gaussian process (GP) regression is a flexible, nonparametric approach to regression that naturally quantifies uncertainty. In many applications, the number of responses and covariates are both large, and a goal is to select covariates that are related to the response. For this setting, we propose a novel, scalable algorithm, coined VGPR, which optimizes a penalized GP log-likelihood based on the… ▽ More Gaussian process (GP) regression is a flexible, nonparametric approach to regression that naturally quantifies uncertainty. In many applications, the number of responses and covariates are both large, and a goal is to select covariates that are related to the response. For this setting, we propose a novel, scalable algorithm, coined VGPR, which optimizes a penalized GP log-likelihood based on the Vecchia GP approximation, an ordered conditional approximation from spatial statistics that implies a sparse Cholesky factor of the precision matrix. We traverse the regularization path from strong to weak penalization, sequentially adding candidate covariates based on the gradient of the log-likelihood and deselecting irrelevant covariates via a new quadratic constrained coordinate descent algorithm. We propose Vecchia-based mini-batch subsampling, which provides unbiased gradient estimators. The resulting procedure is scalable to millions of responses and thousands of covariates. Theoretical analysis and numerical studies demonstrate the improved scalability and accuracy relative to existing methods. △ Less

Submitted 10 October, 2022; v1 submitted 25 February, 2022; originally announced February 2022.

Comments: 30 pages, 9 figures

Log-Gaussian Cox Process Modeling of Large Spatial Lightning Data using Spectral and Laplace Approximations

Authors: Megan L. Gelsinger, Maryclare Griffin, David S. Matteson, Joseph Guinness

Abstract: Lightning is a destructive and highly visible product of severe storms, yet there is still much to be learned about the conditions under which lightning is most likely to occur. The GOES-16 and GOES-17 satellites, launched in 2016 and 2018 by NOAA and NASA, collect a wealth of data regarding individual lightning strike occurrence and potentially related atmospheric variables. The acute nature and… ▽ More Lightning is a destructive and highly visible product of severe storms, yet there is still much to be learned about the conditions under which lightning is most likely to occur. The GOES-16 and GOES-17 satellites, launched in 2016 and 2018 by NOAA and NASA, collect a wealth of data regarding individual lightning strike occurrence and potentially related atmospheric variables. The acute nature and inherent spatial correlation in lightning data renders standard regression analyses inappropriate. Further, computational considerations are foregrounded by the desire to analyze the immense and rapidly increasing volume of lightning data. We present a new computationally feasible method that combines spectral and Laplace approximations in an EM algorithm, denoted SLEM, to fit the widely popular log-Gaussian Cox process model to large spatial point pattern datasets. In simulations, we find SLEM is competitive with contemporary techniques in terms of speed and accuracy. When applied to two lightning datasets, SLEM provides better out-of-sample prediction scores and quicker runtimes, suggesting its particular usefulness for analyzing lightning data, which tend to have sparse signals. △ Less

Submitted 30 November, 2021; originally announced November 2021.

Ordered conditional approximation of Potts models

Authors: Anirban Chakraborty, Matthias Katzfuss, Joseph Guinness

Abstract: Potts models, which can be used to analyze dependent observations on a lattice, have seen widespread application in a variety of areas, including statistical mechanics, neuroscience, and quantum computing. To address the intractability of Potts likelihoods for large spatial fields, we propose fast ordered conditional approximations that enable rapid inference for observed and hidden Potts models.… ▽ More Potts models, which can be used to analyze dependent observations on a lattice, have seen widespread application in a variety of areas, including statistical mechanics, neuroscience, and quantum computing. To address the intractability of Potts likelihoods for large spatial fields, we propose fast ordered conditional approximations that enable rapid inference for observed and hidden Potts models. Our methods can be used to directly obtain samples from the approximate joint distribution of an entire Potts field. The computational complexity of our approximation methods is linear in the number of spatial locations; in addition, some of the necessary computations are naturally parallel. We illustrate the advantages of our approach using simulated data and a satellite image. △ Less

Submitted 13 October, 2021; originally announced October 2021.

Scaled Vecchia approximation for fast computer-model emulation

Authors: Matthias Katzfuss, Joseph Guinness, Earl Lawrence

Abstract: Many scientific phenomena are studied using computer experiments consisting of multiple runs of a computer model while varying the input settings. Gaussian processes (GPs) are a popular tool for the analysis of computer experiments, enabling interpolation between input settings, but direct GP inference is computationally infeasible for large datasets. We adapt and extend a powerful class of GP met… ▽ More Many scientific phenomena are studied using computer experiments consisting of multiple runs of a computer model while varying the input settings. Gaussian processes (GPs) are a popular tool for the analysis of computer experiments, enabling interpolation between input settings, but direct GP inference is computationally infeasible for large datasets. We adapt and extend a powerful class of GP methods from spatial statistics to enable the scalable analysis and emulation of large computer experiments. Specifically, we apply Vecchia's ordered conditional approximation in a transformed input space, with each input scaled according to how strongly it relates to the computer-model response. The scaling is learned from the data, by estimating parameters in the GP covariance function using Fisher scoring. Our methods are highly scalable, enabling estimation, joint prediction and simulation in near-linear time in the number of model runs. In several numerical examples, our approach substantially outperformed existing methods. △ Less

Submitted 20 July, 2021; v1 submitted 1 May, 2020; originally announced May 2020.

Comments: R code available at https://github.com/katzfuss-group/scaledVecchia

An Observational Study of the Effect of Nike Vaporfly Shoes on Marathon Performance

Authors: Joseph Guinness, Debasmita Bhattacharya, Jenny Chen, Max Chen, Angela Loh

Abstract: We collected marathon performance data from a systematic sample of elite and sub-elite athletes over the period 2015 to 2019, then searched the internet for publicly-available photographs of these performances, identifying whether the Nike Vaporfly shoes were worn or not in each performance. Controlling for athlete ability and race difficulty, we estimated the effect on marathon times of wearing t… ▽ More We collected marathon performance data from a systematic sample of elite and sub-elite athletes over the period 2015 to 2019, then searched the internet for publicly-available photographs of these performances, identifying whether the Nike Vaporfly shoes were worn or not in each performance. Controlling for athlete ability and race difficulty, we estimated the effect on marathon times of wearing the Vaporfly shoes. Assuming that the effect of Vaporfly shoes is additive, we estimate that the Vaporfly shoes improve men's times between 2.0 and 3.9 minutes, while they improve women's times between 0.8 and 3.5 minutes. Assuming that the effect of Vaporfly shoes is multiplicative, we estimate that they improve men's times between 1.4 and 2.8 percent and women's performances between 0.6 and 2.2 percent. The improvements are in comparison to the shoe the athlete was wearing before switching to Vaporfly shoes, and represents an expected improvement rather than a guaranteed improvement. △ Less

Submitted 2 October, 2020; v1 submitted 14 February, 2020; originally announced February 2020.

Inverses of Matern Covariances on Grids

Authors: Joseph Guinness

Abstract: We conduct a study of the aliased spectral densities of Matérn covariance functions on a regular grid of points, providing clarity on the properties of a popular approximation based on stochastic partial differential equations; while others have shown that it can approximate the covariance function well, we find that it assigns too much power at high frequencies and does not provide increasingly a… ▽ More We conduct a study of the aliased spectral densities of Matérn covariance functions on a regular grid of points, providing clarity on the properties of a popular approximation based on stochastic partial differential equations; while others have shown that it can approximate the covariance function well, we find that it assigns too much power at high frequencies and does not provide increasingly accurate approximations to the inverse as the grid spacing goes to zero, except in the one-dimensional exponential covariance case. We provide numerical results to support our theory, and in a simulation study, we investigate the implications for parameter estimation, finding that the SPDE approximation tends to overestimate spatial range parameters. △ Less

Submitted 1 March, 2021; v1 submitted 26 December, 2019; originally announced December 2019.

Mean-dependent nonstationary spatial models

Authors: Geoffrey Colin Lee Peterson, Joseph Guinness, Adam Terando, Brian J. Reich

Abstract: Nonstationarity is a major challenge in analyzing spatial data. For example, daily precipitation measurements may have increased variability and decreased spatial smoothness in areas with high mean rainfall. Common nonstationary covariance models introduce parameters specific to each location, giving a highly-parameterized model which is difficult to fit. We develop a nonstationary spatial model t… ▽ More Nonstationarity is a major challenge in analyzing spatial data. For example, daily precipitation measurements may have increased variability and decreased spatial smoothness in areas with high mean rainfall. Common nonstationary covariance models introduce parameters specific to each location, giving a highly-parameterized model which is difficult to fit. We develop a nonstationary spatial model that uses the mean to determine the covariance in a region, resulting in a far simpler, albeit more specialized, model. We explore inferential and predictive properties of the model under various simulated data situations. We show that this model in certain circumstances improves predictions compared to a standard stationary spatial model. We further propose a computationally efficient approximation that has comparable predictive accuracy. We also develop a test for nonstationary data and show it reliably identifies nonstationarity. We apply these methods to daily precipitation in Puerto Rico. △ Less

Submitted 29 May, 2019; originally announced May 2019.

Gaussian Process Learning via Fisher Scoring of Vecchia's Approximation

Authors: Joseph Guinness

Abstract: We derive a single pass algorithm for computing the gradient and Fisher information of Vecchia's Gaussian process loglikelihood approximation, which provides a computationally efficient means for applying the Fisher scoring algorithm for maximizing the loglikelihood. The advantages of the optimization techniques are demonstrated in numerical examples and in an application to Argo ocean temperature… ▽ More We derive a single pass algorithm for computing the gradient and Fisher information of Vecchia's Gaussian process loglikelihood approximation, which provides a computationally efficient means for applying the Fisher scoring algorithm for maximizing the loglikelihood. The advantages of the optimization techniques are demonstrated in numerical examples and in an application to Argo ocean temperature data. The new methods are more accurate and much faster than an optimization method that uses only function evaluations, especially when the covariance function has many parameters. This allows practitioners to fit nonstationary models to large spatial and spatial-temporal datasets. △ Less

Submitted 20 May, 2019; originally announced May 2019.

Smooth Density Spatial Quantile Regression

Authors: Halley Brantley, Montserrat Fuentes, Joseph Guinness, Eben Thoma

Abstract: We derive the properties and demonstrate the desirability of a model-based method for estimating the spatially-varying effects of covariates on the quantile function. By modeling the quantile function as a combination of I-spline basis functions and Pareto tail distributions, we allow for flexible parametric modeling of the extremes while preserving non-parametric flexibility in the center of the… ▽ More We derive the properties and demonstrate the desirability of a model-based method for estimating the spatially-varying effects of covariates on the quantile function. By modeling the quantile function as a combination of I-spline basis functions and Pareto tail distributions, we allow for flexible parametric modeling of the extremes while preserving non-parametric flexibility in the center of the distribution. We further establish that the model guarantees the desired degree of differentiability in the density function and enables the estimation of non-stationary covariance functions dependent on the predictors. We demonstrate through a simulation study that the proposed method produces more efficient estimates of the effects of predictors than other methods, particularly in distributions with heavy tails. To illustrate the utility of the model we apply it to measurements of benzene collected around an oil refinery to determine the effect of an emission source within the refinery on the distribution of the fence line measurements. △ Less

Submitted 30 April, 2019; originally announced May 2019.

Comments: Submitted to Statistica Sinica

Baseline Drift Estimation for Air Quality Data Using Quantile Trend Filtering

Authors: Halley L. Brantley, Joseph Guinness, Eric C. Chi

Abstract: We address the problem of estimating smoothly varying baseline trends in time series data. This problem arises in a wide range of fields, including chemistry, macroeconomics, and medicine; however, our study is motivated by the analysis of data from low cost air quality sensors. Our methods extend the quantile trend filtering framework to enable the estimation of multiple quantile trends simultane… ▽ More We address the problem of estimating smoothly varying baseline trends in time series data. This problem arises in a wide range of fields, including chemistry, macroeconomics, and medicine; however, our study is motivated by the analysis of data from low cost air quality sensors. Our methods extend the quantile trend filtering framework to enable the estimation of multiple quantile trends simultaneously while ensuring that the quantiles do not cross. To handle the computational challenge posed by very long time series, we propose a parallelizable alternating direction method of moments (ADMM) algorithm. The ADMM algorthim enables the estimation of trends in a piecewise manner, both reducing the computation time and extending the limits of the method to larger data sizes. We also address smoothing parameter selection and propose a modified criterion based on the extended Bayesian Information Criterion. Through simulation studies and our motivating application to low cost air quality sensor data, we demonstrate that our model provides better quantile trend estimates than existing methods and improves signal classification of low-cost air quality sensor output. △ Less

Submitted 23 April, 2019; originally announced April 2019.

Geostatistical Modeling of Positive Definite Matrices: An Application to Diffusion Tensor Imaging

Authors: Zhou Lan, Brian J. Reich, Joseph Guinness, Dipankar Bandyopadhyay, Liangsuo Ma, F. Gerard Moeller

Abstract: Geostatistical modeling for continuous point-referenced data has been extensively applied to neuroimaging because it produces efficient and valid statistical inference. However, diffusion tensor imaging (DTI), a neuroimaging characterizing the brain structure produces a positive definite (p.d.) matrix for each voxel. Current geostatistical modeling has not been extended to p.d. matrices because in… ▽ More Geostatistical modeling for continuous point-referenced data has been extensively applied to neuroimaging because it produces efficient and valid statistical inference. However, diffusion tensor imaging (DTI), a neuroimaging characterizing the brain structure produces a positive definite (p.d.) matrix for each voxel. Current geostatistical modeling has not been extended to p.d. matrices because introducing spatial dependence among positive definite matrices properly is challenging. In this paper, we use the spatial Wishart process, a spatial stochastic process (random field) where each p.d. matrix-variate marginally follows a Wishart distribution, and spatial dependence between random matrices is induced by latent Gaussian processes. This process is valid on an uncountable collection of spatial locations and is almost surely continuous, leading to a reasonable means of modeling spatial dependence. Motivated by a DTI dataset of cocaine users, we propose a spatial matrix-variate regression model based on the spatial Wishart process. A problematic issue is that the spatial Wishart process has no closed-form density function. Hence, we propose approximation methods to obtain a feasible working model. A local likelihood approximation method is also applied to achieve fast computation. The simulation studies and real data analysis demonstrate that the working model produces reliable inference and improved performance compared to other methods. △ Less

Submitted 13 June, 2019; v1 submitted 8 April, 2019; originally announced April 2019.

Estimating Atmospheric Motion Winds from Satellite Image Data using Space-time Drift Models

Authors: Indranil Sahoo, Joseph Guinness, Brian J. Reich

Abstract: Geostationary satellites collect high-resolution weather data comprising a series of images which can be used to estimate wind speed and direction at different altitudes. The Derived Motion Winds (DMW) Algorithm is commonly used to process these data and estimate atmospheric winds by tracking features in images taken by the GOES-R series of the NOAA geostationary meteorological satellites. However… ▽ More Geostationary satellites collect high-resolution weather data comprising a series of images which can be used to estimate wind speed and direction at different altitudes. The Derived Motion Winds (DMW) Algorithm is commonly used to process these data and estimate atmospheric winds by tracking features in images taken by the GOES-R series of the NOAA geostationary meteorological satellites. However, the wind estimates from the DMW Algorithm are sparse and do not come with uncertainty measures. This motivates us to statistically model wind motions as a spatial process drifting in time. We propose a covariance function that depends on spatial and temporal lags and a drift parameter to capture the wind speed and wind direction. We estimate the parameters by local maximum likelihood. Our method allows us to compute standard errors of the estimates, enabling spatial smoothing of the estimates using a Gaussian kernel weighted by the inverses of the estimated variances. We conduct extensive simulation studies to determine the situations where our method performs well. The proposed method is applied to the GOES-15 brightness temperature data over Colorado and reduces prediction error of brightness temperature compared to the DMW Algorithm. △ Less

Submitted 15 June, 2021; v1 submitted 25 February, 2019; originally announced February 2019.

Nonparametric Spectral Methods for Multivariate Spatial and Spatial-Temporal Data

Authors: Joseph Guinness

Abstract: We propose computationally efficient methods for estimating stationary multivariate spatial and spatial-temporal spectra from incomplete gridded data. The methods are iterative and rely on successive imputation of data and updating of model estimates. Imputations are done according to a periodic model on an expanded domain. The periodicity of the imputations is a key feature that reduces edge effe… ▽ More We propose computationally efficient methods for estimating stationary multivariate spatial and spatial-temporal spectra from incomplete gridded data. The methods are iterative and rely on successive imputation of data and updating of model estimates. Imputations are done according to a periodic model on an expanded domain. The periodicity of the imputations is a key feature that reduces edge effects in the periodogram and is facilitated by efficient circulant embedding techniques. In addition, we describe efficient methods for decomposing the estimated cross spectral density function into a linear model of coregionalization plus a residual process. The methods are applied to two storm datasets, one of which is from Hurricane Florence, which struck the souteastern United States in September 2018. The application demonstrates how fitted models from different datasets can be compared, and how the methods are computationally feasible on datasets with more than 200,000 total observations. △ Less

Submitted 3 November, 2018; originally announced November 2018.

Non-Stationary Covariance Estimation using the Stochastic Score Approximation for Large Spatial Data

Authors: Amanda Muyskens, Joseph Guinness, Montserrat Fuentes

Abstract: We introduce computational methods that allow for effective estimation of a flexible, parametric non-stationary spatial model when the field size is too large to compute the multivariate normal likelihood directly. In this method, the field is defined as a weighted spatially varying linear combination of a globally stationary process and locally stationary processes. Often in such a model, the dif… ▽ More We introduce computational methods that allow for effective estimation of a flexible, parametric non-stationary spatial model when the field size is too large to compute the multivariate normal likelihood directly. In this method, the field is defined as a weighted spatially varying linear combination of a globally stationary process and locally stationary processes. Often in such a model, the difficulty in its practical use is in the definition of the boundaries for the local processes, and therefore we describe one such selection procedure that generally captures complex non-stationary relationships. We generalize the use of stochastic approximation to the score equations for data on a partial grid in this non-stationary case and provide tools for evaluating the approximate score in $O(n\log n)$ operations and $O(n)$ storage. We perform various simulations to explore the effectiveness and speed of the proposed methods and conclude by making inference on the accumulation behavior of arsenic applied to a sand grain. △ Less

Submitted 18 September, 2018; originally announced September 2018.

Multivariate Spatial-Temporal Variable Selection with Applications to Seasonal Tropical Cyclone Modeling

Authors: Marcela Alfaro Córdoba, Montserrat Fuentes, Joseph Guinness, Lian Xie

Abstract: Tropical cyclone and sea surface temperature data have been used in several studies to forecast the total number of hurricanes in the Atlantic Basin. Sea surface temperature (SST) and latent heat flux (LHF) are correlated with tropical cyclone occurrences, but this correlation is known to vary with location and strength of the storm. The objective of this article is to identify features of SST and… ▽ More Tropical cyclone and sea surface temperature data have been used in several studies to forecast the total number of hurricanes in the Atlantic Basin. Sea surface temperature (SST) and latent heat flux (LHF) are correlated with tropical cyclone occurrences, but this correlation is known to vary with location and strength of the storm. The objective of this article is to identify features of SST and LHF that can explain the spatial-temporal variation of tropical cyclone counts, categorized by their strength. We develop a variable selection procedure for multivariate spatial-temporally varying coefficients, under a Poisson hurdle model (PHM) framework, which takes into account the zero-inflated nature of the counts. The method differs from current spatial-temporal variable selection techniques by offering a dynamic variable selection procedure, that shares information between responses, locations, time and levels in the PHM context. The model is used to study the association between SST and LHF and the number of tropical cyclones of different strengths in 400 locations in the Atlantic Basin over the period of 1950-2013. Results show that it is possible to estimate the number of tropical storms by season and region. Furthermore, the model delimits areas with a significant correlation between SST and LHF features and the occurrence and strength of TCs in the North Atlantic Basin. △ Less

Submitted 8 May, 2018; originally announced May 2018.

Comments: 21 pages, 8 figures

Vecchia approximations of Gaussian-process predictions

Authors: Matthias Katzfuss, Joseph Guinness, Wenlong Gong, Daniel Zilber

Abstract: Gaussian processes (GPs) are highly flexible function estimators used for geospatial analysis, nonparametric regression, and machine learning, but they are computationally infeasible for large datasets. Vecchia approximations of GPs have been used to enable fast evaluation of the likelihood for parameter inference. Here, we study Vecchia approximations of spatial predictions at observed and unobse… ▽ More Gaussian processes (GPs) are highly flexible function estimators used for geospatial analysis, nonparametric regression, and machine learning, but they are computationally infeasible for large datasets. Vecchia approximations of GPs have been used to enable fast evaluation of the likelihood for parameter inference. Here, we study Vecchia approximations of spatial predictions at observed and unobserved locations, including obtaining joint predictive distributions at large sets of locations. We consider a general Vecchia framework for GP predictions, which contains some novel and some existing special cases. We study the accuracy and computational properties of these approaches theoretically and numerically, proving that our new methods exhibit linear computational complexity in the total number of spatial locations. We show that certain choices within the framework can have a strong effect on uncertainty quantification and computational cost, which leads to specific recommendations on which methods are most suitable for various settings. We also apply our methods to a satellite dataset of chlorophyll fluorescence, showing that the new methods are faster or more accurate than existing methods, and reduce unrealistic artifacts in prediction maps. △ Less

Submitted 14 May, 2020; v1 submitted 8 May, 2018; originally announced May 2018.

Journal ref: Journal of Agricultural, Biological, and Environmental Statistics, 25(3), 383-414 (2020)

Spatial shrinkage via the product independent Gaussian process prior

Authors: Arkaprava Roy, Brian J. Reich, Joseph Guinness, Russell T. Shinohara, Ana-Maria Staicu

Abstract: We study the problem of sparse signal detection on a spatial domain. We propose a novel approach to model continuous signals that are sparse and piecewise smooth as product of independent Gaussian processes (PING) with a smooth covariance kernel. The smoothness of the PING process is ensured by the smoothness of the covariance kernels of Gaussian components in the product, and sparsity is controll… ▽ More We study the problem of sparse signal detection on a spatial domain. We propose a novel approach to model continuous signals that are sparse and piecewise smooth as product of independent Gaussian processes (PING) with a smooth covariance kernel. The smoothness of the PING process is ensured by the smoothness of the covariance kernels of Gaussian components in the product, and sparsity is controlled by the number of components. The bivariate kurtosis of the PING process shows more components in the product results in thicker tail and sharper peak at zero. The simulation results demonstrate the improvement in estimation using the PING prior over Gaussian process (GP) prior for different image regressions. We apply our method to a longitudinal MRI dataset to detect the regions that are affected by multiple sclerosis (MS) in the greatest magnitude through an image-on-scalar regression model. Due to huge dimensionality of these images, we transform the data into the spectral domain and develop methods to conduct computation in this domain. In our MS imaging study, the estimates from the PING model are more informative than those from the GP model. △ Less

Submitted 5 June, 2020; v1 submitted 8 May, 2018; originally announced May 2018.

A Test for Isotropy on a Sphere using Spherical Harmonic Functions

Authors: Indranil Sahoo, Joseph Guinness, Brian J. Reich

Abstract: Analysis of geostatistical data is often based on the assumption that the spatial random field is isotropic. This assumption, if erroneous, can adversely affect model predictions and statistical inference. Nowadays many applications consider data over the entire globe and hence it is necessary to check the assumption of isotropy on a sphere. In this paper, a test for spatial isotropy on a sphere i… ▽ More Analysis of geostatistical data is often based on the assumption that the spatial random field is isotropic. This assumption, if erroneous, can adversely affect model predictions and statistical inference. Nowadays many applications consider data over the entire globe and hence it is necessary to check the assumption of isotropy on a sphere. In this paper, a test for spatial isotropy on a sphere is proposed. The data are first projected onto the set of spherical harmonic functions. Under isotropy, the spherical harmonic coefficients are uncorrelated whereas they are correlated if the underlying fields are not isotropic. This motivates a test based on the sample correlation matrix of the spherical harmonic coefficients. In particular, we use the largest eigenvalue of the sample correlation matrix as the test statistic. Extensive simulations are conducted to assess the Type I errors of the test under different scenarios. We show how temporal correlation affects the test and provide a method for handling temporal correlation. We also gauge the power of the test as we move away from isotropy. The method is applied to the near-surface air temperature data which is part of the HadCM3 model output. Although we do not expect global temperature fields to be isotropic, we propose several anisotropic models with increasing complexity, each of which has an isotropic process as model component and we apply the test to the isotropic component in a sequence of such models as a method of determining how well the models capture the anisotropy in the fields. △ Less

Submitted 14 November, 2017; v1 submitted 11 November, 2017; originally announced November 2017.

Journal ref: Statistica Sinica 29 (2019): 1253-1276

Spectral Density Estimation for Random Fields via Periodic Embeddings

Authors: Joseph Guinness

Abstract: We introduce methods for estimating the spectral density of a random field on a $d$-dimensional lattice from incomplete gridded data. Data are iteratively imputed onto an expanded lattice according to a model with a periodic covariance function. The imputations are convenient computationally, in that circulant embedding and preconditioned conjugate gradient methods can produce imputations in… ▽ More We introduce methods for estimating the spectral density of a random field on a $d$-dimensional lattice from incomplete gridded data. Data are iteratively imputed onto an expanded lattice according to a model with a periodic covariance function. The imputations are convenient computationally, in that circulant embedding and preconditioned conjugate gradient methods can produce imputations in $O(n\log n)$ time and $O(n)$ memory. However, these so-called periodic imputations are motivated mainly by their ability to produce accurace spectral density estimates. In addition, we introduce a parametric filtering method that is designed to reduce periodogram smoothing bias. The paper contains theoretical results studying properties of the imputed data periodogram and numerical and simulation studies comparing the performance of the proposed methods to existing approaches in a number of scenarios. We present an application to a gridded satellite surface temperature dataset with missing values. △ Less

Submitted 9 August, 2019; v1 submitted 24 October, 2017; originally announced October 2017.

Journal ref: Biometrika, 106(2), 267-286 (2019)

A general framework for Vecchia approximations of Gaussian processes

Authors: Matthias Katzfuss, Joseph Guinness

Abstract: Gaussian processes (GPs) are commonly used as models for functions, time series, and spatial fields, but they are computationally infeasible for large datasets. Focusing on the typical setting of modeling data as a GP plus an additive noise term, we propose a generalization of the Vecchia (1988) approach as a framework for GP approximations. We show that our general Vecchia approach contains many… ▽ More Gaussian processes (GPs) are commonly used as models for functions, time series, and spatial fields, but they are computationally infeasible for large datasets. Focusing on the typical setting of modeling data as a GP plus an additive noise term, we propose a generalization of the Vecchia (1988) approach as a framework for GP approximations. We show that our general Vecchia approach contains many popular existing GP approximations as special cases, allowing for comparisons among the different methods within a unified framework. Representing the models by directed acyclic graphs, we determine the sparsity of the matrices necessary for inference, which leads to new insights regarding the computational properties. Based on these results, we propose a novel sparse general Vecchia approximation, which ensures computational feasibility for large spatial datasets but can lead to considerable improvements in approximation accuracy over Vecchia's original approach. We provide several theoretical results and conduct numerical comparisons. We conclude with guidelines for the use of Vecchia approximations in spatial statistics. △ Less

Submitted 17 August, 2019; v1 submitted 21 August, 2017; originally announced August 2017.

Journal ref: Statistical Science, 36(1), 124-141 (2021)

Permutation and Grou** Methods for Sharpening Gaussian Process Approximations

Authors: Joseph Guinness

Abstract: Vecchia's approximate likelihood for Gaussian process parameters depends on how the observations are ordered, which can be viewed as a deficiency because the exact likelihood is permutation-invariant. This article takes the alternative standpoint that the ordering of the observations can be tuned to sharpen the approximations. Advantageously chosen orderings can drastically improve the approximati… ▽ More Vecchia's approximate likelihood for Gaussian process parameters depends on how the observations are ordered, which can be viewed as a deficiency because the exact likelihood is permutation-invariant. This article takes the alternative standpoint that the ordering of the observations can be tuned to sharpen the approximations. Advantageously chosen orderings can drastically improve the approximations, and in fact, completely random orderings often produce far more accurate approximations than default coordinate-based orderings do. In addition to the permutation results, automatic methods for grou** calculations of components of the approximation are introduced, having the result of simultaneously improving the quality of the approximation and reducing its computational burden. In common settings, reordering combined with grou** reduces Kullback-Leibler divergence from the target model by a factor of 80 and computation time by a factor of 2 compared to ungrouped approximations with default ordering. The claims are supported by theory and numerical results with comparisons to other approximations, including tapered covariances and stochastic partial differential equation approximations. Computational details are provided, including efficiently finding the orderings and ordered nearest neighbors, and profiling out linear mean parameters and using the approximations for prediction and conditional simulation. An application to space-time satellite data is presented. △ Less

Submitted 19 February, 2018; v1 submitted 17 September, 2016; originally announced September 2016.

Compression and Conditional Emulation of Climate Model Output

Authors: Joseph Guinness, Dorit Hammerling

Abstract: Numerical climate model simulations run at high spatial and temporal resolutions generate massive quantities of data. As our computing capabilities continue to increase, storing all of the data is not sustainable, and thus it is important to develop methods for representing the full datasets by smaller compressed versions. We propose a statistical compression and decompression algorithm based on s… ▽ More Numerical climate model simulations run at high spatial and temporal resolutions generate massive quantities of data. As our computing capabilities continue to increase, storing all of the data is not sustainable, and thus it is important to develop methods for representing the full datasets by smaller compressed versions. We propose a statistical compression and decompression algorithm based on storing a set of summary statistics as well as a statistical model describing the conditional distribution of the full dataset given the summary statistics. The statistical model can be used to generate realizations representing the full dataset, along with characterizations of the uncertainties in the generated data. Thus, the methods are capable of both compression and conditional emulation of the climate models. Considerable attention is paid to accurately modeling the original dataset--one year of daily mean temperature data--particularly with regard to the inherent spatial nonstationarity in global fields, and to determining the statistics to be stored, so that the variation in the original data can be closely captured, while allowing for fast decompression and conditional emulation on modest computers. △ Less

Submitted 19 February, 2018; v1 submitted 25 May, 2016; originally announced May 2016.

A generalized quadratic estimate for random field nonstationarity

Authors: Ethan Anderes, Joe Guinness

Abstract: In this paper, we attempt to shed light on a new class of nonstationary random fields which exhibit, what we call, local invariant nonstationarity. We argue that the local invariant property has a special interaction with a new generalized quadratic estimate---also derived here---which extends an estimate originally developed for gravitational lensing of the Cosmic Microwave Background in Cosmolog… ▽ More In this paper, we attempt to shed light on a new class of nonstationary random fields which exhibit, what we call, local invariant nonstationarity. We argue that the local invariant property has a special interaction with a new generalized quadratic estimate---also derived here---which extends an estimate originally developed for gravitational lensing of the Cosmic Microwave Background in Cosmology \cite{hu2001map**, hu2002mass}. The nature of this interaction not only encourages low estimation bias but also enables accurate (and fast) quantification of Frequentist mean square error quantification of the estimated nonstationarity. These quadratic estimates are interesting, in their own right, as they detect and estimate nonstationarity by probing correlation among Fourier frequencies, the absence of which is the characterizing feature of weak stationarity (by Bochner's Theorem). Moreover, this generalized quadratic estimate can be computed with a Fourier characterization that runs in $\mathcal O(n\log n)$ time when observing the field on a uniform grid of size $n$ in $\Bbb R^d $. Finally, the work presented here partially addresses two other problems associated with the statistical theory of nonstationarity: 1) estimating the phase of a spatially varying modulated stationary random field and 2) identifying a larger class of nonstationary random fields which admit an extension of the quadratic estimator of gravitational lensing that extends the same attractive statistical properties. △ Less

Submitted 10 March, 2016; originally announced March 2016.

An Evolutionary Spectrum Approach to Incorporate Large-scale Geographical Descriptors on Global Processes

Authors: Stefano Castruccio, Joseph Guinness

Abstract: We introduce a nonstationary spatio-temporal statistical model for gridded data on the sphere. The model specifies a computationally convenient covariance structure that depends on heterogeneous geography. Widely used statistical models on a spherical domain are nonstationary for different latitudes, but stationary at the same latitude (axial symmetry). This assumption has been acknowledged to be… ▽ More We introduce a nonstationary spatio-temporal statistical model for gridded data on the sphere. The model specifies a computationally convenient covariance structure that depends on heterogeneous geography. Widely used statistical models on a spherical domain are nonstationary for different latitudes, but stationary at the same latitude (axial symmetry). This assumption has been acknowledged to be too restrictive for quantities such as surface temperature, whose statistical behavior is influenced by large scale geographical descriptors such as land and ocean. We propose an evolutionary spectrum approach that is able to account for different regimes across the Earth's geography, and results in a more general and flexible class of models that vastly outperforms axially symmetric models and captures longitudinal patterns that would otherwise be assumed constant. The model can be estimated with in a multi-step conditional likelihood approximation that preserves the nonstationary features while allowing for easily distributed computations: we show how the fit of a data sets larger than 20 million data can be performed in less than one day on a state-of-the-art workstation. Once the parameters are estimated, it is possible to instantaneously generate surrogate runs from a common laptop. Further, the resulting estimates from the statistical model can be regarded as a synthetic description (i.e. a compression) of the space-time characteristics of an entire initial condition ensemble. Compared to traditional algorithms aiming at compressing the bit-by-bit information on each climate model run, the proposed approach achieves vastly superior compression rates. △ Less

Submitted 24 February, 2016; v1 submitted 13 July, 2015; originally announced July 2015.

Efficient Computation of Gaussian Likelihoods for Stationary Markov Random Field Models

Authors: Joseph Guinness, Ilse C. F. Ipsen

Abstract: Rue and Held (2005) proposed a method for efficiently computing the Gaussian likelihood for stationary Markov random field models, when the data locations fall on a complete regular grid, and the model has no additive error term. The calculations rely on the availability of the covariances. We prove a theorem giving the rate of convergence of a spectral method of computing the covariances, establi… ▽ More Rue and Held (2005) proposed a method for efficiently computing the Gaussian likelihood for stationary Markov random field models, when the data locations fall on a complete regular grid, and the model has no additive error term. The calculations rely on the availability of the covariances. We prove a theorem giving the rate of convergence of a spectral method of computing the covariances, establishing that the error decays faster than any polynomial in the size of the computing grid. We extend the exact likelihood calculations to the case of non-rectangular domains and missing values on the interior of the grid and to the case when an additive uncorrelated error term (nugget) is present in the model. We also give an alternative formulation of the likelihood that has a smaller memory burden, parts of which can be computed in parallel. We show in simulations that using the exact likelihood can give far better parameter estimates than using standard Markov random field approximations. Having access to the exact likelihood allows for model comparisons via likelihood ratios on large datasets, so as an application of the methods, we compare several state-of-the-art methods for large spatial datasets on an aerosol optical thickness dataset. We find that simple block independent likelihood and composite likelihood methods outperform stochastic partial differential equation approximations in terms of computation time and returning parameter estimates that nearly maximize the likelihood. △ Less

Submitted 12 December, 2019; v1 submitted 30 May, 2015; originally announced June 2015.

Interpolation of nonstationary high frequency spatial-temporal temperature data

Authors: Joseph Guinness, Michael L. Stein

Abstract: The Atmospheric Radiation Measurement program is a U.S. Department of Energy project that collects meteorological observations at several locations around the world in order to study how weather processes affect global climate change. As one of its initiatives, it operates a set of fixed but irregularly-spaced monitoring facilities in the Southern Great Plains region of the U.S. We describe method… ▽ More The Atmospheric Radiation Measurement program is a U.S. Department of Energy project that collects meteorological observations at several locations around the world in order to study how weather processes affect global climate change. As one of its initiatives, it operates a set of fixed but irregularly-spaced monitoring facilities in the Southern Great Plains region of the U.S. We describe methods for interpolating temperature records from these fixed facilities to locations at which no observations were made, which can be useful when values are required on a spatial grid. We interpolate by conditionally simulating from a fitted nonstationary Gaussian process model that accounts for the time-varying statistical characteristics of the temperatures, as well as the dependence on solar radiation. The model is fit by maximizing an approximate likelihood, and the conditional simulations result in well-calibrated confidence intervals for the predicted temperatures. We also describe methods for handling spatial-temporal jumps in the data to interpolate a slow-moving cold front. △ Less

Submitted 29 November, 2013; originally announced November 2013.

Comments: Published in at http://dx.doi.org/10.1214/13-AOAS633 the Annals of Applied Statistics (http://www.imstat.org/aoas/) by the Institute of Mathematical Statistics (http://www.imstat.org)

Report number: IMS-AOAS-AOAS633

Journal ref: Annals of Applied Statistics 2013, Vol. 7, No. 3, 1684-1708