Showing 1–50 of 64 results for author: Sugasawa, S

Robust Linear Mixed Models using Hierarchical Gamma-Divergence

Authors: Shonosuke Sugasawa, Francis K. C. Hui, Alan H. Welsh

Abstract: Linear mixed models (LMMs), which typically assume normality for both the random effects and error terms, are a popular class of methods for analyzing longitudinal and clustered data. However, such models can be sensitive to outliers, and this can lead to poor statistical results (e.g., biased inference on model parameters and inaccurate prediction of random effects) if the data are contaminated.… ▽ More Linear mixed models (LMMs), which typically assume normality for both the random effects and error terms, are a popular class of methods for analyzing longitudinal and clustered data. However, such models can be sensitive to outliers, and this can lead to poor statistical results (e.g., biased inference on model parameters and inaccurate prediction of random effects) if the data are contaminated. We propose a new approach to robust estimation and inference for LMMs using a hierarchical gamma divergence, which offers an automated, data-driven approach to downweight the effects of outliers occurring in both the error, and the random effects, using normalized powered density weights. For estimation and inference, we develop a computationally scalable minorization-maximization algorithm for the resulting objective function, along with a clustered bootstrap method for uncertainty quantification and a Hyvarinen score criterion for selecting a tuning parameter controlling the degree of robustness. When the genuine and contamination mixed effects distributions are sufficiently separated, then under suitable regularity conditions assuming the number of clusters tends to infinity, we show the resulting robust estimates can be asymptotically controlled even under a heavy level of (covariate-dependent) contamination. Simulation studies demonstrate hierarchical gamma divergence consistently outperforms several currently available methods for robustifying LMMs, under a wide range of scenarios of outlier generation at both the response and random effects levels. We illustrate the proposed method using data from a multi-center AIDS cohort study, where the use of a robust LMMs using hierarchical gamma divergence approach produces noticeably different results compared to methods that do not adequately adjust for potential outlier contamination. △ Less

Submitted 1 July, 2024; originally announced July 2024.

Comments: 30 pages (main) + 6 pages (supplement)

State-Space Modeling of Shape-constrained Functional Time Series

Authors: Daichi Hiraki, Yasuyuki Hamura, Kaoru Irie, Shonosuke Sugasawa

Abstract: Functional time series data frequently appears in economic applications, where the functions of interest are subject to some shape constraints, including monotonicity and convexity, as typical of the estimation of the Lorenz curve. This paper proposes a state-space model for time-varying functions to extract trends and serial dependence from functional time series while imposing the shape constrai… ▽ More Functional time series data frequently appears in economic applications, where the functions of interest are subject to some shape constraints, including monotonicity and convexity, as typical of the estimation of the Lorenz curve. This paper proposes a state-space model for time-varying functions to extract trends and serial dependence from functional time series while imposing the shape constraints on the estimated functions. The function of interest is modeled by a convex combination of selected basis functions to satisfy the shape constraints, where the time-varying convex weights on simplex follow the dynamic multi-logit models. For the complicated likelihood of this model, a novel data augmentation technique is devised to enable posterior computation by an efficient Markov chain Monte Carlo method. The proposed method is applied to the estimation of time-varying Lorenz curves, and its utility is illustrated through numerical experiments and analysis of panel data of household incomes in Japan. △ Less

Submitted 11 April, 2024; originally announced April 2024.

Comments: 34 pages, 7 figures, 6 tables

Sub-model aggregation for scalable eigenvector spatial filtering: Application to spatially varying coefficient modeling

Authors: Daisuke Murakami, Shonosuke Sugasawa, Hajime Seya, Daniel A. Griffith

Abstract: This study proposes a method for aggregating/synthesizing global and local sub-models for fast and flexible spatial regression modeling. Eigenvector spatial filtering (ESF) was used to model spatially varying coefficients and spatial dependence in the residuals by sub-model, while the generalized product-of-experts method was used to aggregate these sub-models. The major advantages of the proposed… ▽ More This study proposes a method for aggregating/synthesizing global and local sub-models for fast and flexible spatial regression modeling. Eigenvector spatial filtering (ESF) was used to model spatially varying coefficients and spatial dependence in the residuals by sub-model, while the generalized product-of-experts method was used to aggregate these sub-models. The major advantages of the proposed method are as follows: (i) it is highly scalable for large samples in terms of accuracy and computational efficiency; (ii) it is easily implemented by estimating sub-models independently first and aggregating/averaging them thereafter; and (iii) likelihood-based inference is available because the marginal likelihood is available in closed-form. The accuracy and computational efficiency of the proposed method are confirmed using Monte Carlo simulation experiments. This method was then applied to residential land price analysis in Japan. The results demonstrate the usefulness of this method for improving the interpretability of spatially varying coefficients. The proposed method is implemented in an R package spmoran (version 0.3.0 or later). △ Less

Submitted 24 January, 2024; v1 submitted 23 January, 2024; originally announced January 2024.

Semiparametric Copula Estimation for Spatially Correlated Multivariate Mixed Outcomes: Analyzing Visual Sightings of Fin Whales from Line Transect Survey

Authors: Tomotaka Momozaki, Tomoyuki Nakagawa, Shonosuke Sugasawa, Hiroko Kato Solvang

Abstract: Multivariate data having both continuous and discrete variables is known as mixed outcomes and has widely appeared in a variety of fields such as ecology, epidemiology, and climatology. In order to understand the probability structure of multivariate data, the estimation of the dependence structure among mixed outcomes is very important. However, when location information is equipped with multivar… ▽ More Multivariate data having both continuous and discrete variables is known as mixed outcomes and has widely appeared in a variety of fields such as ecology, epidemiology, and climatology. In order to understand the probability structure of multivariate data, the estimation of the dependence structure among mixed outcomes is very important. However, when location information is equipped with multivariate data, the spatial correlation should be adequately taken into account; otherwise, the estimation of the dependence structure would be severely biased. To solve this issue, we propose a semiparametric Bayesian inference for the dependence structure among mixed outcomes while eliminating spatial correlation. To this end, we consider a hierarchical spatial model based on the rank likelihood and a latent multivariate Gaussian process. We develop an efficient algorithm for computing the posterior using the Markov Chain Monte Carlo. We also provide a scalable implementation of the model using the nearest-neighbor Gaussian process under large spatial datasets. We conduct a simulation study to validate our proposed procedure and demonstrate that the procedure successfully accounts for spatial correlation and correctly infers the dependence structure among outcomes. Furthermore, the procedure is applied to a real example collected during an international synoptic krill survey in the Scotia Sea of the Antarctic Peninsula, which includes sighting data of fin whales (Balaenoptera physalus), and the relevant oceanographic data. △ Less

Submitted 19 December, 2023; originally announced December 2023.

Comments: 23 pages, 5 figures

MSC Class: 62F15 (Primary); 62H11 (Secondary)

Hierarchical Regression Discontinuity Design: Pursuing Subgroup Treatment Effects

Authors: Shonosuke Sugasawa, Takuya Ishihara, Daisuke Kurisu

Abstract: Regression discontinuity design (RDD) is widely adopted for causal inference under intervention determined by a continuous variable. While one is interested in treatment effect heterogeneity by subgroups in many applications, RDD typically suffers from small subgroup-wise sample sizes, which makes the estimation results highly instable. To solve this issue, we introduce hierarchical RDD (HRDD), a… ▽ More Regression discontinuity design (RDD) is widely adopted for causal inference under intervention determined by a continuous variable. While one is interested in treatment effect heterogeneity by subgroups in many applications, RDD typically suffers from small subgroup-wise sample sizes, which makes the estimation results highly instable. To solve this issue, we introduce hierarchical RDD (HRDD), a hierarchical Bayes approach for pursuing treatment effect heterogeneity in RDD. A key feature of HRDD is to employ a pseudo-model based on a loss function to estimate subgroup-level parameters of treatment effects under RDD, and assign a hierarchical prior distribution to ''borrow strength'' from other subgroups. The posterior computation can be easily done by a simple Gibbs sampling, and the optimal bandwidth can be automatically selected by the Hyvärinen scores for unnormalized models. We demonstrate the proposed HRDD through simulation and real data analysis, and show that HRDD provides much more stable point and interval estimation than separately applying the standard RDD method to each subgroup. △ Less

Submitted 19 June, 2024; v1 submitted 4 September, 2023; originally announced September 2023.

Comments: 24 pages

An Unbiased Predictor for Skewed Response Variable with Measurement Error in Covariate

Authors: Sepideh Mosaferi, Malay Ghosh, Shonosuke Sugasawa

Abstract: We introduce a new small area predictor when the Fay-Herriot normal error model is fitted to a logarithmically transformed response variable, and the covariate is measured with error. This framework has been previously studied by Mosaferi et al. (2023). The empirical predictor given in their manuscript cannot perform uniformly better than the direct estimator. Our proposed predictor in this manusc… ▽ More We introduce a new small area predictor when the Fay-Herriot normal error model is fitted to a logarithmically transformed response variable, and the covariate is measured with error. This framework has been previously studied by Mosaferi et al. (2023). The empirical predictor given in their manuscript cannot perform uniformly better than the direct estimator. Our proposed predictor in this manuscript is unbiased and can perform uniformly better than the one proposed in Mosaferi et al. (2023). We derive an approximation of the mean squared error (MSE) for the predictor. The prediction intervals based on the MSE suffer from coverage problems. Thus, we propose a non-parametric bootstrap prediction interval which is more accurate. This problem is of great interest in small area applications since statistical agencies and agricultural surveys are often asked to produce estimates of right skewed variables with covariates measured with errors. With Monte Carlo simulation studies and two Census Bureau's data sets, we demonstrate the superiority of our proposed methodology. △ Less

Submitted 21 August, 2023; originally announced August 2023.

Predicting COVID-19 hospitalisation using a mixture of Bayesian predictive syntheses

Authors: Genya Kobayashi, Shonosuke Sugasawa, Yuki Kawakubo, Dongu Han, Taeryon Choi

Abstract: This paper proposes a novel methodology called the mixture of Bayesian predictive syntheses (MBPS) for multiple time series count data for the challenging task of predicting the numbers of COVID-19 inpatients and isolated cases in Japan and Korea at the subnational-level. MBPS combines a set of predictive models and partitions the multiple time series into clusters based on their contribution to p… ▽ More This paper proposes a novel methodology called the mixture of Bayesian predictive syntheses (MBPS) for multiple time series count data for the challenging task of predicting the numbers of COVID-19 inpatients and isolated cases in Japan and Korea at the subnational-level. MBPS combines a set of predictive models and partitions the multiple time series into clusters based on their contribution to predicting the outcome. In this way, MBPS leverages the shared information within each cluster and is suitable for predicting COVID-19 inpatients since the data exhibit similar dynamics over multiple areas. Also, MBPS avoids using a multivariate count model, which is generally cumbersome to develop and implement. Our Japanese and Korean data analyses demonstrate that the proposed MBPS methodology has improved predictive accuracy and uncertainty quantification. △ Less

Submitted 19 March, 2024; v1 submitted 11 August, 2023; originally announced August 2023.

Similarity-based Random Partition Distribution for Clustering Functional Data

Authors: Tomoya Wakayama, Shonosuke Sugasawa, Genya Kobayashi

Abstract: Random partition distribution is a crucial tool for model-based clustering. This study advances the field of random partition in the context of functional spatial data, focusing on the challenges posed by hourly population data across various regions and dates. We propose an extended generalized Dirichlet process, named the similarity-based generalized Dirichlet process (SGDP), to address the limi… ▽ More Random partition distribution is a crucial tool for model-based clustering. This study advances the field of random partition in the context of functional spatial data, focusing on the challenges posed by hourly population data across various regions and dates. We propose an extended generalized Dirichlet process, named the similarity-based generalized Dirichlet process (SGDP), to address the limitations of simple random partition distributions (e.g., those induced by the Dirichlet process), such as an overabundance of clusters. This model prevents excess cluster production as well as incorporates pairwise similarity information to ensure accurate and meaningful grou**. The theoretical properties of the SGDP are studied. Then, SGDP-based random partition is applied to a real-world dataset of hourly population flow in $500\text{m}^2$ meshes in the central part of Tokyo. In this empirical context, our method excels at detecting meaningful patterns in the data while accounting for spatial nuances. The results underscore the adaptability and utility of the method, showcasing its prowess in revealing intricate spatiotemporal dynamics. The proposed SGDP will significantly contribute to urban planning, transportation, and policy-making and will be a helpful tool for understanding population dynamics and their implications. △ Less

Submitted 22 June, 2024; v1 submitted 3 August, 2023; originally announced August 2023.

Comments: 27 pages

Bayesian Causal Synthesis for Meta-Inference on Heterogeneous Treatment Effects

Authors: Shonosuke Sugasawa, Kosaku Takanashi, Kenichiro McAlinn, Edoardo M. Airoldi

Abstract: The estimation of heterogeneous treatment effects in the potential outcome setting is biased when there exists model misspecification or unobserved confounding. As these biases are unobservable, what model to use when remains a critical open question. In this paper, we propose a novel Bayesian methodology to mitigate misspecification and improve estimation via a synthesis of multiple causal estima… ▽ More The estimation of heterogeneous treatment effects in the potential outcome setting is biased when there exists model misspecification or unobserved confounding. As these biases are unobservable, what model to use when remains a critical open question. In this paper, we propose a novel Bayesian methodology to mitigate misspecification and improve estimation via a synthesis of multiple causal estimates, which we call Bayesian causal synthesis. Our development is built upon identifying a synthesis function that correctly specifies the heterogeneous treatment effect under no unobserved confounding, and achieves the irreducible bias under unobserved confounding. We show that our proposed method results in consistent estimates of the heterogeneous treatment effect; either with no bias or with irreducible bias. We provide a computational algorithm for fast posterior sampling. Several benchmark simulations and an empirical study highlight the efficacy of the proposed approach compared to existing methodologies, providing improved point and density estimation of the heterogeneous treatment effect, even under unobserved confounding. △ Less

Submitted 8 May, 2024; v1 submitted 16 April, 2023; originally announced April 2023.

Comments: 30 pages (Main document) + 14 pages (Supplement)

Posterior Robustness with Milder Conditions: Contamination Models Revisited

Authors: Yasuyuki Hamura, Kaoru Irie, Shonosuke Sugasawa

Abstract: Robust Bayesian linear regression is a classical but essential statistical tool. Although novel robustness properties of posterior distributions have been proved recently under a certain class of error distributions, their sufficient conditions are restrictive and exclude several important situations. In this work, we revisit a classical two-component mixture model for response variables, also kno… ▽ More Robust Bayesian linear regression is a classical but essential statistical tool. Although novel robustness properties of posterior distributions have been proved recently under a certain class of error distributions, their sufficient conditions are restrictive and exclude several important situations. In this work, we revisit a classical two-component mixture model for response variables, also known as contamination model, where one component is a light-tailed regression model and the other component is heavy-tailed. The latter component is independent of the regression parameters, which is crucial in proving the posterior robustness. We obtain new sufficient conditions for posterior (non-)robustness and reveal non-trivial robustness results by using those conditions. In particular, we find that even the Student-$t$ error distribution can achieve the posterior robustness in our framework. A numerical study is performed to check the Kullback-Leibler divergence between the posterior distribution based on full data and that based on data obtained by removing outliers. △ Less

Submitted 3 April, 2024; v1 submitted 1 March, 2023; originally announced March 2023.

Comments: 19 pages, 1 figure

Gibbs Sampler for Matrix Generalized Inverse Gaussian Distributions

Authors: Yasuyuki Hamura, Kaoru Irie, Shonosuke Sugasawa

Abstract: Sampling from matrix generalized inverse Gaussian (MGIG) distributions is required in Markov Chain Monte Carlo (MCMC) algorithms for a variety of statistical models. However, an efficient sampling scheme for the MGIG distributions has not been fully developed. We here propose a novel blocked Gibbs sampler for the MGIG distributions, based on the Choleski decomposition. We show that the full condit… ▽ More Sampling from matrix generalized inverse Gaussian (MGIG) distributions is required in Markov Chain Monte Carlo (MCMC) algorithms for a variety of statistical models. However, an efficient sampling scheme for the MGIG distributions has not been fully developed. We here propose a novel blocked Gibbs sampler for the MGIG distributions, based on the Choleski decomposition. We show that the full conditionals of the diagonal and unit lower-triangular entries are univariate generalized inverse Gaussian and multivariate normal distributions, respectively. Several variants of the Metropolis-Hastings algorithm can also be considered for this problem, but we mathematically prove that the average acceptance rates become extremely low in particular scenarios. We demonstrate the computational efficiency of the proposed Gibbs sampler through simulation studies and data analysis. △ Less

Submitted 19 February, 2023; originally announced February 2023.

Comments: 34 pages, 5 figures

Spatiotemporal factor models for functional data with application to population map forecast

Authors: Tomoya Wakayama, Shonosuke Sugasawa

Abstract: The proliferation of mobile devices has led to the collection of large amounts of population data. This situation has prompted the need to utilize this rich, multidimensional data in practical applications. In response to this trend, we have integrated functional data analysis (FDA) and factor analysis to address the challenge of predicting hourly population changes across various districts in Tok… ▽ More The proliferation of mobile devices has led to the collection of large amounts of population data. This situation has prompted the need to utilize this rich, multidimensional data in practical applications. In response to this trend, we have integrated functional data analysis (FDA) and factor analysis to address the challenge of predicting hourly population changes across various districts in Tokyo. Specifically, by assuming a Gaussian process, we avoided the large covariance matrix parameters of the multivariate normal distribution. In addition, the data were both time and spatially dependent between districts. To capture these characteristics, a Bayesian factor model was introduced, which modeled the time series of a small number of common factors and expressed the spatial structure through factor loading matrices. Furthermore, the factor loading matrices were made identifiable and sparse to ensure the interpretability of the model. We also proposed a Bayesian shrinkage method as a systematic approach for factor selection. Through numerical experiments and data analysis, we investigated the predictive accuracy and interpretability of our proposed method. We concluded that the flexibility of the method allows for the incorporation of additional time series features, thereby improving its accuracy. △ Less

Submitted 6 June, 2023; v1 submitted 8 February, 2023; originally announced February 2023.

Fully Data-driven Normalized and Exponentiated Kernel Density Estimator with Hyvärinen Score

Authors: Shunsuke Imai, Takuya Koriyama, Shouto Yonekura, Shonosuke Sugasawa, Yoshihiko Nishiyama

Abstract: We introduce a new deal of kernel density estimation using an exponentiated form of kernel density estimators. The density estimator has two hyperparameters flexibly controlling the smoothness of the resulting density. We tune them in a data-driven manner by minimizing an objective function based on the Hyvärinen score to avoid the optimization involving the intractable normalizing constant due to… ▽ More We introduce a new deal of kernel density estimation using an exponentiated form of kernel density estimators. The density estimator has two hyperparameters flexibly controlling the smoothness of the resulting density. We tune them in a data-driven manner by minimizing an objective function based on the Hyvärinen score to avoid the optimization involving the intractable normalizing constant due to the exponentiation. We show the asymptotic properties of the proposed estimator and emphasize the importance of including the two hyperparameters for flexible density estimation. Our simulation studies and application to income data show that the proposed density estimator is appealing when the underlying density is multi-modal or observations contain outliers. △ Less

Submitted 13 February, 2024; v1 submitted 2 December, 2022; originally announced December 2022.

Fast and Locally Adaptive Bayesian Quantile Smoothing using Calibrated Variational Approximations

Authors: Takahiro Onizuka, Shintaro Hashimoto, Shonosuke Sugasawa

Abstract: Quantiles are useful characteristics of random variables that can provide substantial information on distributions compared with commonly used summary statistics such as means. In this paper, we propose a Bayesian quantile trend filtering method to estimate non-stationary trend of quantiles. We introduce general shrinkage priors to induce locally adaptive Bayesian inference on trends and mixture r… ▽ More Quantiles are useful characteristics of random variables that can provide substantial information on distributions compared with commonly used summary statistics such as means. In this paper, we propose a Bayesian quantile trend filtering method to estimate non-stationary trend of quantiles. We introduce general shrinkage priors to induce locally adaptive Bayesian inference on trends and mixture representation of the asymmetric Laplace likelihood. To quickly compute the posterior distribution, we develop calibrated mean-field variational approximations to guarantee that the frequentist coverage of credible intervals obtained from the approximated posterior is a specified nominal level. Simulation and empirical studies show that the proposed algorithm is computationally much more efficient than the Gibbs sampler and tends to provide stable inference results, especially for high/low quantiles. △ Less

Submitted 20 October, 2023; v1 submitted 8 November, 2022; originally announced November 2022.

Comments: 51 pages, 7 figures. arXiv admin note: text overlap with arXiv:2202.09534

Semiparametric imputation using latent sparse conditional Gaussian mixtures for multivariate mixed outcomes

Authors: Shonosuke Sugasawa, Jae Kwang Kim, Kosuke Morikawa

Abstract: This paper proposes a flexible Bayesian approach to multiple imputation using conditional Gaussian mixtures. We introduce novel shrinkage priors for covariate-dependent mixing proportions in the mixture models to automatically select the suitable number of components used in the imputation step. We develop an efficient sampling algorithm for posterior computation and multiple imputation via Markov… ▽ More This paper proposes a flexible Bayesian approach to multiple imputation using conditional Gaussian mixtures. We introduce novel shrinkage priors for covariate-dependent mixing proportions in the mixture models to automatically select the suitable number of components used in the imputation step. We develop an efficient sampling algorithm for posterior computation and multiple imputation via Markov Chain Monte Carlo methods. The proposed method can be easily extended to the situation where the data contains not only continuous variables but also discrete variables such as binary and count values. We also propose approximate Bayesian inference for parameters defined by loss functions based on posterior predictive distributing of missing observations, by extending bootstrap-based Bayesian inference for complete data. The proposed method is demonstrated through numerical studies using simulated and real data. △ Less

Submitted 16 August, 2022; originally announced August 2022.

Comments: 29 pages, 5 figures

Locally Adaptive Bayesian Isotonic Regression using Half Shrinkage Priors

Authors: Ryo Okano, Yasuyuki Hamura, Kaoru Irie, Shonosuke Sugasawa

Abstract: Isotonic regression or monotone function estimation is a problem of estimating function values under monotonicity constraints, which appears naturally in many scientific fields. This paper proposes a new Bayesian method with global-local shrinkage priors for estimating monotone function values. Specifically, we introduce half shrinkage priors for positive valued random variables and assign them fo… ▽ More Isotonic regression or monotone function estimation is a problem of estimating function values under monotonicity constraints, which appears naturally in many scientific fields. This paper proposes a new Bayesian method with global-local shrinkage priors for estimating monotone function values. Specifically, we introduce half shrinkage priors for positive valued random variables and assign them for the first-order differences of function values. We also develop fast and simple Gibbs sampling algorithms for full posterior analysis. By incorporating advanced shrinkage priors, the proposed method is adaptive to local abrupt changes or jumps in target functions. We show this adaptive property theoretically by proving that the posterior mean estimators are robust to large differences and that asymptotic risk for unchanged points can be improved. Finally, we demonstrate the proposed methods through simulations and applications to a real data set. △ Less

Submitted 6 February, 2024; v1 submitted 9 August, 2022; originally announced August 2022.

Comments: 47 pages

Journal ref: Scandinavian Journal of Statistics, 2024

Spatio-temporal smoothing, interpolation and prediction of income distributions based on grouped data

Authors: Genya Kobayashi, Shonosuke Sugasawa, Yuki Kawakubo

Abstract: In Japan, the Housing and Land Survey (HLS) provides municipality-level grouped data on household incomes. Although these data can be used for effective local policymaking, their analyses are hindered by several challenges, such as limited information attributed to grou**, the presence of non-sampled areas, and the very low frequency of implementing surveys. To address these challenges, we propo… ▽ More In Japan, the Housing and Land Survey (HLS) provides municipality-level grouped data on household incomes. Although these data can be used for effective local policymaking, their analyses are hindered by several challenges, such as limited information attributed to grou**, the presence of non-sampled areas, and the very low frequency of implementing surveys. To address these challenges, we propose a novel grouped-data-based spatio-temporal finite mixture model to model the income distributions of multiple spatial units at multiple time points. A unique feature of the proposed method is that all the areas share common latent distributions and that the mixing proportions that include the spatial and temporal effects capture the potential area-wise heterogeneity. Thus, incorporating these effects can smooth out the quantities of interest over time and space, impute missing values, and predict future values. By treating the HLS data with the proposed method, we obtain complete maps of the income and poverty measures at an arbitrary time point, which can be used to facilitate rapid and efficient policymaking with fine granularity. △ Less

Submitted 30 June, 2023; v1 submitted 18 July, 2022; originally announced July 2022.

Functional Horseshoe Smoothing for Functional Trend Estimation

Authors: Tomoya Wakayama, Shonosuke Sugasawa

Abstract: Due to developments in instruments and computers, functional observations are increasingly popular. However, effective methodologies for flexibly estimating the underlying trends with valid uncertainty quantification for a sequence of functional data (e.g. functional time series) are still scarce. In this work, we develop a locally adaptive smoothing method, called functional horseshoe smoothing,… ▽ More Due to developments in instruments and computers, functional observations are increasingly popular. However, effective methodologies for flexibly estimating the underlying trends with valid uncertainty quantification for a sequence of functional data (e.g. functional time series) are still scarce. In this work, we develop a locally adaptive smoothing method, called functional horseshoe smoothing, by introducing a shrinkage prior to the general order of differences of functional variables. This allows us to capture abrupt changes by making the most of the shrinkage capability and also to assess uncertainty by Bayesian inference. The fully Bayesian framework allows the selection of the number of basis functions via the posterior predictive loss. We provide theoretical properties of the model, which support the shrinkage ability. Also, by taking advantage of the nature of functional data, this method is able to handle heterogeneously observed data without data augmentation. Simulation studies and real data analysis demonstrate that the proposed method has desirable properties. △ Less

Submitted 20 September, 2022; v1 submitted 21 April, 2022; originally announced April 2022.

Sparse Bayesian inference on gamma-distributed observations using shape-scale inverse-gamma mixtures

Authors: Yasuyuki Hamura, Takahiro Onizuka, Shintaro Hashimoto, Shonosuke Sugasawa

Abstract: In various applications, we deal with high-dimensional positive-valued data that often exhibits sparsity. This paper develops a new class of continuous global-local shrinkage priors tailored to analyzing gamma-distributed observations where most of the underlying means are concentrated around a certain value. Unlike existing shrinkage priors, our new prior is a shape-scale mixture of inverse-gamma… ▽ More In various applications, we deal with high-dimensional positive-valued data that often exhibits sparsity. This paper develops a new class of continuous global-local shrinkage priors tailored to analyzing gamma-distributed observations where most of the underlying means are concentrated around a certain value. Unlike existing shrinkage priors, our new prior is a shape-scale mixture of inverse-gamma distributions, which has a desirable interpretation of the form of posterior mean and admits flexible shrinkage. We show that the proposed prior has two desirable theoretical properties; Kullback-Leibler super-efficiency under sparsity and robust shrinkage rules for large observations. We propose an efficient sampling algorithm for posterior inference. The performance of the proposed method is illustrated through simulation and two real data examples, the average length of hospital stay for COVID-19 in South Korea and adaptive variance estimation of gene expression data. △ Less

Submitted 30 November, 2022; v1 submitted 16 March, 2022; originally announced March 2022.

Comments: 57 pages, 8 figures

Bayesian Spatial Predictive Synthesis

Authors: Danielle Cabel, Shonosuke Sugasawa, Masahiro Kato, Kosaku Takanashi, Kenichiro McAlinn

Abstract: Spatial data are characterized by their spatial dependence, which is often complex, non-linear, and difficult to capture with a single model. Significant levels of model uncertainty -- arising from these characteristics -- cannot be resolved by model selection or simple ensemble methods. We address this issue by proposing a novel methodology that captures spatially varying model uncertainty, which… ▽ More Spatial data are characterized by their spatial dependence, which is often complex, non-linear, and difficult to capture with a single model. Significant levels of model uncertainty -- arising from these characteristics -- cannot be resolved by model selection or simple ensemble methods. We address this issue by proposing a novel methodology that captures spatially varying model uncertainty, which we call Bayesian spatial predictive synthesis. Our proposal is derived by identifying the theoretically best approximate model under reasonable conditions, which is a latent factor spatially varying coefficient model in the Bayesian predictive synthesis framework. We then show that our proposed method produces exact minimax predictive distributions, providing finite sample guarantees. Two MCMC strategies are implemented for full uncertainty quantification, as well as a variational inference strategy for fast point inference. We also extend the estimation strategy for general responses. Through simulation examples and two real data applications, we demonstrate that our proposed spatial Bayesian predictive synthesis outperforms standard spatial models and advanced machine learning methods in terms of predictive accuracy. △ Less

Submitted 20 January, 2023; v1 submitted 10 March, 2022; originally announced March 2022.

Comments: 41 pages

On Data Augmentation for Models Involving Reciprocal Gamma Functions

Authors: Yasuyuki Hamura, Kaoru Irie, Shonosuke Sugasawa

Abstract: In this paper, we introduce a new and efficient data augmentation approach to the posterior inference of the models with shape parameters when the reciprocal gamma function appears in full conditional densities. Our approach is to approximate full conditional densities of shape parameters by using Gauss's multiplication formula and Stirling's formula for the gamma function, where the approximation… ▽ More In this paper, we introduce a new and efficient data augmentation approach to the posterior inference of the models with shape parameters when the reciprocal gamma function appears in full conditional densities. Our approach is to approximate full conditional densities of shape parameters by using Gauss's multiplication formula and Stirling's formula for the gamma function, where the approximation error can be made arbitrarily small. We use the techniques to construct efficient Gibbs and Metropolis-Hastings algorithms for a variety of models that involve the gamma distribution, Student's $t$-distribution, the Dirichlet distribution, the negative binomial distribution, and the Wishart distribution. The proposed sampling method is numerically demonstrated through simulation studies. △ Less

Submitted 26 August, 2022; v1 submitted 3 March, 2022; originally announced March 2022.

Comments: 41 pages, 6 figures

Locally Adaptive Spatial Quantile Smoothing: Application to Monitoring Crime Density in Tokyo

Authors: Takahiro Onizuka, Shintaro Hashimoto, Shonosuke Sugasawa

Abstract: Spatial trend estimation under potential heterogeneity is an important problem to extract spatial characteristics and hazards such as criminal activity. By focusing on quantiles, which provide substantial information on distributions compared with commonly used summary statistics such as means, it is often useful to estimate not only the average trend but also the high (low) risk trend additionall… ▽ More Spatial trend estimation under potential heterogeneity is an important problem to extract spatial characteristics and hazards such as criminal activity. By focusing on quantiles, which provide substantial information on distributions compared with commonly used summary statistics such as means, it is often useful to estimate not only the average trend but also the high (low) risk trend additionally. In this paper, we propose a Bayesian quantile trend filtering method to estimate the non-stationary trend of quantiles on graphs and apply it to crime data in Tokyo between 2013 and 2017. By modeling multiple observation cases, we can estimate the potential heterogeneity of spatial crime trends over multiple years in the application. To induce locally adaptive Bayesian inference on trends, we introduce general shrinkage priors for graph differences. Introducing so-called shadow priors with multivariate distribution for local scale parameters and mixture representation of the asymmetric Laplace distribution, we provide a simple Gibbs sampling algorithm to generate posterior samples. The numerical performance of the proposed method is demonstrated through simulation studies. △ Less

Submitted 23 October, 2023; v1 submitted 19 February, 2022; originally announced February 2022.

Comments: 38 pages, 9 figures

Dynamic Spatio-temporal Zero-inflated Poisson Models for Predicting Capelin Distribution in the Barents Sea

Authors: Shonosuke Sugasawa, Tomoyuki Nakagawa, Hiroko Kato Solvang, Sam Subbey, Salah Alrabeei

Abstract: We consider modeling and prediction of Capelin distribution in the Barents sea based on zero-inflated count observation data that vary continuously over a specified survey region. The model is a mixture of two components; a one-point distribution at the origin and a Poisson distribution with spatio-temporal intensity, where both intensity and mixing proportions are modeled by some auxiliary variab… ▽ More We consider modeling and prediction of Capelin distribution in the Barents sea based on zero-inflated count observation data that vary continuously over a specified survey region. The model is a mixture of two components; a one-point distribution at the origin and a Poisson distribution with spatio-temporal intensity, where both intensity and mixing proportions are modeled by some auxiliary variables and unobserved spatio-temporal effects. The spatio-temporal effects are modeled by a dynamic linear model combined with the predictive Gaussian process. We develop an efficient posterior computational algorithm for the model using a data augmentation strategy. The performance of the proposed model is demonstrated through simulation studies, and an application to the number of Capelin caught in the Barents sea from 2014 to 2019. △ Less

Submitted 19 October, 2022; v1 submitted 1 November, 2021; originally announced November 2021.

Comments: 25 pages

Adaptively Robust Small Area Estimation: Balancing Robustness and Efficiency of Empirical Bayes Confidence Intervals

Authors: Daisuke Kurisu, Takuya Ishihara, Shonosuke Sugasawa

Abstract: Empirical Bayes small area estimation based on the well-known Fay-Herriot model may produce unreliable estimates when outlying areas exist. Existing robust methods against outliers or model misspecification are generally inefficient when the assumed distribution is plausible. This paper proposes a simple modification of the standard empirical Bayes methods with adaptively balancing robustness and… ▽ More Empirical Bayes small area estimation based on the well-known Fay-Herriot model may produce unreliable estimates when outlying areas exist. Existing robust methods against outliers or model misspecification are generally inefficient when the assumed distribution is plausible. This paper proposes a simple modification of the standard empirical Bayes methods with adaptively balancing robustness and efficiency. The proposed method employs gamma-divergence instead of the marginal log-likelihood and optimizes a tuning parameter controlling robustness by pursuing the efficiency of empirical Bayes confidence intervals for areal parameters. We provide an asymptotic theory of the proposed method under both the correct specification of the assumed distribution and the existence of outlying areas. We investigate the numerical performance of the proposed method through simulations and an application to small area estimation of average crime numbers. △ Less

Submitted 27 June, 2022; v1 submitted 25 August, 2021; originally announced August 2021.

Comments: 20 pages (main text) + 19 pages (supplementary material)

Adaptively Robust Geographically Weighted Regression

Authors: Shonosuke Sugasawa, Daisuke Murakami

Abstract: We develop a new robust geographically weighted regression method in the presence of outliers. We embed the standard geographically weighted regression in robust objective function based on $γ$-divergence. A novel feature of the proposed approach is that two tuning parameters that control robustness and spatial smoothness are automatically tuned in a data-dependent manner. Further, the proposed me… ▽ More We develop a new robust geographically weighted regression method in the presence of outliers. We embed the standard geographically weighted regression in robust objective function based on $γ$-divergence. A novel feature of the proposed approach is that two tuning parameters that control robustness and spatial smoothness are automatically tuned in a data-dependent manner. Further, the proposed method can produce robust standard error estimates of the robust estimator and give us a reasonable quantity for local outlier detection. We demonstrate that the proposed method is superior to the existing robust version of geographically weighted regression through simulation and data analysis. △ Less

Submitted 14 October, 2021; v1 submitted 30 June, 2021; originally announced June 2021.

Comments: 22 pages

On Selection Criteria for the Tuning Parameter in Robust Divergence

Authors: Shonosuke Sugasawa, Shouto Yonekura

Abstract: While robust divergence such as density power divergence and $γ$-divergence is helpful for robust statistical inference in the presence of outliers, the tuning parameter that controls the degree of robustness is chosen in a rule-of-thumb, which may lead to an inefficient inference. We here propose a selection criterion based on an asymptotic approximation of the Hyvarinen score applied to an unnor… ▽ More While robust divergence such as density power divergence and $γ$-divergence is helpful for robust statistical inference in the presence of outliers, the tuning parameter that controls the degree of robustness is chosen in a rule-of-thumb, which may lead to an inefficient inference. We here propose a selection criterion based on an asymptotic approximation of the Hyvarinen score applied to an unnormalized model defined by robust divergence. The proposed selection criterion only requires first and second-order partial derivatives of an assumed density function with respect to observations, which can be easily computed regardless of the number of parameters. We demonstrate the usefulness of the proposed method via numerical studies using normal distributions and regularized linear regression. △ Less

Submitted 22 June, 2021; originally announced June 2021.

Comments: 15 pages

Robust Bayesian Modeling of Counts with Zero inflation and Outliers: Theoretical Robustness and Efficient Computation

Authors: Yasuyuki Hamura, Kaoru Irie, Shonosuke Sugasawa

Abstract: Count data with zero inflation and large outliers are ubiquitous in many scientific applications. However, posterior analysis under a standard statistical model, such as Poisson or negative binomial distribution, is sensitive to such contamination. This study introduces a novel framework for Bayesian modeling of counts that is robust to both zero inflation and large outliers. In doing so, we intro… ▽ More Count data with zero inflation and large outliers are ubiquitous in many scientific applications. However, posterior analysis under a standard statistical model, such as Poisson or negative binomial distribution, is sensitive to such contamination. This study introduces a novel framework for Bayesian modeling of counts that is robust to both zero inflation and large outliers. In doing so, we introduce rescaled beta distribution and adopt it to absorb undesirable effects from zero and outlying counts. The proposed approach has two appealing features: the efficiency of the posterior computation via a custom Gibbs sampling algorithm and a theoretically guaranteed posterior robustness, where extreme outliers are automatically removed from the posterior distribution. We demonstrate the usefulness of the proposed method by applying it to trend filtering and spatial modeling using predictive Gaussian processes. △ Less

Submitted 8 May, 2024; v1 submitted 19 June, 2021; originally announced June 2021.

Comments: 32 pages (main text) and 23 pages (supplementary material)

Adaptation of the Tuning Parameter in General Bayesian Inference with Robust Divergence

Authors: Shouto Yonekura, Shonosuke Sugasawa

Abstract: We introduce a methodology for robust Bayesian estimation with robust divergence (e.g., density power divergence or γ-divergence), indexed by a single tuning parameter. It is well known that the posterior density induced by robust divergence gives highly robust estimators against outliers if the tuning parameter is appropriately and carefully chosen. In a Bayesian framework, one way to find the op… ▽ More We introduce a methodology for robust Bayesian estimation with robust divergence (e.g., density power divergence or γ-divergence), indexed by a single tuning parameter. It is well known that the posterior density induced by robust divergence gives highly robust estimators against outliers if the tuning parameter is appropriately and carefully chosen. In a Bayesian framework, one way to find the optimal tuning parameter would be using evidence (marginal likelihood). However, we numerically illustrate that evidence induced by the density power divergence does not work to select the optimal tuning parameter since robust divergence is not regarded as a statistical model. To overcome the problems, we treat the exponential of robust divergence as an unnormalized statistical model, and we estimate the tuning parameter via minimizing the Hyvarinen score. We also provide adaptive computational methods based on sequential Monte Carlo (SMC) samplers, which enables us to obtain the optimal tuning parameter and samples from posterior distributions simultaneously. The empirical performance of the proposed method through simulations and an application to real data are also provided. △ Less

Submitted 30 June, 2022; v1 submitted 12 June, 2021; originally announced June 2021.

General Unbiased Estimating Equations for Variance Components in Linear Mixed Models

Authors: Tatsuya Kubokawa, Shonosuke Sugasawa, Hiromasa Tamae, Sanjay Chaudhuri

Abstract: This paper introduces a general framework for estimating variance components in the linear mixed models via general unbiased estimating equations, which include some well-used estimators such as the restricted maximum likelihood estimator. We derive the asymptotic covariance matrices and second-order biases under general estimating equations without assuming the normality of the underlying distrib… ▽ More This paper introduces a general framework for estimating variance components in the linear mixed models via general unbiased estimating equations, which include some well-used estimators such as the restricted maximum likelihood estimator. We derive the asymptotic covariance matrices and second-order biases under general estimating equations without assuming the normality of the underlying distributions and identify a class of second-order unbiased estimators of variance components. It is also shown that the asymptotic covariance matrices and second-order biases do not depend on whether the regression coefficients are estimated by the generalized or ordinary least squares methods. We carry out numerical studies to check the performance of the proposed method based on typical linear mixed models. △ Less

Submitted 16 May, 2021; originally announced May 2021.

Comments: 15 pages

Trend Filtering for Functional Data

Authors: Tomoya Wakayama, Shonosuke Sugasawa

Abstract: Despite increasing accessibility to function data, effective methods for flexibly estimating underlying functional trend are still scarce. We thereby develop functional version of trend filtering for estimating trend of functional data indexed by time or on general graph by extending the conventional trend filtering, a powerful nonparametric trend estimation technique, for scalar data. We formulat… ▽ More Despite increasing accessibility to function data, effective methods for flexibly estimating underlying functional trend are still scarce. We thereby develop functional version of trend filtering for estimating trend of functional data indexed by time or on general graph by extending the conventional trend filtering, a powerful nonparametric trend estimation technique, for scalar data. We formulate the new trend filtering by introducing penalty terms based on $L_2$-norm of the differences of adjacent trend functions. We develop an efficient iteration algorithm for optimizing the objective function obtained by orthonormal basis expansion. Furthermore, we introduce additional penalty terms to eliminate redundant basis functions, which leads to automatic adaptation of the number of basis functions. The tuning parameter in the proposed method is selected via cross validation. We demonstrate the proposed method through simulation studies and applications to real world datasets. △ Less

Submitted 18 February, 2022; v1 submitted 6 April, 2021; originally announced April 2021.

Comments: 29 pages

Spatially Clustered Regression

Authors: Shonosuke Sugasawa, Daisuke Murakami

Abstract: Spatial regression or geographically weighted regression models have been widely adopted to capture the effects of auxiliary information on a response variable of interest over a region. In contrast, relationships between response and auxiliary variables are expected to exhibit complex spatial patterns in many applications. This paper proposes a new approach for spatial regression, called spatiall… ▽ More Spatial regression or geographically weighted regression models have been widely adopted to capture the effects of auxiliary information on a response variable of interest over a region. In contrast, relationships between response and auxiliary variables are expected to exhibit complex spatial patterns in many applications. This paper proposes a new approach for spatial regression, called spatially clustered regression, to estimate possibly clustered spatial patterns of the relationships. We combine K-means-based clustering formulation and penalty function motivated from a spatial process known as Potts model for encouraging similar clustering in neighboring locations. We provide a simple iterative algorithm to fit the proposed method, scalable for large spatial datasets. Through simulation studies, the proposed method demonstrates its superior performance to existing methods even under the true structure does not admit spatial clustering. Finally, the proposed method is applied to crime event data in Tokyo and produces interpretable results for spatial patterns. The R code is available at https://github.com/sshonosuke/SCR. △ Less

Submitted 28 April, 2021; v1 submitted 3 November, 2020; originally announced November 2020.

Comments: 28 pages, 6 figures

Parametric Bootstrap Confidence Intervals for the Multivariate Fay-Herriot Model

Authors: Takumi Saegusa, Shonosuke Sugasawa, Partha Lahiri

Abstract: The multivariate Fay-Herriot model is quite effective in combining information through correlations among small area survey estimates of related variables or historical survey estimates of the same variable or both. Though the literature on small area estimation is already very rich, construction of second-order efficient confidence intervals from multivariate models have so far received very litt… ▽ More The multivariate Fay-Herriot model is quite effective in combining information through correlations among small area survey estimates of related variables or historical survey estimates of the same variable or both. Though the literature on small area estimation is already very rich, construction of second-order efficient confidence intervals from multivariate models have so far received very little attention. In this paper, we develop a parametric bootstrap method for constructing a second-order efficient confidence interval for a general linear combination of small area means using the multivariate Fay-Herriot normal model. The proposed parametric bootstrap method replaces difficult and tedious analytical derivations by the power of efficient algorithm and high speed computer. Moreover, the proposed method is more versatile than the analytical method because the parametric bootstrap method can be easily applied to any method of model parameter estimation and any specific structure of the variance-covariance matrix of the multivariate Fay-Herriot model avoiding all the cumbersome and time-consuming calculations required in the analytical method. We apply our proposed methodology in constructing confidence intervals for the median income of four-person families for the fifty states and the District of Columbia in the United States. Our data analysis demonstrates that the proposed parametric bootstrap method generally provides much shorter confidence intervals compared to the corresponding traditional direct method. Moreover, the confidence intervals obtained from the multivariate model is generally shorter than the corresponding univariate model indicating the potential advantage of exploiting correlations of median income of four-person families with median incomes of three and five person families. △ Less

Submitted 26 June, 2020; originally announced June 2020.

Comments: 21 pages

Grouped Generalized Estimating Equations for Longitudinal Data Analysis

Authors: Tsubasa Ito, Shonosuke Sugasawa

Abstract: Generalized estimating equation (GEE) is widely adopted for regression modeling for longitudinal data, taking account of potential correlations within the same subjects. Although the standard GEE assumes common regression coefficients among all the subjects, such an assumption may not be realistic when there is potential heterogeneity in regression coefficients among subjects. In this paper, we de… ▽ More Generalized estimating equation (GEE) is widely adopted for regression modeling for longitudinal data, taking account of potential correlations within the same subjects. Although the standard GEE assumes common regression coefficients among all the subjects, such an assumption may not be realistic when there is potential heterogeneity in regression coefficients among subjects. In this paper, we develop a flexible and interpretable approach, called grouped GEE analysis, to modeling longitudinal data with allowing heterogeneity in regression coefficients. The proposed method assumes that the subjects are divided into a finite number of groups and subjects within the same group share the same regression coefficient. We provide a simple algorithm for grou** subjects and estimating the regression coefficients simultaneously, and show the asymptotic properties of the proposed estimator. The number of groups can be determined by the cross-validation with averaging method. We demonstrate the proposed method through simulation studies and an application to a real dataset. △ Less

Submitted 8 July, 2022; v1 submitted 11 June, 2020; originally announced June 2020.

Comments: 59 pages

Log-Regularly Varying Scale Mixture of Normals for Robust Regression

Authors: Yasuyuki Hamura, Kaoru Irie, Shonosuke Sugasawa

Abstract: Linear regression with the classical normality assumption for the error distribution may lead to an undesirable posterior inference of regression coefficients due to the potential outliers. This paper considers the finite mixture of two components with thin and heavy tails as the error distribution, which has been routinely employed in applied statistics. For the heavily-tailed component, we intro… ▽ More Linear regression with the classical normality assumption for the error distribution may lead to an undesirable posterior inference of regression coefficients due to the potential outliers. This paper considers the finite mixture of two components with thin and heavy tails as the error distribution, which has been routinely employed in applied statistics. For the heavily-tailed component, we introduce the novel class of distributions; their densities are log-regularly varying and have heavier tails than those of Cauchy distribution, yet they are expressed as a scale mixture of normal distributions and enable the efficient posterior inference by Gibbs sampler. We prove the robustness to outliers of the posterior distributions under the proposed models with a minimal set of assumptions, which justifies the use of shrinkage priors with unbounded densities for the coefficient vector in the presence of outliers. The extensive comparison with the existing methods via simulation study shows the improved performance of our model in point and interval estimation, as well as its computational efficiency. Further, we confirm the posterior robustness of our method in the empirical study with the shrinkage priors for regression coefficients. △ Less

Submitted 9 January, 2021; v1 submitted 6 May, 2020; originally announced May 2020.

Comments: 62 pages

Predicting Infection of COVID-19 in Japan: State Space Modeling Approach

Authors: Genya Kobayashi, Shonosuke Sugasawa, Hiromasa Tamae, Takayuki Ozu

Abstract: The number of confirmed cases of the coronavirus disease (COVID-19) in Japan has been increasing day by day and has had a serious impact on the society especially after the declaration of the state of emergency on April 7, 2020. This study analyzes the real time data from March 1 to April 22, 2020 by adopting a sophisticated statistical modeling tool based on the state space model combined with th… ▽ More The number of confirmed cases of the coronavirus disease (COVID-19) in Japan has been increasing day by day and has had a serious impact on the society especially after the declaration of the state of emergency on April 7, 2020. This study analyzes the real time data from March 1 to April 22, 2020 by adopting a sophisticated statistical modeling tool based on the state space model combined with the well-known susceptible-exposed-infected (SIR) model. The model estimation and forecasting are conducted using the Bayesian methodology. The present study provides the parameter estimates of the unknown parameters that critically determine the epidemic process derived from the SIR model and prediction of the future transition of the infectious proportion including the size and timing of the epidemic peak with the prediction intervals that naturally accounts for the uncertainty. The prediction results under various scenarios reveals that the temporary reduction in the infection rate until the planned lifting of the state on May 6 will only delay the epidemic peak slightly. In order to minimize the spread of the epidemic, it is strongly suggested that an intervention is carried out for an extended period of time and that the government and individuals make a long term effort to reduce the infection rate even after the lifting. △ Less

Submitted 28 April, 2020; originally announced April 2020.

Comments: 12 pages (main part) + 9 pages (supplement)

Robust Fitting of Mixture Models using Weighted Complete Estimating Equations

Authors: Shonosuke Sugasawa, Genya Kobayashi

Abstract: Mixture modeling, which considers the potential heterogeneity in data, is widely adopted for classification and clustering problems. Mixture models can be estimated using the Expectation-Maximization algorithm, which works with the complete estimating equations conditioned by the latent membership variables of the cluster assignment based on the hierarchical expression of mixture models. However,… ▽ More Mixture modeling, which considers the potential heterogeneity in data, is widely adopted for classification and clustering problems. Mixture models can be estimated using the Expectation-Maximization algorithm, which works with the complete estimating equations conditioned by the latent membership variables of the cluster assignment based on the hierarchical expression of mixture models. However, when the mixture components have light tails such as a normal distribution, the mixture model can be sensitive to outliers. This study proposes a method of weighted complete estimating equations (WCE) for the robust fitting of mixture models. Our WCE introduces weights to complete estimating equations such that the weights can automatically downweight the outliers. The weights are constructed similarly to the density power divergence for mixture models, but in our WCE, they depend only on the component distributions and not on the whole mixture. A novel expectation-estimating-equation (EEE) algorithm is also developed to solve the WCE. For illustrative purposes, a multivariate Gaussian mixture, a mixture of experts, and a multivariate skew normal mixture are considered, and how our EEE algorithm can be implemented for these specific models is described. The numerical performance of the proposed robust estimation method was examined using simulated and real datasets. △ Less

Submitted 16 March, 2022; v1 submitted 7 April, 2020; originally announced April 2020.

Comments: 40 pages

Efficient testing and effect size estimation for set-based genetic association inference via semiparametric multilevel mixture modeling: Application to a genome-wide association study of coronary artery disease

Authors: Shonosuke Sugasawa, Hisashi Noma

Abstract: In genetic association studies, rare variants with extremely small allele frequency play a crucial role in complex traits, and the set-based testing methods that jointly assess the effects of groups of single nucleotide polymorphisms (SNPs) were developed to improve powers for the association tests. However, the powers of these tests are still severely limited due to the extremely small allele fre… ▽ More In genetic association studies, rare variants with extremely small allele frequency play a crucial role in complex traits, and the set-based testing methods that jointly assess the effects of groups of single nucleotide polymorphisms (SNPs) were developed to improve powers for the association tests. However, the powers of these tests are still severely limited due to the extremely small allele frequency, and precise estimations for the effect sizes of individual SNPs are substantially impossible. In this article, we provide an efficient set-based inference framework that addresses the two important issues simultaneously based on a Bayesian semiparametric multilevel mixture model. We propose to use the multilevel hierarchical model that incorporate the variations in set-specific effects and variant-specific effects, and to apply the optimal discovery procedure (ODP) that achieves the largest overall power in multiple significance testing. In addition, we provide Bayesian optimal "set-based" estimator of the empirical distribution of effect sizes. Efficiency of the proposed methods is demonstrated through application to a genome-wide association study of coronary artery disease (CAD), and through simulation studies. These results suggested there could be a lot of rare variants with large effect sizes for CAD, and the number of significant sets detected by the ODP was much greater than those by existing methods. △ Less

Submitted 12 March, 2020; originally announced March 2020.

Comments: 22 pages

Shrinkage with Robustness: Log-Adjusted Priors for Sparse Signals

Authors: Yasuyuki Hamura, Kaoru Irie, Shonosuke Sugasawa

Abstract: We introduce a new class of distributions named log-adjusted shrinkage priors for the analysis of sparse signals, which extends the three parameter beta priors by multiplying an additional log-term to their densities. The proposed prior has density tails that are heavier than even those of the Cauchy distribution and realizes the tail-robustness of the Bayes estimator, while kee** the strong shr… ▽ More We introduce a new class of distributions named log-adjusted shrinkage priors for the analysis of sparse signals, which extends the three parameter beta priors by multiplying an additional log-term to their densities. The proposed prior has density tails that are heavier than even those of the Cauchy distribution and realizes the tail-robustness of the Bayes estimator, while kee** the strong shrinkage effect on noises. We verify this property via the improved posterior mean squared errors in the tail. An integral representation with latent variables for the new density is available and enables fast and simple Gibbs samplers for the full posterior analysis. Our log-adjusted prior is significantly different from existing shrinkage priors with logarithms for allowing its further generalization by multiple log-terms in the density. The performance of the proposed priors is investigated through simulation studies and data analysis. △ Less

Submitted 26 January, 2020; v1 submitted 23 January, 2020; originally announced January 2020.

Comments: 40 pages

Robust Bayesian Regression with Synthetic Posterior

Authors: Shintaro Hashimoto, Shonosuke Sugasawa

Abstract: Although linear regression models are fundamental tools in statistical science, the estimation results can be sensitive to outliers. While several robust methods have been proposed in frequentist frameworks, statistical inference is not necessarily straightforward. We here propose a Bayesian approach to robust inference on linear regression models using synthetic posterior distributions based on… ▽ More Although linear regression models are fundamental tools in statistical science, the estimation results can be sensitive to outliers. While several robust methods have been proposed in frequentist frameworks, statistical inference is not necessarily straightforward. We here propose a Bayesian approach to robust inference on linear regression models using synthetic posterior distributions based on $γ$-divergence, which enables us to naturally assess the uncertainty of the estimation through the posterior distribution. We also consider the use of shrinkage priors for the regression coefficients to carry out robust Bayesian variable selection and estimation simultaneously. We develop an efficient posterior computation algorithm by adopting the Bayesian bootstrap within Gibbs sampling. The performance of the proposed method is illustrated through simulation studies and applications to famous datasets. △ Less

Submitted 26 May, 2020; v1 submitted 2 October, 2019; originally announced October 2019.

Comments: 23 pages, 5 figures

Bayesian Semiparametric Modeling of Response Mechanism for Nonignorable Missing Data

Authors: Shonosuke Sugasawa, Kosuke Morikawa, Keisuke Takahata

Abstract: Statistical inference with nonresponse is quite challenging, especially when the response mechanism is nonignorable. In this case, the validity of statistical inference depends on untestable correct specification of the response model. To avoid the misspecification, we propose semiparametric Bayesian estimation in which an outcome model is parametric, but the response model is semiparametric in th… ▽ More Statistical inference with nonresponse is quite challenging, especially when the response mechanism is nonignorable. In this case, the validity of statistical inference depends on untestable correct specification of the response model. To avoid the misspecification, we propose semiparametric Bayesian estimation in which an outcome model is parametric, but the response model is semiparametric in that we do not assume any parametric form for the nonresponse variable. We adopt penalized spline methods to estimate the unknown function. We also consider a fully nonparametric approach to modeling the response mechanism by using radial basis function methods. Using Polya-gamma data augmentation, we developed an efficient posterior computation algorithm via Gibbs sampling in which most full conditional distributions can be obtained in familiar forms. The performance of the proposed method is demonstrated in simulation studies and an application to longitudinal data. △ Less

Submitted 14 January, 2021; v1 submitted 6 September, 2019; originally announced September 2019.

Comments: 25 pages; The title has been changed from "Bayesian semiparametric estimation under nonignorable nonresponse"

Bayesian approach to Lorenz curve using time series grouped data

Authors: Genya Kobayashi, Yuta Yamauchi, Kazuhiko Kakamu, Yuki Kawakubo, Shonosuke Sugasawa

Abstract: This study is concerned with estimating the inequality measures associated with the underlying hypothetical income distribution from the times series grouped data on the Lorenz curve. We adopt the Dirichlet pseudo likelihood approach where the parameters of the Dirichlet likelihood are set to the differences between the Lorenz curve of the hypothetical income distribution for the consecutive incom… ▽ More This study is concerned with estimating the inequality measures associated with the underlying hypothetical income distribution from the times series grouped data on the Lorenz curve. We adopt the Dirichlet pseudo likelihood approach where the parameters of the Dirichlet likelihood are set to the differences between the Lorenz curve of the hypothetical income distribution for the consecutive income classes and propose a state space model which combines the transformed parameters of the Lorenz curve through a time series structure. Furthermore, the information on the sample size in each survey is introduced into the originally nuisance Dirichlet precision parameter to take into account the variability from the sampling. From the simulated data and real data on the Japanese monthly income survey, it is confirmed that the proposed model produces more efficient estimates on the inequality measures than the existing models without time series structures. △ Less

Submitted 19 August, 2019; originally announced August 2019.

On Global-local Shrinkage Priors for Count Data

Authors: Yasuyuki Hamura, Kaoru Irie, Shonosuke Sugasawa

Abstract: Global-local shrinkage prior has been recognized as useful class of priors which can strongly shrink small signals towards prior means while kee** large signals unshrunk. Although such priors have been extensively discussed under Gaussian responses, we intensively encounter count responses in practice in which the previous knowledge of global-local shrinkage priors cannot be directly imported. I… ▽ More Global-local shrinkage prior has been recognized as useful class of priors which can strongly shrink small signals towards prior means while kee** large signals unshrunk. Although such priors have been extensively discussed under Gaussian responses, we intensively encounter count responses in practice in which the previous knowledge of global-local shrinkage priors cannot be directly imported. In this paper, we discuss global-local shrinkage priors for analyzing sequence of counts. We provide sufficient conditions under which the posterior mean keeps the observation as it is for very large signals, known as tail robustness property. Then, we propose tractable priors to meet the derived conditions approximately or exactly and develop an efficient posterior computation algorithm for Bayesian inference. The proposed methods are free from tuning parameters, that is, all the hyperparameters are automatically estimated based on the data. We demonstrate the proposed methods through simulation and an application to a real dataset. △ Less

Submitted 16 August, 2020; v1 submitted 2 July, 2019; originally announced July 2019.

Comments: 28 pages (main text) + 14 pages (supplementary material)

Improved Confidence Regions in Meta-analysis of Diagnostic Test Accuracy

Authors: Tsubasa Ito, Shonosuke Sugasawa

Abstract: Meta-analyses of diagnostic test accuracy (DTA) studies have been gathering attention in research in clinical epidemiology and health technology development, and bivariate random-effects model is becoming a standard tool. However, standard inference methods usually underestimate statistical errors and possibly provide highly overconfident results under realistic situations since they ignore the va… ▽ More Meta-analyses of diagnostic test accuracy (DTA) studies have been gathering attention in research in clinical epidemiology and health technology development, and bivariate random-effects model is becoming a standard tool. However, standard inference methods usually underestimate statistical errors and possibly provide highly overconfident results under realistic situations since they ignore the variability in the estimation of variance parameters. To overcome the difficulty, a new improved inference method, namely, an accurate confidence region for the meta-analysis of DTA, by asymptotically expanding the coverage probability of the standard confidence region. The advantage of the proposed confidence region is that it holds a relatively simple expression and does not require any repeated calculations such as Bootstrap or Monte Carlo methods to compute the region, thereby the proposed method can be easily carried out in practical applications. The effectiveness of the proposed method is demonstrated through simulation studies and an application to meta-analysis of screening test accuracy for alcohol problems. △ Less

Submitted 18 June, 2020; v1 submitted 19 June, 2019; originally announced June 2019.

Comments: 15 pages (main text) + 10 pages (supplementary material)

An Approximate Bayesian Approach to Model-assisted Survey Estimation with Many Auxiliary Variables

Authors: Shonosuke Sugasawa, Jae Kwang Kim

Abstract: Model-assisted estimation with complex survey data is an important practical problem in survey sampling. When there are many auxiliary variables, selecting significant variables associated with the study variable would be necessary to achieve efficient estimation of population parameters of interest. In this paper, we formulate a regularized regression estimator in the framework of Bayesian infere… ▽ More Model-assisted estimation with complex survey data is an important practical problem in survey sampling. When there are many auxiliary variables, selecting significant variables associated with the study variable would be necessary to achieve efficient estimation of population parameters of interest. In this paper, we formulate a regularized regression estimator in the framework of Bayesian inference using the penalty function as the shrinkage prior for model selection. The proposed Bayesian approach enables us to get not only efficient point estimates but also reasonable credible intervals. Results from two limited simulation studies are presented to facilitate comparison with existing frequentist methods. △ Less

Submitted 31 March, 2020; v1 submitted 11 June, 2019; originally announced June 2019.

Comments: 37 pages

Efficient screening of predictive biomarkers for individual treatment selection

Authors: Shonosuke Sugasawa, Hisashi Noma

Abstract: The development of molecular diagnostic tools to achieve individualized medicine requires identifying predictive biomarkers associated with subgroups of individuals who might receive beneficial or harmful effects from different available treatments. However, due to the large number of candidate biomarkers in the large-scale genetic and molecular studies, and complex relationships among clinical ou… ▽ More The development of molecular diagnostic tools to achieve individualized medicine requires identifying predictive biomarkers associated with subgroups of individuals who might receive beneficial or harmful effects from different available treatments. However, due to the large number of candidate biomarkers in the large-scale genetic and molecular studies, and complex relationships among clinical outcome, biomarkers and treatments, the ordinary statistical tests for the interactions between treatments and covariates have difficulties from their limited statistical powers. In this paper, we propose an efficient method for detecting predictive biomarkers. We employ weighted loss functions of Chen et al. (2017) to directly estimate individual treatment scores and propose synthetic posterior inference for effect sizes of biomarkers. We develop an empirical Bayes approach, namely, we estimate unknown hyperparameters in the prior distribution based on data. We then provide efficient screening methods for the candidate biomarkers via optimal discovery procedure with adequate control of false discovery rate. The proposed method is demonstrated in simulation studies and an application to a breast cancer clinical study in which the proposed method was shown to detect the much larger numbers of significant biomarkers than existing standard methods. △ Less

Submitted 17 January, 2020; v1 submitted 4 May, 2019; originally announced May 2019.

Comments: 22 pages

Estimation and inference for area-wise spatial income distributions from grouped data

Authors: Shonosuke Sugasawa, Genya Kobayashi, Yuki Kawakubo

Abstract: Estimating income distributions plays an important role in the measurement of inequality and poverty over space. The existing literature on income distributions predominantly focuses on estimating an income distribution for a country or a region separately and the simultaneous estimation of multiple income distributions has not been discussed in spite of its practical importance. In this work, we… ▽ More Estimating income distributions plays an important role in the measurement of inequality and poverty over space. The existing literature on income distributions predominantly focuses on estimating an income distribution for a country or a region separately and the simultaneous estimation of multiple income distributions has not been discussed in spite of its practical importance. In this work, we develop an effective method for the simultaneous estimation and inference for area-wise spatial income distributions taking account of geographical information from grouped data. Based on the multinomial likelihood function for grouped data, we propose a spatial state-space model for area-wise parameters of parametric income distributions. We provide an efficient Bayesian approach to estimation and inference for area-wise latent parameters, which enables us to compute area-wise summary measures of income distributions such as mean incomes and Gini indices, not only for sampled areas but also for areas without any samples thanks to the latent spatial state-space structure. The proposed method is demonstrated using the Japanese municipality-wise grouped income data. The simulation studies show the superiority of the proposed method to a crude conventional approach which estimates the income distributions separately. △ Less

Submitted 3 July, 2019; v1 submitted 24 April, 2019; originally announced April 2019.

Comments: 25 pages

Grouped Heterogeneous Mixture Modeling for Clustered Data

Authors: Shonosuke Sugasawa

Abstract: Clustered data is ubiquitous in a variety of scientific fields. In this paper, we propose a flexible and interpretable modeling approach, called grouped heterogenous mixture modeling, for clustered data, which models cluster-wise conditional distributions by mixtures of latent conditional distributions common to all the clusters. In the model, we assume that clusters are divided into a finite numb… ▽ More Clustered data is ubiquitous in a variety of scientific fields. In this paper, we propose a flexible and interpretable modeling approach, called grouped heterogenous mixture modeling, for clustered data, which models cluster-wise conditional distributions by mixtures of latent conditional distributions common to all the clusters. In the model, we assume that clusters are divided into a finite number of groups and mixing proportions are the same within the same group. We provide a simple generalized EM algorithm for computing the maximum likelihood estimator, and an information criterion to select the numbers of groups and latent distributions. We also propose structured grou** strategies by introducing penalties on grou** parameters in the likelihood function. Under the settings where both the number of clusters and cluster sizes tend to infinity, we present asymptotic properties of the maximum likelihood estimator and the information criterion. We demonstrate the proposed method through simulation studies and an application to crime risk modeling in Tokyo. △ Less

Submitted 6 February, 2020; v1 submitted 3 April, 2018; originally announced April 2018.

Comments: 34 pages

A Unified Method for Improved Inference in Random-effects Meta-analysis

Authors: Shonosuke Sugasawa, Hisashi Noma

Abstract: Random-effects meta-analyses have been widely applied in evidence synthesis for various types of medical studies. However, standard inference methods (e.g. restricted maximum likelihood estimation) usually underestimate statistical errors and possibly provide highly overconfident results under realistic situations; for instance, coverage probabilities of confidence intervals can be substantially b… ▽ More Random-effects meta-analyses have been widely applied in evidence synthesis for various types of medical studies. However, standard inference methods (e.g. restricted maximum likelihood estimation) usually underestimate statistical errors and possibly provide highly overconfident results under realistic situations; for instance, coverage probabilities of confidence intervals can be substantially below the nominal level. The main reason is that these inference methods rely on large sample approximations even though the number of synthesized studies is usually small or moderate in practice. In this article we solve this problem using a unified inference method based on Monte Carlo conditioning for broad application to random-effects meta-analysis. The developed method provides improved confidence intervals with coverage probabilities that are closer to the nominal level than standard methods. As specific applications, we provide new inference procedures for three types of meta-analysis: conventional univariate meta-analysis for pairwise treatment comparisons, meta-analysis of diagnostic test accuracy, and multiple treatment comparisons via network meta-analysis. We also illustrate the practical effectiveness of these methods via real data applications and simulation studies. △ Less

Submitted 9 May, 2019; v1 submitted 16 November, 2017; originally announced November 2017.

Comments: 29 pages

Adaptively Transformed Mixed Model Prediction of General Finite Population Parameters

Authors: Shonosuke Sugasawa, Tatsuya Kubokawa

Abstract: For estimating area-specific parameters (quantities) in a finite population, a mixed model prediction approach is attractive. However, this approach strongly depends on the normality assumption of the response values although we often encounter a non-normal case in practice. In such a case, transforming observations to make them suitable for normality assumption is a useful tool, but the problem o… ▽ More For estimating area-specific parameters (quantities) in a finite population, a mixed model prediction approach is attractive. However, this approach strongly depends on the normality assumption of the response values although we often encounter a non-normal case in practice. In such a case, transforming observations to make them suitable for normality assumption is a useful tool, but the problem of selecting suitable transformation still remains open. To overcome the difficulty, we here propose a new empirical best predicting method by using a parametric family of transformations to estimate a suitable transformation based on the data. We suggest a simple estimating method for transformation parameters based on the profile likelihood function, which achieves consistency under some conditions on transformation functions. For measuring variability of point prediction, we construct an empirical Bayes confidence interval of the population parameter of interest. Through simulation studies, we investigate numerical performance of the proposed methods. Finally, we apply the proposed method to synthetic income data in Spanish provinces in which the resulting estimates indicate that the commonly used log-transformation would not be appropriate. △ Less

Submitted 11 June, 2018; v1 submitted 11 May, 2017; originally announced May 2017.

Comments: 32 pages

On Bootstrap Averaging Empirical Bayes Estimators

Authors: Shonosuke Sugasawa

Abstract: Parametric empirical Bayes (EB) estimators have been widely used in variety of fields including small area estimation, disease map**. Since EB estimator is constructed by plugging in the estimator of parameters in prior distributions, it might perform poorly if the estimator of parameters is unstable. This can happen when the number of samples are small or moderate. This paper suggests bootstrap… ▽ More Parametric empirical Bayes (EB) estimators have been widely used in variety of fields including small area estimation, disease map**. Since EB estimator is constructed by plugging in the estimator of parameters in prior distributions, it might perform poorly if the estimator of parameters is unstable. This can happen when the number of samples are small or moderate. This paper suggests bootstrap** averaging approach, known as "bagging" in machine learning literatures, to improve the performances of EB estimators. We consider two typical hierarchical models, two-stage normal hierarchical model and Poisson-gamma model, and compare the proposed method with the classical parametric EB method through simulation and empirical studies. △ Less

Submitted 27 April, 2017; originally announced April 2017.

Comments: 10 pages