Showing 1–37 of 37 results for author: Kneib, T

Bayesian Penalized Transformation Models: Structured Additive Location-Scale Regression for Arbitrary Conditional Distributions

Authors: Johannes Brachem, Paul F. V. Wiemann, Thomas Kneib

Abstract: Penalized transformation models (PTMs) are a novel form of location-scale regression. In PTMs, the shape of the response's conditional distribution is estimated directly from the data, and structured additive predictors are placed on its location and scale. The core of the model is a monotonically increasing transformation function that relates the response distribution to a reference distribution… ▽ More Penalized transformation models (PTMs) are a novel form of location-scale regression. In PTMs, the shape of the response's conditional distribution is estimated directly from the data, and structured additive predictors are placed on its location and scale. The core of the model is a monotonically increasing transformation function that relates the response distribution to a reference distribution. The transformation function is equipped with a smoothness prior that regularizes how much the estimated distribution diverges from the reference distribution. These models can be seen as a bridge between conditional transformation models and generalized additive models for location, scale and shape. Markov chain Monte Carlo inference for PTMs can be conducted with the No-U-Turn sampler and offers straightforward uncertainty quantification for the conditional distribution as well as for the covariate effects. A simulation study demonstrates the effectiveness of the approach. We apply the model to data from the Fourth Dutch Growth Study and the Framingham Heart Study. A full-featured implementation is available as a Python library. △ Less

Submitted 7 May, 2024; v1 submitted 10 April, 2024; originally announced April 2024.

Demystifying Spatial Confounding

Authors: Emiko Dupont, Isa Marques, Thomas Kneib

Abstract: Spatial confounding is a fundamental issue in regression models for spatially indexed data. It arises because spatial random effects, included to approximate unmeasured spatial variation, are typically not independent of the covariates in the model. This can lead to significant bias in covariate effect estimates. Despite extensive research, it is still a topic of much confusion with sometimes puzz… ▽ More Spatial confounding is a fundamental issue in regression models for spatially indexed data. It arises because spatial random effects, included to approximate unmeasured spatial variation, are typically not independent of the covariates in the model. This can lead to significant bias in covariate effect estimates. Despite extensive research, it is still a topic of much confusion with sometimes puzzling and seemingly contradictory results. In this paper we develop a broad theoretical framework that brings mathematical clarity to the mechanisms of spatial confounding, providing explicit and interpretable analytical expressions for the resulting bias. From these, we see that it is a problem directly linked to spatial smoothing, and we can identify exactly how the features of the model and the data generation process affect the size and occurrence of bias. We also use our framework to understand and generalise some of the main results on spatial confounding in the past, including suggested methods for bias adjustment. Thus, our comprehensive and mathematically explicit approach clears up existing confusion and, indeed, demystifies the issue of spatial confounding. △ Less

Submitted 28 September, 2023; originally announced September 2023.

A simplified spatial+ approach to mitigate spatial confounding in multivariate spatial areal models

Authors: A. Urdangarin, T. Goicoa, T. Kneib, M. D. Ugarte

Abstract: Spatial areal models encounter the well-known and challenging problem of spatial confounding. This issue makes it arduous to distinguish between the impacts of observed covariates and spatial random effects. Despite previous research and various proposed methods to tackle this problem, finding a definitive solution remains elusive. In this paper, we propose a simplified version of the spatial+ app… ▽ More Spatial areal models encounter the well-known and challenging problem of spatial confounding. This issue makes it arduous to distinguish between the impacts of observed covariates and spatial random effects. Despite previous research and various proposed methods to tackle this problem, finding a definitive solution remains elusive. In this paper, we propose a simplified version of the spatial+ approach that involves dividing the covariate into two components. One component captures large-scale spatial dependence, while the other accounts for short-scale dependence. This approach eliminates the need to separately fit spatial models for the covariates. We apply this method to analyse two forms of crimes against women, namely rapes and dowry deaths, in Uttar Pradesh, India, exploring their relationship with socio-demographic covariates. To evaluate the performance of the new approach, we conduct extensive simulation studies under different spatial confounding scenarios. The results demonstrate that the proposed method provides reliable estimates of fixed effects and posterior correlations between different responses. △ Less

Submitted 5 January, 2024; v1 submitted 22 August, 2023; originally announced August 2023.

Journal ref: Spatial Statistics (2024)

"Spatial Joint Models through Bayesian Structured Piece-wise Additive Joint Modelling for Longitudinal and Time-to-Event Data"

Authors: Anja Rappl, Thomas Kneib, Stefan Lang, Elisabeth Bergherr

Abstract: Joint models for longitudinal and time-to-event data have seen many developments in recent years. Though spatial joint models are still rare and the traditional proportional hazards formulation of the time-to-event part of the model is accompanied by computational challenges. We propose a joint model with a piece-wise exponential formulation of the hazard using the counting process representation… ▽ More Joint models for longitudinal and time-to-event data have seen many developments in recent years. Though spatial joint models are still rare and the traditional proportional hazards formulation of the time-to-event part of the model is accompanied by computational challenges. We propose a joint model with a piece-wise exponential formulation of the hazard using the counting process representation of a hazard and structured additive predictors able to estimate (non-)linear, spatial and random effects. Its capabilities are assessed in a simulation study comparing our approach to an established one and highlighted by an example on physical functioning after cardiovascular events from the German Ageing Survey. The Structured Piecewise Additive Joint Model yielded good estimation performance, also and especially in spatial effects, while being double as fast as the chosen benchmark approach and performing stable in imbalanced data setting with few events. △ Less

Submitted 14 February, 2023; originally announced February 2023.

Neural Additive Models for Location Scale and Shape: A Framework for Interpretable Neural Regression Beyond the Mean

Authors: Anton Thielmann, René-Marcel Kruse, Thomas Kneib, Benjamin Säfken

Abstract: Deep neural networks (DNNs) have proven to be highly effective in a variety of tasks, making them the go-to method for problems requiring high-level predictive power. Despite this success, the inner workings of DNNs are often not transparent, making them difficult to interpret or understand. This lack of interpretability has led to increased research on inherently interpretable neural networks in… ▽ More Deep neural networks (DNNs) have proven to be highly effective in a variety of tasks, making them the go-to method for problems requiring high-level predictive power. Despite this success, the inner workings of DNNs are often not transparent, making them difficult to interpret or understand. This lack of interpretability has led to increased research on inherently interpretable neural networks in recent years. Models such as Neural Additive Models (NAMs) achieve visual interpretability through the combination of classical statistical methods with DNNs. However, these approaches only concentrate on mean response predictions, leaving out other properties of the response distribution of the underlying data. We propose Neural Additive Models for Location Scale and Shape (NAMLSS), a modelling framework that combines the predictive power of classical deep learning models with the inherent advantages of distributional regression while maintaining the interpretability of additive models. The code is available at the following link: https://github.com/AnFreTh/NAMpy △ Less

Submitted 29 February, 2024; v1 submitted 27 January, 2023; originally announced January 2023.

Comments: Accepted at the 27th International Conference on Artificial Intelligence and Statistics (AISTATS) 2024

Liesel: A Probabilistic Programming Framework for Develo** Semi-Parametric Regression Models and Custom Bayesian Inference Algorithms

Authors: Hannes Riebl, Paul F. V. Wiemann, Thomas Kneib

Abstract: Liesel is a new probabilistic programming framework developed with the aim of supporting research on Bayesian inference based on Markov chain Monte Carlo (MCMC) simulations in general and semi-parametric regression specifications in particular. Its three main components are (i) an R interface (RLiesel) for the configuration of an initial semi-parametric regression model, (ii) a graph-based model b… ▽ More Liesel is a new probabilistic programming framework developed with the aim of supporting research on Bayesian inference based on Markov chain Monte Carlo (MCMC) simulations in general and semi-parametric regression specifications in particular. Its three main components are (i) an R interface (RLiesel) for the configuration of an initial semi-parametric regression model, (ii) a graph-based model building library, where the initial model graph can be manipulated to incorporate new research ideas, and (iii) an MCMC library for designing modular inference algorithms combining multiple types of well-tested and possibly customized MCMC kernels. The graph builder as well as the MCMC library are implemented in Python, relying on JAX as a numerical computing library, and can therefore benefit from the latest machine learning technology such as automatic differentiation, just-in-time (JIT) compilation, and the use of high-performance computing devices such as tensor processing units (TPUs). Liesel provides all required tools for efficient and reliable statistical research on complex models and estimation algorithms. Its modular design allows users to expand the model library and inference algorithms, offering the flexibility and customization options to tailor the software to any specific research needs. △ Less

Submitted 29 November, 2023; v1 submitted 22 September, 2022; originally announced September 2022.

Comments: 27 pages, 10 figures, updated for compatibility with Liesel v0.2.7, added second case study

Multivariate Distributional Stochastic Frontier Models

Authors: Rouven Schmidt, Thomas Kneib

Abstract: The primary objective of Stochastic Frontier (SF) Analysis is the deconvolution of the estimated composed error terms into noise and inefficiency. Assuming a parametric production function (e.g. Cobb-Douglas, Translog, etc.), might lead to false inefficiency estimates. To overcome this limiting assumption, the production function can be modelled utilizing P-splines. Application of this powerful an… ▽ More The primary objective of Stochastic Frontier (SF) Analysis is the deconvolution of the estimated composed error terms into noise and inefficiency. Assuming a parametric production function (e.g. Cobb-Douglas, Translog, etc.), might lead to false inefficiency estimates. To overcome this limiting assumption, the production function can be modelled utilizing P-splines. Application of this powerful and flexible tool enables modelling of a wide range of production functions. Additionally, one can allow the parameters of the composed error distribution to depend on covariates in a functional form. The SF model can then be cast into the framework of a Generalized Additive Model for Location, Scale and Shape (GAMLSS). Furthermore, a decision-making unit (DMU) typically produces multiple outputs. It does this by operating several sub-DMUs, which each employ a production process to produce a single output. Therefore, the production processes of the sub-DMUs are typically not independent. Consequently, the inefficiencies may be expected to be dependent, too. In this paper, the Distributional Stochastic Frontier Model (DSFM) is introduced. The multivariate distribution of the composed error term is modeled using a copula. As a result, the presented model is a generalization of the model for seemingly unrelated stochastic frontier regressions by Lai and Huang (2013). △ Less

Submitted 22 August, 2022; originally announced August 2022.

Comments: 17 pages, 3 figures

Twitmo: A Twitter Data Topic Modeling and Visualization Package for R

Authors: Andreas Buchmüller, Gillian Kant, Christoph Weisser, Benjamin Säfken, Krisztina Kis-Katos, Thomas Kneib

Abstract: We present Twitmo, a package that provides a broad range of methods to collect, pre-process, analyze and visualize geo-tagged Twitter data. Twitmo enables the user to collect geo-tagged Tweets from Twitter and and provides a comprehensive and user-friendly toolbox to generate topic distributions from Latent Dirichlet Allocations (LDA), correlated topic models (CTM) and structural topic models (STM… ▽ More We present Twitmo, a package that provides a broad range of methods to collect, pre-process, analyze and visualize geo-tagged Twitter data. Twitmo enables the user to collect geo-tagged Tweets from Twitter and and provides a comprehensive and user-friendly toolbox to generate topic distributions from Latent Dirichlet Allocations (LDA), correlated topic models (CTM) and structural topic models (STM). Functions are included for pre-processing of text, model building and prediction. In addition, one of the innovations of the package is the automatic pooling of Tweets into longer pseudo-documents using hashtags and cosine similarities for better topic coherence. The package additionally comes with functionality to visualize collected data sets and fitted models in static as well as interactive ways and offers built-in support for model visualizations via LDAvis providing great convenience for researchers in this area. The Twitmo package is an innovative toolbox that can be used to analyze public discourse of various topics, political parties or persons of interest in space and time. △ Less

Submitted 8 July, 2022; originally announced July 2022.

Comments: 16 pages, 4 figures

MSC Class: 68N30 (Primary) 62P25; 97K80 (Secondary)

Bayesian Discrete Conditional Transformation Models

Authors: Manuel Carlan, Thomas Kneib

Abstract: We propose a novel Bayesian model framework for discrete ordinal and count data based on conditional transformations of the responses. The conditional transformation function is estimated from the data in conjunction with an a priori chosen reference distribution. For count responses, the resulting transformation model is novel in the sense that it is a Bayesian fully parametric yet distribution-f… ▽ More We propose a novel Bayesian model framework for discrete ordinal and count data based on conditional transformations of the responses. The conditional transformation function is estimated from the data in conjunction with an a priori chosen reference distribution. For count responses, the resulting transformation model is novel in the sense that it is a Bayesian fully parametric yet distribution-free approach that can additionally account for excess zeros with additive transformation function specifications. For ordinal categoric responses, our cumulative link transformation model allows the inclusion of linear and nonlinear covariate effects that can additionally be made category-specific, resulting in (non-)proportional odds or hazards models and more, depending on the choice of the reference distribution. Inference is conducted by a generic modular Markov chain Monte Carlo algorithm where multivariate Gaussian priors enforce specific properties such as smoothness on the functional effects. To illustrate the versatility of Bayesian discrete conditional transformation models, applications to counts of patent citations in the presence of excess zeros and on treating forest health categories in a discrete partial proportional odds model are presented. △ Less

Submitted 17 May, 2022; originally announced May 2022.

Distributional Gradient Boosting Machines

Authors: Alexander März, Thomas Kneib

Abstract: We present a unified probabilistic gradient boosting framework for regression tasks that models and predicts the entire conditional distribution of a univariate response variable as a function of covariates. Our likelihood-based approach allows us to either model all conditional moments of a parametric distribution, or to approximate the conditional cumulative distribution function via Normalizing… ▽ More We present a unified probabilistic gradient boosting framework for regression tasks that models and predicts the entire conditional distribution of a univariate response variable as a function of covariates. Our likelihood-based approach allows us to either model all conditional moments of a parametric distribution, or to approximate the conditional cumulative distribution function via Normalizing Flows. As underlying computational backbones, our framework is based on XGBoost and LightGBM. Modelling and predicting the entire conditional distribution greatly enhances existing tree-based gradient boosting implementations, as it allows to create probabilistic forecasts from which prediction intervals and quantiles of interest can be derived. Empirical results show that our framework achieves state-of-the-art forecast accuracy. △ Less

Submitted 2 April, 2022; originally announced April 2022.

Comments: Distributional Regression, LightGBM, Normalizing Flow, Probabilistic Forecasting, XGBoost

Using the Softplus Function to Construct Alternative Link Functions in Generalized Linear Models and Beyond

Authors: Paul F. V. Wiemann, Thomas Kneib, Julien Hambuckers

Abstract: Response functions linking regression predictors to properties of the response distribution are fundamental components in many statistical models. However, the choice of these functions is typically based on the domain of the modeled quantities and is not further scrutinized. For example, the exponential response function is usually assumed for parameters restricted to be positive although it impl… ▽ More Response functions linking regression predictors to properties of the response distribution are fundamental components in many statistical models. However, the choice of these functions is typically based on the domain of the modeled quantities and is not further scrutinized. For example, the exponential response function is usually assumed for parameters restricted to be positive although it implies a multiplicative model which may not necessarily be desired. Consequently, applied researchers might easily face misleading results when relying on defaults without further investigation. As an alternative to the exponential response function, we propose the use of the softplus function to construct alternative link functions for parameters restricted to be positive. As a major advantage, we can construct differentiable link functions corresponding closely to the identity function for positive values of the regression predictor, which implies an quasi-additive model and thus allows for an additive interpretation of the estimated effects by practitioners. We demonstrate the applicability of the softplus response function using both simulations and real data. In four applications featuring count data regression and Bayesian distributional regression, we contrast our approach to the commonly used exponential response function. △ Less

Submitted 28 November, 2021; originally announced November 2021.

A multivariate Gaussian random field prior against spatial confounding

Authors: Isa Marques, Thomas Kneib, Nadja Klein

Abstract: Spatial models are used in a variety research areas, such as environmental sciences, epidemiology, or physics. A common phenomenon in many spatial regression models is spatial confounding. This phenomenon takes place when spatially indexed covariates modeling the mean of the response are correlated with the spatial random effect. As a result, estimates for regression coefficients of the covariates… ▽ More Spatial models are used in a variety research areas, such as environmental sciences, epidemiology, or physics. A common phenomenon in many spatial regression models is spatial confounding. This phenomenon takes place when spatially indexed covariates modeling the mean of the response are correlated with the spatial random effect. As a result, estimates for regression coefficients of the covariates can be severely biased and interpretation of these is no longer valid. Recent literature has shown that typical solutions for reducing spatial confounding can lead to misleading and counterintuitive results. In this paper, we develop a computationally efficient spatial model in a Bayesian framework integrating novel prior structure that reduces spatial confounding. Starting from the univariate case, we extend our prior structure to case of multiple spatially confounded covariates. In a simulation study, we show that our novel model flexibly detects and reduces spatial confounding in spatial datasets, and it performs better than typically used methods such as restricted spatial regression. These results are promising for any applied researcher who wishes to interpret covariate effects in spatial regression models. As a real data illustration, we study the effect of elevation and temperature on the mean of daily precipitation in Germany. △ Less

Submitted 7 June, 2021; originally announced June 2021.

Comments: Submitted to Environmetrics and currently under review

Flexible Bayesian Modeling of Counts: Constructing Penalized Complexity Priors

Authors: Mahsa Nadifar, Hossein Baghishani, Thomas Kneib, Afshin Fallah

Abstract: Many of the data, particularly in medicine and disease map** are count. Indeed, the under or overdispersion problem in count data distrusts the performance of the classical Poisson model. For taking into account this problem, in this paper, we introduce a new Bayesian structured additive regression model, called gamma count, with enough flexibility in modeling dispersion. Setting convenient prio… ▽ More Many of the data, particularly in medicine and disease map** are count. Indeed, the under or overdispersion problem in count data distrusts the performance of the classical Poisson model. For taking into account this problem, in this paper, we introduce a new Bayesian structured additive regression model, called gamma count, with enough flexibility in modeling dispersion. Setting convenient prior distributions on the model parameters is a momentous issue in Bayesian statistics that characterize the nature of our uncertainty parameters. Relying on a recently proposed class of penalized complexity priors, motivated from a general set of construction principles, we derive the prior structure. The model can be formulated as a latent Gaussian model, and consequently, we can carry out the fast computation by using the integrated nested Laplace approximation method. We investigate the proposed methodology simulation study. Different expropriate prior distribution are examined to provide reasonable sensitivity analysis. To explain the applicability of the proposed model, we analyzed two real-world data sets related to the larynx mortality cancer in Germany and the handball champions league. △ Less

Submitted 18 May, 2021; originally announced May 2021.

Adaptive shrinkage of smooth functional effects towards a predefined functional subspace

Authors: Paul Wiemann, Thomas Kneib

Abstract: In this paper, we propose a new horseshoe-type prior hierarchy for adaptively shrinking spline-based functional effects towards a predefined vector space of parametric functions. Instead of shrinking each spline coefficient towards zero, we use an adapted horseshoe prior to control the deviation from the predefined vector space. For this purpose, the modified horseshoe prior is set up with one sca… ▽ More In this paper, we propose a new horseshoe-type prior hierarchy for adaptively shrinking spline-based functional effects towards a predefined vector space of parametric functions. Instead of shrinking each spline coefficient towards zero, we use an adapted horseshoe prior to control the deviation from the predefined vector space. For this purpose, the modified horseshoe prior is set up with one scale parameter per spline and not one per coefficient. The presented prior allows for a large number of basis functions to capture all kinds of functional effects while the estimated functional effect is prevented from a highly oscillating overfit. We achieve this by integrating a smoothing penalty similar to the random walk prior commonly applied in Bayesian P-spline priors. In a simulation study, we demonstrate the properties of the new prior specification and compare it to other approaches from the literature. Furthermore, we showcase the applicability of the proposed method by estimating the energy consumption in Germany over the course of a day. For inference, we rely on Markov chain Monte Carlo simulations combining Gibbs sampling for the spline coefficients with slice sampling for all scale parameters in the model. △ Less

Submitted 14 January, 2021; originally announced January 2021.

Bayesian Conditional Transformation Models

Authors: Manuel Carlan, Thomas Kneib, Nadja Klein

Abstract: Recent developments in statistical regression methodology shift away from pure mean regression towards distributional regression models. One important strand thereof is that of conditional transformation models (CTMs). CTMs infer the entire conditional distribution directly by applying a transformation function to the response conditionally on a set of covariates towards a simple log-concave refer… ▽ More Recent developments in statistical regression methodology shift away from pure mean regression towards distributional regression models. One important strand thereof is that of conditional transformation models (CTMs). CTMs infer the entire conditional distribution directly by applying a transformation function to the response conditionally on a set of covariates towards a simple log-concave reference distribution. Thereby, CTMs allow not only variance, kurtosis or skewness but the complete conditional distribution to depend on the explanatory variables. We propose a Bayesian notion of conditional transformation models (BCTMs) focusing on exactly observed continuous responses, but also incorporating extensions to randomly censored and discrete responses. Rather than relying on Bernstein polynomials that have been considered in likelihood-based CTMs, we implement a spline-based parametrization for monotonic effects that are supplemented with smoothness priors. Furthermore, we are able to benefit from the Bayesian paradigm via easily obtainable credible intervals and other quantities without relying on large sample approximations. A simulation study demonstrates the competitiveness of our approach against its likelihood-based counterpart but also Bayesian additive models of location, scale and shape and Bayesian quantile regression. Two applications illustrate the versatility of BCTMs in problems involving real world data, again including the comparison with various types of competitors. △ Less

Submitted 21 May, 2022; v1 submitted 20 December, 2020; originally announced December 2020.

Beyond unidimensional poverty analysis using distributional copula models for mixed ordered-continuous outcomes

Authors: Maike Hohberg, Francesco Donat, Giampiero Marra, Thomas Kneib

Abstract: Poverty is a multidimensional concept often comprising a monetary outcome and other welfare dimensions such as education, subjective well-being or health, that are measured on an ordinal scale. In applied research, multidimensional poverty is ubiquitously assessed by studying each poverty dimension independently in univariate regression models or by combining several poverty dimensions into a scal… ▽ More Poverty is a multidimensional concept often comprising a monetary outcome and other welfare dimensions such as education, subjective well-being or health, that are measured on an ordinal scale. In applied research, multidimensional poverty is ubiquitously assessed by studying each poverty dimension independently in univariate regression models or by combining several poverty dimensions into a scalar index. This inhibits a thorough analysis of the potentially varying interdependence between the poverty dimensions. We propose a multivariate copula generalized additive model for location, scale and shape (copula GAMLSS or distributional copula model) to tackle this challenge. By relating the copula parameter to covariates, we specifically examine if certain factors determine the dependence between poverty dimensions. Furthermore, specifying the full conditional bivariate distribution, allows us to derive several features such as poverty risks and dependence measures coherently from one model for different individuals. We demonstrate the approach by studying two important poverty dimensions: income and education. Since the level of education is measured on an ordinal scale while income is continuous, we extend the bivariate copula GAMLSS to the case of mixed ordered-continuous outcomes. The new model is integrated into the GJRM package in R and applied to data from Indonesia. Particular emphasis is given to the spatial variation of the income-education dependence and groups of individuals at risk of being simultaneously poor in both education and income dimensions. △ Less

Submitted 8 September, 2020; originally announced September 2020.

Analytic expressions for the Cumulative Distribution Function of the Composed Error Term in Stochastic Frontier Analysis with Truncated Normal and Exponential Inefficiencies

Authors: Rouven Schmidt, Thomas Kneib

Abstract: In the stochastic frontier model, the composed error term consists of the measurement error and the inefficiency term. A general assumption is that the inefficiency term follows a truncated normal or exponential distribution. In a wide variety of models evaluating the cumulative distribution function of the composed error term is required. This work introduces and proves four representation theore… ▽ More In the stochastic frontier model, the composed error term consists of the measurement error and the inefficiency term. A general assumption is that the inefficiency term follows a truncated normal or exponential distribution. In a wide variety of models evaluating the cumulative distribution function of the composed error term is required. This work introduces and proves four representation theorems for these distributions - two for each distributional assumptions. These representations can be utilized for a fast and accurate evaluation. △ Less

Submitted 5 June, 2020; originally announced June 2020.

Comments: 16 pages, 2 figures

Noncrossing structured additive multiple-output Bayesian quantile regression models

Authors: Bruno Santos, Thomas Kneib

Abstract: Quantile regression models are a powerful tool for studying different points of the conditional distribution of univariate response variables. Their multivariate counterpart extension though is not straightforward, starting with the definition of multivariate quantiles. We propose here a flexible Bayesian quantile regression model when the response variable is multivariate, where we are able to de… ▽ More Quantile regression models are a powerful tool for studying different points of the conditional distribution of univariate response variables. Their multivariate counterpart extension though is not straightforward, starting with the definition of multivariate quantiles. We propose here a flexible Bayesian quantile regression model when the response variable is multivariate, where we are able to define a structured additive framework for all predictor variables. We build on previous ideas considering a directional approach to define the quantiles of a response variable with multiple-outputs and we define noncrossing quantiles in every directional quantile model. We define a Markov Chain Monte Carlo (MCMC) procedure for model estimation, where the noncrossing property is obtained considering a Gaussian process design to model the correlation between several quantile regression models. We illustrate the results of these models using two data sets: one on dimensions of inequality in the population, such as income and health; the second on scores of students in the Brazilian High School National Exam, considering three dimensions for the response variable. △ Less

Submitted 18 October, 2019; originally announced October 2019.

Generalised Joint Regression for Count Data with a Focus on Modelling Football Matches

Authors: Hendrik van der Wurp, Andreas Groll, Thomas Kneib, Giampiero Marra, Rosalba Radice

Abstract: We propose a versatile joint regression framework for count responses. The method is implemented in the R add-on package GJRM and allows for modelling linear and non-linear dependence through the use of several copulae. Moreover, the parameters of the marginal distributions of the count responses and of the copula can be specified as flexible functions of covariates. Motivated by a football applic… ▽ More We propose a versatile joint regression framework for count responses. The method is implemented in the R add-on package GJRM and allows for modelling linear and non-linear dependence through the use of several copulae. Moreover, the parameters of the marginal distributions of the count responses and of the copula can be specified as flexible functions of covariates. Motivated by a football application, we also discuss an extension which forces the regression coefficients of the marginal (linear) predictors to be equal via a suitable penalisation. Model fitting is based on a trust region algorithm which estimates simultaneously all the parameters of the joint models. We investigate the proposal's empirical performance in two simulation studies, the first one designed for arbitrary count data, the other one reflecting football-specific settings. Finally, the method is applied to FIFA World Cup data, showing its competitiveness to the standard approach with regard to predictive performance. △ Less

Submitted 21 August, 2019; v1 submitted 2 August, 2019; originally announced August 2019.

Multivariate Conditional Transformation Models

Authors: Nadja Klein, Torsten Hothorn, Luisa Barbanti, Thomas Kneib

Abstract: Regression models describing the joint distribution of multivariate response variables conditional on covariate information have become an important aspect of contemporary regression analysis. However, a limitation of such models is that they often rely on rather simplistic assumptions, e.g. a constant dependency structure that is not allowed to vary with the covariates or the restriction to linea… ▽ More Regression models describing the joint distribution of multivariate response variables conditional on covariate information have become an important aspect of contemporary regression analysis. However, a limitation of such models is that they often rely on rather simplistic assumptions, e.g. a constant dependency structure that is not allowed to vary with the covariates or the restriction to linear dependence between the responses only. We propose a general framework for multivariate conditional transformation models that overcomes these limitations and describes the entire distribution in a tractable and interpretable yet flexible way conditional on nonlinear effects of covariates. The framework can be embedded into likelihood-based inference, including results on asymptotic normality, and allows the dependence structure to vary with covariates. In addition, the framework scales well beyond bivariate response situations, which were the main focus of most earlier investigations. We illustrate the application of multivariate conditional transformation models in a trivariate analysis of childhood undernutrition and demonstrate empirically that our approach can be beneficial compared to existing benchmarks such that complex truly multivariate data-generating processes can be inferred from observations. △ Less

Submitted 3 September, 2020; v1 submitted 7 June, 2019; originally announced June 2019.

Journal ref: Scandinavian Journal of Statistics (2022); 49: 116-142

Bayesian Effect Selection in Structured Additive Distributional Regression Models

Authors: Nadja Klein, Manuel Carlan, Thomas Kneib, Stefan Lang, Helga Wagner

Abstract: We propose a novel spike and slab prior specification with scaled beta prime marginals for the importance parameters of regression coefficients to allow for general effect selection within the class of structured additive distributional regression. This enables us to model effects on all distributional parameters for arbitrary parametric distributions, and to consider various effect types such as… ▽ More We propose a novel spike and slab prior specification with scaled beta prime marginals for the importance parameters of regression coefficients to allow for general effect selection within the class of structured additive distributional regression. This enables us to model effects on all distributional parameters for arbitrary parametric distributions, and to consider various effect types such as non-linear or spatial effects as well as hierarchical regression structures. Our spike and slab prior relies on a parameter expansion that separates blocks of regression coefficients into overall scalar importance parameters and vectors of standardised coefficients. Hence, we can work with a scalar quantity for effect selection instead of a possibly high-dimensional effect vector, which yields improved shrinkage and sampling performance compared to the classical normal-inverse-gamma prior. We investigate the propriety of the posterior, show that the prior yields desirable shrinkage properties, propose a way of eliciting prior parameters and provide efficient Markov Chain Monte Carlo sampling. Using both simulated and three large-scale data sets, we show that our approach is applicable for data with a potentially large number of covariates, multilevel predictors accounting for hierarchically nested data and non-standard response distributions, such as bivariate normal or zero-inflated Poisson. △ Less

Submitted 27 February, 2019; originally announced February 2019.

Journal ref: Bayesian Anal., Advance publication (2020), 29 pages

Treatment effects beyond the mean using GAMLSS

Authors: Maike Hohberg, Peter Pütz, Thomas Kneib

Abstract: This paper introduces distributional regression, also known as generalized additive models for location, scale and shape (GAMLSS), as a modeling framework for analyzing treatment effects beyond the mean. By relating each parameter of the response distribution to explanatory variables, GAMLSS model the treatment effect on the whole conditional distribution. Additionally, any nonnormal outcome and n… ▽ More This paper introduces distributional regression, also known as generalized additive models for location, scale and shape (GAMLSS), as a modeling framework for analyzing treatment effects beyond the mean. By relating each parameter of the response distribution to explanatory variables, GAMLSS model the treatment effect on the whole conditional distribution. Additionally, any nonnormal outcome and nonlinear effects of explanatory variables can be incorporated. We elaborate on the combination of GAMLSS with program evaluation methods in economics and provide practical guidance on the usage of GAMLSS by reanalyzing data from the Mexican \textit{Progresa} program. Contrary to expectations, no significant effects of a cash transfer on the conditional inequality level between treatment and control group are found. △ Less

Submitted 28 March, 2019; v1 submitted 25 June, 2018; originally announced June 2018.

Lost in translation: On the impact of data coding on penalized regression with interactions

Authors: Johannes W R Martini, Francisco Rosales, Ngoc-Thuy Ha, Thomas Kneib, Johannes Heise, Valentin Wimmer

Abstract: Penalized regression approaches are standard tools in quantitative genetics. It is known that the fit of an \emph{ordinary least squares} (OLS) regression is independent of certain transformations of the coding of the predictor variables, and that the standard mixed model \emph{ridge regression best linear unbiased prediction} (RRBLUP) is neither affected by translations of the variable coding, no… ▽ More Penalized regression approaches are standard tools in quantitative genetics. It is known that the fit of an \emph{ordinary least squares} (OLS) regression is independent of certain transformations of the coding of the predictor variables, and that the standard mixed model \emph{ridge regression best linear unbiased prediction} (RRBLUP) is neither affected by translations of the variable coding, nor by global scaling. However, it has been reported that an extended version of this mixed model, which incorporates interactions by products of markers as additional predictor variables is affected by translations of the marker coding. In this work, we identify the cause of this loss of invariance in a general context of penalized regression on polynomials in the predictor variables. We show that in most cases, translating the coding of the predictor variables has an impact on effect estimates, with an exception when only the size of the coefficients of monomials of highest total degree are penalized. The invariance of RRBLUP can thus be considered as a special case of this setting, with a polynomial of total degree 1, where the size of the fixed effect (total degree 0) is not penalized but all coefficients of monomials of total degree 1 are. The extended RRBLUP, which includes interactions, is not invariant to translations because it does not only penalize interactions (total degree 2), but also additive effects (total degree 1). Our observations are not restricted to ridge regression, but generally valid for penalized regressions, for instance also for the $\ell_1$ penalty of LASSO. △ Less

Submitted 10 June, 2018; originally announced June 2018.

MSC Class: 62J05 62J07

Journal ref: G3 (2019) https://doi.org/10.1534/g3.118.200961

Conditional Model Selection in Mixed-Effects Models with cAIC4

Authors: Benjamin Säfken, David Rügamer, Thomas Kneib, Sonja Greven

Abstract: Model selection in mixed models based on the conditional distribution is appropriate for many practical applications and has been a focus of recent statistical research. In this paper we introduce the R-package cAIC4 that allows for the computation of the conditional Akaike Information Criterion (cAIC). Computation of the conditional AIC needs to take into account the uncertainty of the random eff… ▽ More Model selection in mixed models based on the conditional distribution is appropriate for many practical applications and has been a focus of recent statistical research. In this paper we introduce the R-package cAIC4 that allows for the computation of the conditional Akaike Information Criterion (cAIC). Computation of the conditional AIC needs to take into account the uncertainty of the random effects variance and is therefore not straightforward. We introduce a fast and stable implementation for the calculation of the cAIC for linear mixed models estimated with lme4 and additive mixed models estimated with gamm4 . Furthermore, cAIC4 offers a stepwise function that allows for a fully automated stepwise selection scheme for mixed models based on the conditional AIC. Examples of many possible applications are presented to illustrate the practical impact and easy handling of the package. △ Less

Submitted 17 March, 2018; v1 submitted 15 March, 2018; originally announced March 2018.

Bayesian Measurement Error Correction in Structured Additive Distributional Regression with an Application to the Analysis of Sensor Data on Soil-Plant Variability

Authors: Alessio Pollice, Giovanna Jona Lasinio, Roberta Rossi, Mariana Amato, Thomas Kneib, Stefan Lang

Abstract: The flexibility of the Bayesian approach to account for covariates with measurement error is combined with semiparametric regression models for a class of continuous, discrete and mixed univariate response distributions with potentially all parameters depending on a structured additive predictor. Markov chain Monte Carlo enables a modular and numerically efficient implementation of Bayesian measur… ▽ More The flexibility of the Bayesian approach to account for covariates with measurement error is combined with semiparametric regression models for a class of continuous, discrete and mixed univariate response distributions with potentially all parameters depending on a structured additive predictor. Markov chain Monte Carlo enables a modular and numerically efficient implementation of Bayesian measurement error correction based on the imputation of unobserved error-free covariate values. We allow for very general measurement errors, including correlated replicates with heterogeneous variances. The proposal is first assessed by a simulation trial, then it is applied to the assessment of a soil-plant relationship crucial for implementing efficient agricultural management practices. Observations on multi-depth soil information forage ground-cover for a seven hectares Alfalfa stand in South Italy were obtained using sensors with very refined spatial resolution. Estimating a functional relation between ground-cover and soil with these data involves addressing issues linked to the spatial and temporal misalignment and the large data size. We propose a preliminary spatial interpolation on a lattice covering the field and subsequent analysis by a structured additive distributional regression model accounting for measurement error in the soil covariate. Results are interpreted and commented in connection to possible Alfalfa management strategies. △ Less

Submitted 29 November, 2017; originally announced November 2017.

Gradient boosting in Markov-switching generalized additive models for location, scale and shape

Authors: Timo Adam, Andreas Mayr, Thomas Kneib

Abstract: We propose a novel class of flexible latent-state time series regression models which we call Markov-switching generalized additive models for location, scale and shape. In contrast to conventional Markov-switching regression models, the presented methodology allows us to model different state-dependent parameters of the response distribution - not only the mean, but also variance, skewness and ku… ▽ More We propose a novel class of flexible latent-state time series regression models which we call Markov-switching generalized additive models for location, scale and shape. In contrast to conventional Markov-switching regression models, the presented methodology allows us to model different state-dependent parameters of the response distribution - not only the mean, but also variance, skewness and kurtosis parameters - as potentially smooth functions of a given set of explanatory variables. In addition, the set of possible distributions that can be specified for the response is not limited to the exponential family but additionally includes, for instance, a variety of Box-Cox-transformed, zero-inflated and mixture distributions. We propose an estimation approach based on the EM algorithm, where we use the gradient boosting framework to prevent overfitting while simultaneously performing variable selection. The feasibility of the suggested approach is assessed in simulation experiments and illustrated in a real-data setting, where we model the conditional distribution of the daily average price of energy in Spain over time. △ Less

Submitted 17 May, 2018; v1 submitted 6 October, 2017; originally announced October 2017.

Boosting Joint Models for Longitudinal and Time-to-Event Data

Authors: Elisabeth Waldmann, David Taylor-Robinson, Nadja Klein, Thomas Kneib, Tania Pressler, Matthias Schmid, Andreas Mayr

Abstract: Joint Models for longitudinal and time-to-event data have gained a lot of attention in the last few years as they are a helpful technique to approach common a data structure in clinical studies where longitudinal outcomes are recorded alongside event times. Those two processes are often linked and the two outcomes should thus be modeled jointly in order to prevent the potential bias introduced by… ▽ More Joint Models for longitudinal and time-to-event data have gained a lot of attention in the last few years as they are a helpful technique to approach common a data structure in clinical studies where longitudinal outcomes are recorded alongside event times. Those two processes are often linked and the two outcomes should thus be modeled jointly in order to prevent the potential bias introduced by independent modelling. Commonly, joint models are estimated in likelihood based expectation maximization or Bayesian approaches using frameworks where variable selection is problematic and which do not immediately work for high-dimensional data. In this paper, we propose a boosting algorithm tackling these challenges by being able to simultaneously estimate predictors for joint models and automatically select the most influential variables even in high-dimensional data situations. We analyse the performance of the new algorithm in a simulation study and apply it to the Danish cystic fibrosis registry which collects longitudinal lung function data on patients with cystic fibrosis together with data regarding the onset of pulmonary infections. This is the first approach to combine state-of-the art algorithms from the field of machine-learning with the model class of joint models, providing a fully data-driven mechanism to select variables and predictor effects in a unified framework of boosting joint models. △ Less

Submitted 22 December, 2016; v1 submitted 9 September, 2016; originally announced September 2016.

Bayesian structured additive distributional regression with an application to regional income inequality in Germany

Authors: Nadja Klein, Thomas Kneib, Stefan Lang, Alexander Sohn

Abstract: We propose a generic Bayesian framework for inference in distributional regression models in which each parameter of a potentially complex response distribution and not only the mean is related to a structured additive predictor. The latter is composed additively of a variety of different functional effect types such as nonlinear effects, spatial effects, random coefficients, interaction surfaces… ▽ More We propose a generic Bayesian framework for inference in distributional regression models in which each parameter of a potentially complex response distribution and not only the mean is related to a structured additive predictor. The latter is composed additively of a variety of different functional effect types such as nonlinear effects, spatial effects, random coefficients, interaction surfaces or other (possibly nonstandard) basis function representations. To enforce specific properties of the functional effects such as smoothness, informative multivariate Gaussian priors are assigned to the basis function coefficients. Inference can then be based on computationally efficient Markov chain Monte Carlo simulation techniques where a generic procedure makes use of distribution-specific iteratively weighted least squares approximations to the full conditionals. The framework of distributional regression encompasses many special cases relevant for treating nonstandard response structures such as highly skewed nonnegative responses, overdispersed and zero-inflated counts or shares including the possibility for zero- and one-inflation. We discuss distributional regression along a study on determinants of labour incomes for full-time working males in Germany with a particular focus on regional differences after the German reunification. Controlling for age, education, work experience and local disparities, we estimate full conditional income distributions allowing us to study various distributional quantities such as moments, quantiles or inequality measures in a consistent manner in one joint model. Detailed guidance on practical aspects of model choice including the selection of several competing distributions for labour incomes and the consideration of different covariate effects on the income distribution complete the distributional regression analysis. We find that next to a lower expected income, full-time working men in East Germany also face a more unequal income distribution than men in the West, ceteris paribus. △ Less

Submitted 17 September, 2015; originally announced September 2015.

Comments: Published at http://dx.doi.org/10.1214/15-AOAS823 in the Annals of Applied Statistics (http://www.imstat.org/aoas/) by the Institute of Mathematical Statistics (http://www.imstat.org)

Report number: IMS-AOAS-AOAS823

Journal ref: Annals of Applied Statistics 2015, Vol. 9, No. 2, 1024-1052

Markov-switching generalized additive models

Authors: Roland Langrock, Thomas Kneib, Richard Glennie, Théo Michelot

Abstract: We consider Markov-switching regression models, i.e. models for time series regression analyses where the functional relationship between covariates and response is subject to regime switching controlled by an unobservable Markov chain. Building on the powerful hidden Markov model machinery and the methods for penalized B-splines routinely used in regression analyses, we develop a framework for no… ▽ More We consider Markov-switching regression models, i.e. models for time series regression analyses where the functional relationship between covariates and response is subject to regime switching controlled by an unobservable Markov chain. Building on the powerful hidden Markov model machinery and the methods for penalized B-splines routinely used in regression analyses, we develop a framework for nonparametrically estimating the functional form of the effect of the covariates in such a regression model, assuming an additive structure of the predictor. The resulting class of Markov-switching generalized additive models is immensely flexible, and contains as special cases the common parametric Markov-switching regression models and also generalized additive and generalized linear models. The feasibility of the suggested maximum penalized likelihood approach is demonstrated by simulation and further illustrated by modelling how energy price in Spain depends on the Euro/Dollar exchange rate. △ Less

Submitted 10 May, 2015; v1 submitted 14 June, 2014; originally announced June 2014.

A Unified Framework of Constrained Regression

Authors: Benjamin Hofner, Thomas Kneib, Torsten Hothorn

Abstract: Generalized additive models (GAMs) play an important role in modeling and understanding complex relationships in modern applied statistics. They allow for flexible, data-driven estimation of covariate effects. Yet researchers often have a priori knowledge of certain effects, which might be monotonic or periodic (cyclic) or should fulfill boundary conditions. We propose a unified framework to incor… ▽ More Generalized additive models (GAMs) play an important role in modeling and understanding complex relationships in modern applied statistics. They allow for flexible, data-driven estimation of covariate effects. Yet researchers often have a priori knowledge of certain effects, which might be monotonic or periodic (cyclic) or should fulfill boundary conditions. We propose a unified framework to incorporate these constraints for both univariate and bivariate effect estimates and for varying coefficients. As the framework is based on component-wise boosting methods, variables can be selected intrinsically, and effects can be estimated for a wide range of different distributional assumptions. Bootstrap confidence intervals for the effect estimates are derived to assess the models. We present three case studies from environmental sciences to illustrate the proposed seamless modeling framework. All discussed constrained effect estimates are implemented in the comprehensive R package mboost for model-based boosting. △ Less

Submitted 7 November, 2014; v1 submitted 27 March, 2014; originally announced March 2014.

Comments: This is a preliminary version of the manuscript. The final publication is available at http://link.springer.com/article/10.1007/s11222-014-9520-y

Bayesian Geoadditive Expectile Regression

Authors: Elisabeth Waldmann, Fabian Sobotka, Thomas Kneib

Abstract: Regression classes modeling more than the mean of the response have found a lot of attention in the last years. Expectile regression is a special and computationally convenient case of this family of models. Expectiles offer a quantile-like characterisation of a complete distribution and include the mean as a special case. In the frequentist framework the impact of a lot of covariates with very di… ▽ More Regression classes modeling more than the mean of the response have found a lot of attention in the last years. Expectile regression is a special and computationally convenient case of this family of models. Expectiles offer a quantile-like characterisation of a complete distribution and include the mean as a special case. In the frequentist framework the impact of a lot of covariates with very different structures have been made possible. We propose Bayesian expectile regression based on the asymmetric normal distribution. This renders possible incorporating for example linear, nonlinear, spatial and random effects in one model. Furthermore a detailed inference on the estimated parameters can be conducted. Proposal densities based on iterativly weighted least squares updates for the resulting Markov chain Monte Carlo (MCMC) simulation algorithm are proposed and the potential of the approach for extending the flexibility of expectile regression towards complex semiparametric regression specifications is discussed. △ Less

Submitted 18 December, 2013; originally announced December 2013.

Maximum penalized likelihood estimation in semiparametric capture-recapture models

Authors: Théo Michelot, Roland Langrock, Thomas Kneib, Ruth King

Abstract: We discuss the semiparametric modeling of mark-recapture-recovery data where the temporal and/or individual variation of model parameters is explained via covariates. Typically, in such analyses a fixed (or mixed) effects parametric model is specified for the relationship between the model parameters and the covariates of interest. In this paper, we discuss the modeling of the relationship via the… ▽ More We discuss the semiparametric modeling of mark-recapture-recovery data where the temporal and/or individual variation of model parameters is explained via covariates. Typically, in such analyses a fixed (or mixed) effects parametric model is specified for the relationship between the model parameters and the covariates of interest. In this paper, we discuss the modeling of the relationship via the use of penalized splines, to allow for considerably more flexible functional forms. Corresponding models can be fitted via numerical maximum penalized likelihood estimation, employing cross-validation to choose the smoothing parameters in a data-driven way. Our contribution builds on and extends the existing literature, providing a unified inferential framework for semiparametric mark-recapture-recovery models for open populations, where the interest typically lies in the estimation of survival probabilities. The approach is applied to two real datasets, corresponding to grey herons (Ardea Cinerea), where we model the survival probability as a function of environmental condition (a time-varying global covariate), and Soay sheep (Ovis Aries), where we model the survival probability as a function of individual weight (a time-varying individual-specific covariate). The proposed semiparametric approach is compared to a standard parametric (logistic) regression and new interesting underlying dynamics are observed in both cases. △ Less

Submitted 20 May, 2015; v1 submitted 5 November, 2013; originally announced November 2013.

Nonparametric inference in hidden Markov models using P-splines

Authors: Roland Langrock, Thomas Kneib, Alexander Sohn, Stacy DeRuiter

Abstract: Hidden Markov models (HMMs) are flexible time series models in which the distributions of the observations depend on unobserved serially correlated states. The state-dependent distributions in HMMs are usually taken from some class of parametrically specified distributions. The choice of this class can be difficult, and an unfortunate choice can have serious consequences for example on state estim… ▽ More Hidden Markov models (HMMs) are flexible time series models in which the distributions of the observations depend on unobserved serially correlated states. The state-dependent distributions in HMMs are usually taken from some class of parametrically specified distributions. The choice of this class can be difficult, and an unfortunate choice can have serious consequences for example on state estimates, on forecasts and generally on the resulting model complexity and interpretation, in particular with respect to the number of states. We develop a novel approach for estimating the state-dependent distributions of an HMM in a nonparametric way, which is based on the idea of representing the corresponding densities as linear combinations of a large number of standardized B-spline basis functions, imposing a penalty term on non-smoothness in order to maintain a good balance between goodness-of-fit and smoothness. We illustrate the nonparametric modeling approach in a real data application concerned with vertical speeds of a diving beaked whale, demonstrating that compared to parametric counterparts it can lead to models that are more parsimonious in terms of the number of states yet fit the data equally well. △ Less

Submitted 17 June, 2014; v1 submitted 2 September, 2013; originally announced September 2013.

Semiparametric stochastic volatility modelling using penalized splines

Authors: Roland Langrock, Théo Michelot, Alexander Sohn, Thomas Kneib

Abstract: Stochastic volatility (SV) models mimic many of the stylized facts attributed to time series of asset returns, while maintaining conceptual simplicity. The commonly made assumption of conditionally normally distributed or Student-t-distributed returns, given the volatility, has however been questioned. In this manuscript, we introduce a novel maximum penalized likelihood approach for estimating th… ▽ More Stochastic volatility (SV) models mimic many of the stylized facts attributed to time series of asset returns, while maintaining conceptual simplicity. The commonly made assumption of conditionally normally distributed or Student-t-distributed returns, given the volatility, has however been questioned. In this manuscript, we introduce a novel maximum penalized likelihood approach for estimating the conditional distribution in an SV model in a nonparametric way, thus avoiding any potentially critical assumptions on the shape. The considered framework exploits the strengths both of the powerful hidden Markov model machinery and of penalized B-splines, and constitutes a powerful and flexible alternative to recently developed Bayesian approaches to semiparametric SV modelling. We demonstrate the feasibility of the approach in a simulation study before outlining its potential in applications to three series of returns on stocks and one series of stock index returns. △ Less

Submitted 17 June, 2014; v1 submitted 27 August, 2013; originally announced August 2013.

Penalized Likelihood and Bayesian Function Selection in Regression Models

Authors: Fabian Scheipl, Thomas Kneib, Ludwig Fahrmeir

Abstract: Challenging research in various fields has driven a wide range of methodological advances in variable selection for regression models with high-dimensional predictors. In comparison, selection of nonlinear functions in models with additive predictors has been considered only more recently. Several competing suggestions have been developed at about the same time and often do not refer to each other… ▽ More Challenging research in various fields has driven a wide range of methodological advances in variable selection for regression models with high-dimensional predictors. In comparison, selection of nonlinear functions in models with additive predictors has been considered only more recently. Several competing suggestions have been developed at about the same time and often do not refer to each other. This article provides a state-of-the-art review on function selection, focusing on penalized likelihood and Bayesian concepts, relating various approaches to each other in a unified framework. In an empirical comparison, also including boosting, we evaluate several methods through applications to simulated and real data, thereby providing some guidance on their performance in practice. △ Less

Submitted 4 March, 2013; originally announced March 2013.

Conditional Transformation Models

Authors: Torsten Hothorn, Thomas Kneib, Peter Bühlmann

Abstract: The ultimate goal of regression analysis is to obtain information about the conditional distribution of a response given a set of explanatory variables. This goal is, however, seldom achieved because most established regression models only estimate the conditional mean as a function of the explanatory variables and assume that higher moments are not affected by the regressors. The underlying reaso… ▽ More The ultimate goal of regression analysis is to obtain information about the conditional distribution of a response given a set of explanatory variables. This goal is, however, seldom achieved because most established regression models only estimate the conditional mean as a function of the explanatory variables and assume that higher moments are not affected by the regressors. The underlying reason for such a restriction is the assumption of additivity of signal and noise. We propose to relax this common assumption in the framework of transformation models. The novel class of semiparametric regression models proposed herein allows transformation functions to depend on explanatory variables. These transformation functions are estimated by regularised optimisation of scoring rules for probabilistic forecasts, e.g. the continuous ranked probability score. The corresponding estimated conditional distribution functions are consistent. Conditional transformation models are potentially useful for describing possible heteroscedasticity, comparing spatially varying distributions, identifying extreme events, deriving prediction intervals and selecting variables beyond mean regression effects. An empirical investigation based on a heteroscedastic varying coefficient simulation model demonstrates that semiparametric estimation of conditional distribution functions can be more beneficial than kernel-based non-parametric approaches or parametric generalised additive models for location, scale and shape. △ Less

Submitted 28 November, 2012; v1 submitted 27 January, 2012; originally announced January 2012.

MSC Class: 62H12; 62G08; 62J02; 62J07

Journal ref: Journal of the Royal Statistical Society, Series B (Methodology), 2014

Spike-and-Slab Priors for Function Selection in Structured Additive Regression Models

Authors: Fabian Scheipl, Ludwig Fahrmeir, Thomas Kneib

Abstract: Structured additive regression provides a general framework for complex Gaussian and non-Gaussian regression models, with predictors comprising arbitrary combinations of nonlinear functions and surfaces, spatial effects, varying coefficients, random effects and further regression terms. The large flexibility of structured additive regression makes function selection a challenging and important tas… ▽ More Structured additive regression provides a general framework for complex Gaussian and non-Gaussian regression models, with predictors comprising arbitrary combinations of nonlinear functions and surfaces, spatial effects, varying coefficients, random effects and further regression terms. The large flexibility of structured additive regression makes function selection a challenging and important task, aiming at (1) selecting the relevant covariates, (2) choosing an appropriate and parsimonious representation of the impact of covariates on the predictor and (3) determining the required interactions. We propose a spike-and-slab prior structure for function selection that allows to include or exclude single coefficients as well as blocks of coefficients representing specific model terms. A novel multiplicative parameter expansion is required to obtain good mixing and convergence properties in a Markov chain Monte Carlo simulation approach and is shown to induce desirable shrinkage properties. In simulation studies and with (real) benchmark classification data, we investigate sensitivity to hyperparameter settings and compare performance to competitors. The flexibility and applicability of our approach are demonstrated in an additive piecewise exponential model with time-varying effects for right-censored survival times of intensive care patients with sepsis. Geoadditive and additive mixed logit model applications are discussed in an extensive appendix. △ Less

Submitted 2 December, 2011; v1 submitted 26 May, 2011; originally announced May 2011.

Journal ref: Journal of the American Statistical Association (2012), 107:500, pages 1518--1532