-
Penalized regression via the restricted bridge estimator
Authors:
Bahadır Yüzbaşı,
Mohammad Arashi,
Fikri Akdeniz
Abstract:
This article is concerned with the Bridge Regression, which is a special family in penalized regression with penalty function $\sum_{j=1}^{p}|β_j|^q$ with $q>0$, in a linear model with linear restrictions. The proposed restricted bridge (RBRIDGE) estimator simultaneously estimates parameters and selects important variables when a prior information about parameters are available in either low dimen…
▽ More
This article is concerned with the Bridge Regression, which is a special family in penalized regression with penalty function $\sum_{j=1}^{p}|β_j|^q$ with $q>0$, in a linear model with linear restrictions. The proposed restricted bridge (RBRIDGE) estimator simultaneously estimates parameters and selects important variables when a prior information about parameters are available in either low dimensional or high dimensional case. Using local quadratic approximation, the penalty term can be approximated around a local initial values vector and the RBRIDGE estimator enjoys a closed-form expression which can be solved when $q>0$. Special cases of our proposal are the restricted LASSO ($q=1$), restricted RIDGE ($q=2$), and restricted Elastic Net ($1< q < 2$) estimators. We provide some theoretical properties of the RBRIDGE estimator under for the low dimensional case, whereas the computational aspects are given for both low and high dimensional cases. An extensive Monte Carlo simulation study is conducted based on different prior pieces of information and the performance of the RBRIDGE estiamtor is compared with some competitive penalty estimators as well as the ORACLE. We also consider four real data examples analysis for comparison sake. The numerical results show that the suggested RBRIDGE estimator outperforms outstandingly when the prior is true or near exact
△ Less
Submitted 8 October, 2019;
originally announced October 2019.
-
LLASSO: A linear unified LASSO for multicollinear situations
Authors:
M. Arashi,
Y. Asar,
B. Yuzbasi
Abstract:
We propose a rescaled LASSO, by premultipying the LASSO with a matrix term, namely linear unified LASSO (LLASSO) for multicollinear situations. Our numerical study has shown that the LLASSO is comparable with other sparse modeling techniques and often outperforms the LASSO and elastic net. Our findings open new visions about using the LASSO still for sparse modeling and variable selection. We conc…
▽ More
We propose a rescaled LASSO, by premultipying the LASSO with a matrix term, namely linear unified LASSO (LLASSO) for multicollinear situations. Our numerical study has shown that the LLASSO is comparable with other sparse modeling techniques and often outperforms the LASSO and elastic net. Our findings open new visions about using the LASSO still for sparse modeling and variable selection. We conclude our study by pointing that the LLASSO can be solved by the same efficient algorithm for solving the LASSO and suggest to follow the same construction technique for other penalized estimators.
△ Less
Submitted 12 October, 2017;
originally announced October 2017.
-
Improved Quantile Regression Estimators when the Errors are Independently and Non-identically Distributed
Authors:
Bahadır Yüzbaşı,
Yasin Asar,
Ahmet Demiralp,
M. Şamil Şık
Abstract:
In a classical regression model, it is usually assumed that the explanatory variables are independent of each other and error terms are normally distributed. But when these assumptions are not met, situations like the error terms are not independent or they are not identically distributed or both of these, LSE will not be robust. Hence, quantile regression has been used to complement this deficien…
▽ More
In a classical regression model, it is usually assumed that the explanatory variables are independent of each other and error terms are normally distributed. But when these assumptions are not met, situations like the error terms are not independent or they are not identically distributed or both of these, LSE will not be robust. Hence, quantile regression has been used to complement this deficiency of classical regression analysis and to improve the least square estimation (LSE). In this study, we consider preliminary test and shrinkage estimation strategies for quantile regression models with independently and non-identically distributed (i.ni.d.) errors. A Monte Carlo simulation study is conducted to assess the relative performance of the estimators. Also, we numerically compare their performance with Ridge, Lasso, Elastic Net penalty estimation strategies. A real data example is presented to illustrate the usefulness of the suggested methods. Finally, we obtain the asymptotic results of suggested estimators
△ Less
Submitted 6 September, 2017;
originally announced September 2017.
-
Liu-type Shrinkage Estimations in Linear Models
Authors:
Bahadır Yüzbaşı,
Yasin Asar,
S. Ejaz Ahmed
Abstract:
In this study, we present the preliminary test, Stein-type and positive part Liu estimators in the linear models when the parameter vector $\boldsymbolβ$ is partitioned into two parts, namely, the main effects $\boldsymbolβ_1$ and the nuisance effects $\boldsymbolβ_2$ such that $\boldsymbolβ=\left(\boldsymbolβ_1, \boldsymbolβ_2 \right)$. We consider the case that a priori known or suspected set of…
▽ More
In this study, we present the preliminary test, Stein-type and positive part Liu estimators in the linear models when the parameter vector $\boldsymbolβ$ is partitioned into two parts, namely, the main effects $\boldsymbolβ_1$ and the nuisance effects $\boldsymbolβ_2$ such that $\boldsymbolβ=\left(\boldsymbolβ_1, \boldsymbolβ_2 \right)$. We consider the case that a priori known or suspected set of the explanatory variables do not contribute to predict the response so that a sub-model may be enough for this purpose. Thus, the main interest is to estimate $\boldsymbolβ_1$ when $\boldsymbolβ_2$ is close to zero. Therefore, we conduct a Monte Carlo simulation study to evaluate the relative efficiency of the suggested estimators, where we demonstrate the superiority of the proposed estimators.
△ Less
Submitted 4 September, 2017;
originally announced September 2017.
-
Preliminary testing derivatives of a linear unified estimator in the logistic regression model
Authors:
Yasin Asar,
Bahadır Yüzbaşı,
Mohammad Arashi,
Jibo Wu
Abstract:
Recently, the well known Liu estimator (Liu, 1993) is attracted researcher's attention in regression parameter estimation for an ill conditioned linear model. It is also argued that imposing sub-space hypothesis restriction on parameters improves estimation by shrinking toward non-sample information. Chang (2015) proposed the almost unbiased Liu estimator (AULE) in the binary logistic regression.…
▽ More
Recently, the well known Liu estimator (Liu, 1993) is attracted researcher's attention in regression parameter estimation for an ill conditioned linear model. It is also argued that imposing sub-space hypothesis restriction on parameters improves estimation by shrinking toward non-sample information. Chang (2015) proposed the almost unbiased Liu estimator (AULE) in the binary logistic regression. In this article, some improved unbiased Liu type estimators, namely, restricted AULE, preliminary test AULE, Stein-type shrinkage AULE and its positive part for estimating the regression parameters in the binary logistic regression model are proposed based on the work Chang (2015). The performances of the newly defined estimators are analysed through some numerical results. A real data example is also provided to support the findings.
△ Less
Submitted 29 August, 2017;
originally announced August 2017.
-
Pretest and Stein-Type Estimations in Quantile Regression Model
Authors:
Bahadır Yüzbaşı,
Yasin Asar,
M. Şamil Şık,
Ahmet Demiralp
Abstract:
In this study, we consider preliminary test and shrinkage estimation strategies for quantile regression models. In classical Least Squares Estimation (LSE) method, the relationship between the explanatory and explained variables in the coordinate plane is estimated with a mean regression line. In order to use LSE, there are three main assumptions on the error terms showing white noise process of t…
▽ More
In this study, we consider preliminary test and shrinkage estimation strategies for quantile regression models. In classical Least Squares Estimation (LSE) method, the relationship between the explanatory and explained variables in the coordinate plane is estimated with a mean regression line. In order to use LSE, there are three main assumptions on the error terms showing white noise process of the regression model, also known as Gauss-Markov Assumptions, must be met: (1) The error terms have zero mean, (2) The variance of the error terms is constant and (3) The covariance between the errors is zero i.e., there is no autocorrelation. However, data in many areas, including econometrics, survival analysis and ecology, etc. does not provide these assumptions. First introduced by Koenker, quantile regression has been used to complement this deficiency of classical regression analysis and to improve the least square estimation. The aim of this study is to improve the performance of quantile regression estimators by using pre-test and shrinkage strategies. A Monte Carlo simulation study including a comparison with quantile $L_1$--type estimators such as Lasso, Ridge and Elastic Net are designed to evaluate the performances of the estimators. Two real data examples are given for illustrative purposes. Finally, we obtain the asymptotic results of suggested estimators
△ Less
Submitted 6 September, 2017; v1 submitted 12 July, 2017;
originally announced July 2017.
-
Shrinkage Estimation Strategies in Generalized Ridge Regression Models Under Low/High-Dimension Regime
Authors:
Bahadır Yüzbaşı,
Mohammad Arashi,
S. Ejaz Ahmed
Abstract:
In this study, we propose shrinkage methods based on {\it generalized ridge regression} (GRR) estimation which is suitable for both multicollinearity and high dimensional problems with small number of samples (large $p$, small $n$). Also, it is obtained theoretical properties of the proposed estimators for Low/High Dimensional cases. Furthermore, the performance of the listed estimators is demonst…
▽ More
In this study, we propose shrinkage methods based on {\it generalized ridge regression} (GRR) estimation which is suitable for both multicollinearity and high dimensional problems with small number of samples (large $p$, small $n$). Also, it is obtained theoretical properties of the proposed estimators for Low/High Dimensional cases. Furthermore, the performance of the listed estimators is demonstrated by both simulation studies and real-data analysis, and compare its performance with existing penalty methods. We show that the proposed methods compare well to competing regularization techniques.
△ Less
Submitted 7 July, 2017;
originally announced July 2017.
-
Improving Estimations in Quantile Regression Model with Autoregressive Errors
Authors:
Bahadır Yüzbaşı,
Yasin Aşar,
Şamil Şık,
Ahmet Demiralp
Abstract:
An important issue is that the respiratory mortality may be a result of air pollution which can be measured by the following variables: temperature, relative humidity, carbon monoxide, sulfur dioxide, nitrogen dioxide, hydrocarbons, ozone and particulates. The usual way is to fit a model using the ordinary least squares regression, which has some assumptions, also known as Gauss-Markov assumptions…
▽ More
An important issue is that the respiratory mortality may be a result of air pollution which can be measured by the following variables: temperature, relative humidity, carbon monoxide, sulfur dioxide, nitrogen dioxide, hydrocarbons, ozone and particulates. The usual way is to fit a model using the ordinary least squares regression, which has some assumptions, also known as Gauss-Markov assumptions, on the error term showing white noise process of the regression model. However, in many applications, especially for this example, these assumptions are not satisfied. Therefore, in this study, a quantile regression approach is used to model the respiratory mortality using the mentioned explanatory variables. Moreover, improved estimation techniques such as preliminary testing and shrinkage strategies are also obtained when the errors are autoregressive. A Monte Carlo simulation experiment, including the quantile penalty estimators such as Lasso, Ridge and Elastic Net, is designed to evaluate the performances of the proposed techniques. Finally, the theoretical risks of the listed estimators are given.
△ Less
Submitted 4 July, 2017;
originally announced July 2017.
-
Big Data Analysis Using Shrinkage Strategies
Authors:
B. Yuzbasi,
M. Arashi,
S. E. Ahmed
Abstract:
In this paper, we apply shrinkage strategies to estimate regression coefficients efficiently for the high-dimensional multiple regression model, where the number of samples is smaller than the number of predictors. We assume in the sparse linear model some of the predictors have very weak influence on the response of interest. We propose to shrink estimators more than usual. Specifically, we use i…
▽ More
In this paper, we apply shrinkage strategies to estimate regression coefficients efficiently for the high-dimensional multiple regression model, where the number of samples is smaller than the number of predictors. We assume in the sparse linear model some of the predictors have very weak influence on the response of interest. We propose to shrink estimators more than usual. Specifically, we use integrated estimation strategies in sub and full models and shrink the integrated estimators by incorporating a bounded measurable function of some weights. The exhibited double shrunken estimators improve the prediction performance of sub models significantly selected from existing Lasso-type variable selection methods. Monte Carlo simulation studies as well as real examples of eye data and Riboavin data confirm the superior performance of the estimators in the high-dimensional regression model.
△ Less
Submitted 17 April, 2017;
originally announced April 2017.
-
Double shrunken selection operator
Authors:
B. Yuzbasi,
M. Arashi
Abstract:
The least absolute shrinkage and selection operator (LASSO) of Tibshirani (1996) is a prominent estimator which selects significant (under some sense) features and kills insignificant ones. Indeed the LASSO shrinks features lager than a noise level to zero. In this paper, we force LASSO to be shrunken more by proposing a Stein-type shrinkage estimator emanating from the LASSO, namely the Stein-typ…
▽ More
The least absolute shrinkage and selection operator (LASSO) of Tibshirani (1996) is a prominent estimator which selects significant (under some sense) features and kills insignificant ones. Indeed the LASSO shrinks features lager than a noise level to zero. In this paper, we force LASSO to be shrunken more by proposing a Stein-type shrinkage estimator emanating from the LASSO, namely the Stein-type LASSO. The newly proposed estimator proposes good performance in risk sense numerically. Variants of this estimator have smaller relative MSE and prediction error, compared to the LASSO, in the analysis of prostate cancer data set.
△ Less
Submitted 19 December, 2016;
originally announced December 2016.
