Modifying Two-Parameter Ridge Liu Estimator Based on Ridge Estimation

In this paper, we introduce the new biased estimator to deal with the problem of multicollinearity. This estimator is considered a modification of Two-Parameter Ridge-Liu estimator based on ridge estimation. Furthermore, the superiority of the new estimator than Ridge, Liu and Two-Parameter Ridge-Liu estimator were discussed. We used the mean squared error matrix (MSEM) criterion to verify the superiority of the new estimate. In addition to, we illustrated the performance of the new estimator at several factors through the simulation study.


Introduction
The problem of multicollinearity is one of the problems that have preoccupied the statisticians for a long time.Many studies have been interested in how to overcome this problem in linear regression models, and was based primarily on ability to overcome the ill condition that appears in mean squared error method.In literature, a set of biased estimators has been proposed to overcome this problem.Horal and Kennard (1970) suggested a ridge estimator which depends on a small constant value known as ridge parameter which adding to the diagonal values of the matrix (XˊX) to overcome the ill condition.In the same context, Liu (1993) introduce Liu estimator which it is a combination ridge and stein estimator which proposed by Stein (1956).Actually, the value of the ridge parameter may not be large enough to overcome the multicollinearity problem.Therefore, Liu (2003) suggested Liu-type estimator which has two parameters, so that the increase in one parameter can be limited by the other.There is a series of studies had been directed at improving Liu and Liu-type estimators.Yalian and Yang (2012) modified Liu estimator with prior information for the vector of parameters.Ozkale and Kaciranlar (2007) introduced two-parameter Ridge-Liu estimator that is superior to the Liu-type estimator through the mean square error matrix criteria.Sadullah and Selahattin, (2008), Yang H. and Chang, X. (2010) suggested a new biased estimator that makes Liu estimator based on ridge estimation.Jibo, (2014) proposed unbiased two parameter estimator based on prior information.In this paper, we introduce a new biased estimator that make two parameter Ridge-Liu estimator based on ridge estimation and we show that the new biased estimator is superiority to ridge, Liu and two-parameter Ridge-Liu estimator and we use the simulation study to explain the theoretical results.

Background:
Consider the linear regression model  =  + (1) Where  represents an  × 1 observation of response vector,  represents an known  ×  design matrix of rank p,  represents an p × 1 vector of unknown parameters and ε is n × 1 of random error with () = 0 n×1 vector and (ˊ) =  =  2   is n×n variance covariance matrix for errors.The ordinary least squares estimator (OLS) of model ( 1) is given by This estimator is the best unbiased estimator.However, the existence of the problem of multicollinearity makes this estimator have large least squares error.To overcome the multicollinearity problem, Hoerl and Kennard (1970) introduced the ridge estimator (RE) that has a lower mean squares error than the (OLS) estimator and it is given by  ̂ (k) = (ˊ + I) −1 ˊ (3) Where  ≥ 0 is ridge biasing parameter.Liu (1993) introduced the biased estimator which is known as Liu estimator (LE) and that has been obtained by combining the stein estimator which is introduced by Stein (1956) and the RE and it is defined by  ̂ () = (ˊ + I) −1 (ˊ +  ̂ ) (4) Where 0 <  < 1 is Liu biasing parameter.This estimator get by augmenting the equation d ̂ = β + ε to the model in (1) and then using the ordinary least squares method.Liu (2003) introduced Liu-type estimator that improve the Liu estimator, since it has two parameters, by augmenting the equation (−d/k 1/2 ) ̂ = β + ε to the model in (1) and then using the ordinary least squares method and is given by by augmenting the equation (−d/ 1/2 ) ̂ () = k 1/2 β + ε to the model in (1) and then using the ordinary least squares method .This estimator has superior to ridge and Liuestimator.Yang H. and Chang, X.(2010) proposed another form of the new Liu biased estimator which defined as  ̂ (, ) = (ˊ + I) −1 (ˊ + )(ˊ + I) −1 ˊ (7) Ozkale and Kaciranlar (2007) introduced two parameter ridge-Liu estimator.This estimator is augmenting the equation (dk 1/2 )β ̂OLS = kβ + ε to the model in (1) and then using the ordinary least squares method and is given by

Superiority for the new biased Two-Parameter Ridge-Liu Estimator:
In this section, we use the mean squared error matrix (MSEM) criteria to illustrate the superiority of the new bias estimators to other estimators.
As follows, we illustrate the superiority of the new bias estimator to the  ̂ (k) ,  ̂ (k, d) and  ̂ (k, d) estimators.The following lemma can be help.

Choice for d and k :
For chose the optimal shrinking parameter (d), we differentiating the trace mean squared error matrix TMSEM( ̂ (k, d)) with respect to d and equating the result to zero and then we can get the optimal estimators for shrinking parameter (d) as the following: We chose the k parameter which minimize the Generalized Cross Validation (GCV): Where (ℎ()) is trace for hat matrix ℎ() = ˊ(Λ + ).

The simulation study:
This section conducts a simulation study to compare the performance of the two-parameter ridge-Liu estimator (α ̂TRLE (k, d)) with other estimators.To generate the explanatory variable with deferent degrees of collinearity, we follow (Liu, 2003) who use the following equation x  = (1 −  2 ) 1/2   +   , i = 1, 2, . . ., n, j = 1, 2, . . ., p − 1 Where  ij and  ip are the independent standard normal pseudo-random numbers and it they are generated independently from N(0,5) and γ is specified so γ 2 is the correlation between any two explanatory variables.We use the three sets of correlations γ = 0.65 , 0.80,0.95 to show the effect of the week and strong correlation between the explanatory variables.The observations on dependent variable are generate by the following equation y i = β 1 x i1 + β 2 x i2 + … + β p x ip + e i , e i ~N(0, σ 2 I n ) , i = 1,2, … ., , j = 1, 2, . . ., p We use sample size n=150,50 and we select σ 2 = 0.01,0.25.The value of d and k are calculate by the equations (24), (25).The parameters β 0i were set to be (1,2, … ,5)  (1,2, … ,10).We repeated the simulation 2000 times and we use the standard mean squares error MSE to illustrate the superior for the new estimator which is defined by Where β ̂i is the estimator in ith replication and β is the true parameter values.The result of the simulation was summarized at table (1-4).We chose the number of independent variable p , the degree of correlation  , the number of observation n and the variance of the disturbance term   .The result of the simulation study showed the OLS estimator had a worst for other estimator in all case.They illustrate that the RE, LRE, TLE, TRLE estimators work will at the several degrees of multicollinearity.The new estimator performs well especially when the degrees of multicollinearity is decreases and also it is not affected by the multicollinearity like the other estimator.Moreover, when n increases and at the same time  2 decreases, the MSE value for our new estimator is decreases.It is clear that, increase in the number of observation n and decreases in the number of independent variable p at the several degrees of multicollinearity had a good effect on the work of all estimators especially on the new estimator.