Main Article Content

Abstract

The ridge regression model has been consistently demonstrated to be an attractive shrinkage method to reduce the effects of multicollinearity. The logistic regression model is a well-known model in application when the response variable is binary data. However, it is known that multicollinearity negatively affects the variance of maximum likelihood estimator of the logistic regression coefficients. To address this problem, a logistic ridge estimator has been proposed by numerous researchers. In this paper, a new Jackknifing logistic ridge estimator (NLRE) is proposed and derived. The idea behind the NLRE is to get diagonal matrix with small values of diagonal elements that leading to decrease the shrinkage parameter and, therefore, the resultant estimator can be better with small amount of bias. Our Monte Carlo simulation results suggest that the NLRE estimator can bring significant improvement relative to other existing estimators. In addition, the real application results demonstrate that the NLRE estimator outperforms both logistic ridge estimator and maximum likelihood estimator in terms of predictive performance.

Keywords

Multicollinearity ridge estimator logistic regression model Jackknife estimator Monte Carlo simulation

Article Details

How to Cite
Algamal, Z. Y., & Hammood, N. (2022). A new Jackknifing ridge estimator for logistic regression model. Pakistan Journal of Statistics and Operation Research, 18(4), 955-961. https://doi.org/10.18187/pjsor.v18i4.3748

References

  1. Algamal, Z. Y., & Lee, M. H. (2015). Penalized Poisson Regression Model using adaptive modified Elastic Net Penalty. Electronic Journal of Applied Statistical Analysis, 8(2), 236-245.
  2. Arashi, M., Kibria, B. M. G., & Valizade, T. (2017). On ridge parameter estimators under stochastic subspace hypothesis. Journal of Statistical Computation and Simulation, 87(5), 966–983. DOI: https://doi.org/10.1080/00949655.2016.1239104
  3. Asar, Y., & Genç, A. (2015). New shrinkage parameters for the Liu-type logistic estimators. Communications in Statistics - Simulation and Computation, 45(3), 1094-1103. doi:10.1080/03610918.2014.995815. DOI: https://doi.org/10.1080/03610918.2014.995815
  4. Batah, F. S. M., Ramanathan, T. V., & Gore, S. D. (2008). The efficiency of modefied jackknife and ridge type regression estimators - A comparison. Surveys in Mathematics and its Applications, 3, 111 – 122.
  5. Firinguettia, L., Kibria, B. M. G., & Araya, R. (2017). Study of Partial Least Squares and Ridge Regression Methods. Communications in Statistics - Simulation and Computation, 46(8), 6631-6644. DOI: https://doi.org/10.1080/03610918.2016.1210168
  6. Hoerl, A. E., & Kennard, R. W. (1970). Ridge regression: Biased estimation for nonorthogonal problems. Technometrics, 12(1), 55-67. DOI: https://doi.org/10.1080/00401706.1970.10488634
  7. Kang, I. J., Wang, L. W., Hsu, S. J., Lee, C. C., Lee, Y. C., Wu, Y. S., . . . Chern, J. H. (2009). Design and efficient synthesis of novel arylthiourea derivatives as potent hepatitis C virus inhibitors. Bioorganic & Medicinal Chemistry Letters, 19(21), 6063-6068. doi:10.1016/j.bmcl.2009.09.037. DOI: https://doi.org/10.1016/j.bmcl.2009.09.037
  8. Kang, I. J., Wang, L. W., Lee, C. C., Lee, Y. C., Chao, Y. S., Hsu, T. A., & Chern, J. H. (2009). Design, synthesis, and anti-HCV activity of thiourea compounds. Bioorganic & Medicinal Chemistry Letters, 19(7), 1950-1955. doi:10.1016/j.bmcl.2009.02.048. DOI: https://doi.org/10.1016/j.bmcl.2009.02.048
  9. Kang, I. J., Wang, L. W., Yeh, T. K., Lee, C. C., Lee, Y. C., Hsu, S. J., . . . Chern, J. H. (2010). Synthesis, activity, and pharmacokinetic properties of a series of conformationally-restricted thiourea analogs as novel hepatitis C virus inhibitors. Bioorganic & Medicinal Chemistry, 18(17), 6414-6421. doi:10.1016/j.bmc.2010.07.002. DOI: https://doi.org/10.1016/j.bmc.2010.07.002
  10. Khatri, N., Lather, V., & Madan, A. K. (2015). Diverse classification models for anti-hepatitis C virus activity of thiourea derivatives. Chemometrics and Intelligent Laboratory Systems 140, 13-21. doi:10.1016/j.chemolab.2014.10.007. DOI: https://doi.org/10.1016/j.chemolab.2014.10.007
  11. Khurana, M., Chaubey, Y. P., & Chandra, S. (2014). Jackknifing the ridge regression estimator: A revisit. Communications in Statistics-Theory and Methods, 43(24), 5249-5262. DOI: https://doi.org/10.1080/03610926.2012.729640
  12. Kibria, B. M. G. (2003). Performance of some new ridge regression estimators. Communications in Statistics - Simulation and Computation, 32(2), 419-435. doi:10.1081/SAC-120017499. DOI: https://doi.org/10.1081/SAC-120017499
  13. Kibria, B. M. G., Månsson, K., & Shukur, G. (2015). A Simulation Study of Some Biasing Parameters for the Ridge Type Estimation of Poisson Regression. Communications in Statistics - Simulation and Computation, 44(4), 943-957. doi:10.1080/03610918.2013.796981. DOI: https://doi.org/10.1080/03610918.2013.796981
  14. Le Cessie, S., & Van Houwelingen, J. C. (1992). Ridge estimators in logistic regression. Journal of the Royal Statistical Society: Series C (Applied Statistics), 41(1), 191-201. DOI: https://doi.org/10.2307/2347628
  15. Lee, A., & Silvapulle, M. (1988). Ridge estimation in logistic regression. Communications in Statistics-Simulation and Computation, 17(4), 1231-1257. DOI: https://doi.org/10.1080/03610918808812723
  16. Månsson, K., & Shukur, G. (2011). A Poisson ridge regression estimator. Economic Modelling, 28(4), 1475-1481. doi:10.1016/j.econmod.2011.02.030. DOI: https://doi.org/10.1016/j.econmod.2011.02.030
  17. Montgomery, D. C., Peck, E. A., & Vining, G. G. (2015). Introduction to linear regression analysis. New York: John Wiley & Sons.
  18. Nyquist, H. (1988). Applications of the jackknife procedure in ridge regression. Computational Statistics & Data Analysis, 6(2), 177-183. DOI: https://doi.org/10.1016/0167-9473(88)90048-5
  19. Özkale, M. R. (2008). A jackknifed ridge estimator in the linear regression model with heteroscedastic or correlated errors. Statistics & Probability Letters, 78(18), 3159-3169. doi:10.1016/j.spl.2008.05.039. DOI: https://doi.org/10.1016/j.spl.2008.05.039
  20. Rashad, N. K., & Algamal, Z. Y. (2019). A New Ridge Estimator for the Poisson Regression Model. Iranian Journal of Science and Technology, Transactions A: Science. doi:10.1007/s40995-019-00769-3. DOI: https://doi.org/10.1007/s40995-019-00769-3
  21. Saleh, A. K. M. E., Arashi, M., & Kibria, B. M. G. (2019). Theory of Ridge Regression Estimation with Applications. New York: Wiley. DOI: https://doi.org/10.1002/9781118644478
  22. Saleh, A. K. M. E., & Kibria, B. M. G. (2012). Improved ridge regression estimators for the logistic regression model. Computational Statistics, 28(6), 2519-2558. DOI: https://doi.org/10.1007/s00180-013-0417-6
  23. Schaefer, R., Roi, L., & Wolfe, R. (1984). A ridge logistic estimator. Communications in Statistics-Theory and Methods, 13(1), 99-113. DOI: https://doi.org/10.1080/03610928408828664
  24. Singh, B., Chaubey, Y., & Dwivedi, T. (1986). An almost unbiased ridge estimator. Sankhyā: The Indian Journal of Statistics, Series B, 13, 342-346.
  25. Türkan, S., & Özel, G. (2015). A new modified Jackknifed estimator for the Poisson regression model. Journal of Applied Statistics, 43(10), 1892-1905. doi:10.1080/02664763.2015.1125861. DOI: https://doi.org/10.1080/02664763.2015.1125861
  26. Williams, U., Kibria, B. M. G., & Mansson, K. (2019). Performance of some ridge regression estimators for the Logistic Regression Model: An Empirical Comparison. International Journal of Statistics and Reliability Engineering, 6(1), 1-12.