1

The following question is an answer for why lm.ridge and glmnet results are different and how to solve that. My question is different since I am asking which functions are more trusted to use because they are giving different results each time when compared with textbook calculation.

I am working with ridge regression in R. I have three functions to use which are glmnet or lm.ridge or textbook calculation.

I can make the glmnet results as textbook calculation results.

p.tmp <- 5
n.tmp <- 30

set.seed(123)
X=scale(matrix(rnorm(n.tmp*p.tmp),ncol = p.tmp))
Y=scale(rnorm(n.tmp))

set.seed(123) 
ridge.fit.cv <- cv.glmnet(X, Y, alpha = 0, standardize = FALSE, intercept = FALSE)
ridge.fit.lambda <- ridge.fit.cv$lambda.1se
ridge_mod <- coef(glmnet(X, Y, alpha=0, standardize = FALSE,   lambda = ridge.fit.lambda) )[-1]
###############################
ridge.coef.DEF = solve(t(X) %*% X + n.tmp* ridge.fit.lambda * diag(p.tmp)) %*% t(X) %*% Y
cbind(glmnet=c(ridge_mod),textbook=c(ridge.coef.DEF), lm.ridge=c(coef( lm.ridge(Y~X - 1, lambda=ridge.fit.lambda))))

The results as follows

          glmnet      textbook     lm.ridge
X1  3.040157e-04  3.092039e-04  0.009102455
X2  9.805261e-04  9.972422e-04  0.028596273
X3  1.514714e-04  1.540631e-04  0.004855324
X4  1.721871e-04  1.751177e-04  0.004789519
X5 -4.282657e-05 -4.355457e-05 -0.001143931

Or I can also make lm.ridge results as textbook calculation as giving below

set.seed(123) 
ridge.fit.cv <- cv.glmnet(X, Y, alpha = 0, standardize = FALSE, intercept = FALSE)
ridge.fit.lambda <- ridge.fit.cv$lambda.1se
ridge_mod <- coef(glmnet(X, Y, alpha=0, standardize = FALSE,   lambda = ridge.fit.lambda) )[-1]
###############################
ridge.coef.DEF = solve(t(X) %*% X + ridge.fit.lambda * diag(p.tmp)) %*% t(X) %*% Y
cbind(glmnet=c(ridge_mod),textbook=c(ridge.coef.DEF), lm.ridge=c(coef( lm.ridge(Y~X - 1, lambda=ridge.fit.lambda*(30/29)))))

The results as

          glmnet     textbook     lm.ridge
X1  3.040157e-04  0.008815056  0.008815056
X2  9.805261e-04  0.027715213  0.027715213
X3  1.514714e-04  0.004692398  0.004692398
X4  1.721871e-04  0.004649196  0.004649196
X5 -4.282657e-05 -0.001112091 -0.001112091

My question is which method I should believe because as you notice the results are different.

Should I make the text calculation as glmnet or as lm.ridge?

jeza
  • 1,527
  • 2
  • 16
  • 37
  • Your edited question does not appear to have any foundation. The results table at the end shows that `textbook` and `lm.ridge` agree to eight decimal places (maybe the textbook used `lm.ridge`!) and the duplicate explains why `glmnet` should *not* agree with the others. – whuber Apr 06 '21 at 20:33
  • Yes, you are right and this is the reason for my question. Based on this difference which functions someone should use and trust with the Ridge regression. Bear in mind that the textbook is also shown to be approximate as glmnet. Also, lambda in lm.ridge is multiplied by (n/n-1) to have the same eight decimal places. So, the question is still open to hearing from other experts in the field. – jeza Apr 06 '21 at 20:52
  • I still don't understand what your question is. It seems to be asking for software recommendations, but those aren't on topic here. – whuber Apr 06 '21 at 20:55
  • The software is only for clarification. I am not asking for a code edit. The R- function is usually related to theoretical work. – jeza Apr 06 '21 at 20:58

0 Answers0