2

I've looked at some Kaggle notebooks lately of people using Lasso/Ridge for linear regression. The majority that I've seen don't seem to standardize the predictors before they fit Lasso/Ridge even though the variables are on disparate scales (e.g., multiple orders of magnitude in difference)

Here are a couple of Jupyter notebooks that I've seen that uses no standardization:

https://www.kaggle.com/mohaiminul101/car-price-prediction

https://www.kaggle.com/burhanykiyakoglu/predicting-house-prices/comments

Most of the notebooks I've seen actually lack this standardization, and I only look at the top rated notebooks for popular datasets, so I was thinking there their methodology may be more reputable. So now I'm wondering if there's something I'm missing, or if people are indeed being negligent/incorrect by not standardizing when using regularization.

Is there any theoretical justification or practical advantage to not performing standardization on regressors when they exist on disparate scales?

student010101
  • 334
  • 2
  • 10
  • 2
    I assume these are just beginner notebooks rather than competition entries – seanv507 May 08 '21 at 16:13
  • All serious regularised regression (LASSO/Ridge) I have read standardised their variables. Especially within Python, where we might often use pipeline, dropping a `StandardScaler`(somewhat mindlessly) is trivial to implement. – usεr11852 May 11 '21 at 00:11
  • @usεr11852 Can you suggest a resource where I may be able to find more reputable linear regression data analyses? Not advanced analysis per se, just something that shows reputable beginning to end procedure – student010101 May 11 '21 at 14:38
  • Kuhn & Johnson (2013) [Applied Predictive Modeling](https://link.springer.com/book/10.1007/978-1-4614-6849-3) is really readable and to the point. – usεr11852 May 11 '21 at 21:23
  • @usεr11852 I'll check that out. Is it more focused on prediction rather than inference? – student010101 May 11 '21 at 21:47
  • Yes, it is. (The clue is in the title. :) ) – usεr11852 May 11 '21 at 21:55

1 Answers1

2

The main reason I can think of is to penalize variables on different scales differently. However, unless different variables are deliberately re-scaled differently, I would rather guess that it's an oversight. See also this previous question.

Björn
  • 21,227
  • 2
  • 26
  • 65
  • I agree with you, I would also add that many libraries standardize variables by default (e.g., glmnet), so maybe models in those notebooks do that, but it's just not done explicitly in the code (read the docs). Also, kaggle notebooks are not necessarily a pinnacle of best modeling practiecs – rep_ho May 12 '21 at 17:28