0

For regression analysis (mine specifically multinomial logit) with the objective of prediction, is it truly necessary to scale variables before fitting the model? What if I want to apply regularization as well?

I cross validated both models (one with scaling and one without) and the performance measures weren't very different (sometimes the non scaled one was better, even). So I wondered, is it absolutely necessary to do that even when the performance of non-scaled variables is better?

amestrian
  • 233
  • 1
  • 7
  • 2
    It's not. If it was 100% always necessary, it would be implemented as default in statistical packages or in the algorithm descriptions. In some cases scaling won't make much of a difference, in other may significantly affect the end results (e.g., for regularization (say lasso) we'd want the penalty to be consistent throughout different variables and penalize them equally in order to better gauge their explanatory information; without scaling we could under/over penalize certain variables, hence in most implementations the scaling is always done under the hood, with the option to disable it) – runr Aug 18 '20 at 11:35
  • 1
    @Nutle why don't you copy that comment into an answer? It nicely explains the unique situation with penalization, the one time that it really matters with regression. – EdM Aug 18 '20 at 15:35
  • See also: https://stats.stackexchange.com/questions/19216/variables-are-often-adjusted-e-g-standardised-before-making-a-model-when-is – cbeleites unhappy with SX Aug 19 '20 at 10:00

1 Answers1

1

It's not. If it was 100% always necessary, it would be implemented as default in statistical packages or in the algorithm descriptions.

In some cases scaling won't make much of a difference, in other may significantly affect the end results. E.g., for regularization (say lasso) we'd want the penalty to be consistent throughout different variables and penalize them equally in order to better gauge their explanatory information; without scaling we could under/over penalize certain variables, hence in most implementations the scaling is always done under the hood, with the option to disable it.

Further reading for more detailed answers:

Is standardization needed before fitting logistic regression?

When conducting multiple regression, when should you center your predictor variables & when should you standardize them?

Is it a good practice to always scale/normalize data for machine learning?

runr
  • 622
  • 3
  • 13
  • Thanks for turning your comment into an answer (+1). Formal answers are particularly helpful to new visitors to the site who might have a similar question. Providing references to other threads for more detail, as you did, is very helpful. – EdM Aug 19 '20 at 14:29