1

I would like to perform multiple regression with two variables that have the same unit but have very different gradient lengths. (one has a range from 1-20, the other has a range from 1-40) How do I cope with variables of different lengths in regression? Is centering of the data enough to avoid that the variable with the longest gradient has the biggest impact?

cheers Leke

Penguin_Knight
  • 11,078
  • 29
  • 48
Leke
  • 11
  • 2
  • 1
    Do you mean one only has 20 cases and the other one has 40 cases? – Penguin_Knight May 20 '14 at 14:54
  • no sorry, they have both over 500 cases, the units differ in length. It is not weight but some complicated unit but lets say it is: one goes from 1-20 kg and the other goes from 1-40kg. I would like to see which of the two has the biggest impact (per kg) – Leke May 20 '14 at 15:22

1 Answers1

1

The short answer is that this is not a mathematical issue for linear regression, though you may get a substantive/interpretability benefit by standardizing your input/predictors/features/covariates.

In linear regression the coefficients $\beta_k$ will be in the proper units/scale for the equation $$ y_i = \epsilon_i + \text{Intercept} + \sum_k \beta_k x_{ik} $$ to make sense in the units of $y_i$ & each $x_{ik}$. Since the coefficients are automatically adjusting their scale, there isn't a reason to be worried about inputs with different ranges 'overpowering' inputs with smaller ranges. Thus, standardizing your input data will not have any effect on the statistical merits of your model in terms of prediction. Note that this is NOT universally the case throughout statistics/machine learning (e.g. support vector machines can benefit from standardization).

However, standardizing your data will allow you to interpret the $\text{Intercept}$ term as the average $y$ output for the average $x$ input. See this question from yesterday. This may or not be useful to you. I usually don't bother unless the situation calls for it.

Community
  • 1