0

Sometimes in OLS model we have constant for example -2345 significant and doesn't have a mean. Why we must keep it in the model? Why when we drop it the results change? What does it mean?

And sometimes it is insignificant. Why?

Nick Cox
  • 48,377
  • 8
  • 110
  • 156
user27746
  • 1
  • 1
  • 1
  • 1
    What do you mean by "doesn't have a mean"? Many questions on omitting the intercept on this site e.g. http://stats.stackexchange.com/questions/7948/when-is-it-ok-to-remove-the-intercept-in-lm/ – Nick Cox Jul 08 '13 at 09:22

1 Answers1

2

In most cases, the constant in the OLS model doesn't make any sense in reality. Because if we'd like to interpret the meaning of the constant, we set all regressors to zero. For instance, we want to estimate a regression of a house's price on their sizes of lot in feet including constant; and apparently it's impossible to find such a house with no size of lot.

Although the constant loses the interpretation, it plays a critical role in obtaining an unbiased estimator. In order to get the "true" estimator, we want to minimise the sum of square error and hence this process usually returns a model with intercept. The model without constant tends to get larger error. Here's a graph of a simple linear regression model from the wikipedia. But if we draw a straight line through the origin, it could be seen that the direction of the regression line would completely change and how larger the error would be.

Earo Wang
  • 306
  • 1
  • 5
  • I'd rephrase some of this. The constant is perfectly intelligible as the value predicted when all predictors are zero. Algebraic and geometric interpretations are easy. I'd agree that it's often not especially interesting or useful, but that is not the same as not making any sense. Conversely, there are occasions when linear proportionality is expected and fitting a line through the origin is exactly the right thing to do (think Hooke's law, Ohm's law, etc.). It is harder to think of really good reasons for fitting a hyperplane through the origin, but perhaps someone has a good example. – Nick Cox Jul 08 '13 at 10:43