0

This question comes from MånsT's answer of question

The least squares estimators of $β_1$,$β_2$,… are not affected by shifting. The reason is that these are the slopes of the fitting surface - how much the surface changes if you change $x_1$,$x_2$,… one unit. This does not depend on location. (The estimator of $β_0$, however, does.)

Such question first arised when I centered the predictor value. Can someone give a rigorous proof maybe by matrix language? And how would it change intercept term? Some related question:
- Show that $\hat{\beta}_0 = \bar{y}$ for OLS when the columns of $\mathbf{X}$ are centered
- Why does the y-intercept of a linear model disappear when I standardize variables?
- Prove Estimated Regression Coefficients are the same with or without an intercept term

Richard Hardy
  • 54,375
  • 10
  • 95
  • 219
Spaceship222
  • 221
  • 1
  • 5
  • 1
    One rigorous solution follows from the final remark in my answer to a related question at https://stats.stackexchange.com/a/108862/919: "The constant term will be the difference between the mean of $y$ and the mean values predicted from the estimates, $X\hat\beta.$" The answer itself shows that $\hat\beta$ depends only on the covariance matrix of all the variables and that obviously does not change when the predictors are shifted. That should also make clear that the conclusion follows *only* when the constant vector lies in the span of the columns of the design matrix. – whuber Apr 28 '19 at 16:05
  • @whuber Thanks! Can you provide a geometric interpretation for this property? I think the answer above are geometric interpretation,but I can't figure out – Spaceship222 Apr 29 '19 at 02:27
  • @whuber By the way, in your posted answer, if $\alpha$ is intercept, I think the system of linear equations should be $$ X' X \hat{b}=X' y $$ where $\hat{b} = (\hat{\alpha},\hat{\beta})$. Then after Gaussian elimination, we get $$ C \hat{\beta}=\left(\operatorname{Cov}\left(X_{i}, y\right)\right)' $$ Am I wrong? – Spaceship222 Apr 29 '19 at 02:28
  • A geometric interpretation is that the fitted value at the barycenter of the regressors is the barycenter of the response variable(s). – whuber Apr 29 '19 at 13:17

0 Answers0