4

From my understanding, the property of OLS regression model is based on the assumption that there exists a set of $β_1$, $β_2$, $β_3$ ... $β_k$ that makes the error $ϵ$ satisfy the OLS assumptions (linear, homoscedasticity, etc.). And our further regression on the sample is to approximate the value of this set of coefficient.

However, I am confused about the logic behind the test of their existence. And also how can we conduct the test for each assumption separately but be able to conclude there is a set of $β_1$, $β_2$, $β_3$...$β_k$ that make all assumptions satisfied simultaneously. For example, we first run the test for autocorrelation and then validated the existence of the set of $β_1$, $β_2$, $β_3$...$β_k$ that make error ϵ derived from the model satisfy the independence assumption. Then we also validated the existence of the set of $β_1$, $β_2$, $β_3$...$β_k$ that make ϵ satisfy the homoscedasticity, but how can we conclude that they are the same set of $β_1$, $β_2$, $β_3$...$β_k$?

I know my understanding might be totally wrong, and I am very confused about the logic right now. Thanks in advance for explaining to me!

PsychometStats
  • 2,147
  • 1
  • 11
  • 27
Yuan
  • 473
  • 2
  • 6

1 Answers1

4

the property of OLS regression model is based on the assumption that there exists a set of β1, β2, β3...βk that make the error ϵ satisfy the OLS assumptions

I'm going to be a bit pedantic and insist this is an incorrect understanding. We don't find betas which force the residuals to be normal, rather we start by assuming the residuals truly are normal, that there is homogeneity of variance, that the mean truly is linear, etc etc, and then find the betas. That is a nuanced but important difference.

When we assess the diagnostics from an OLS, we are really assessing the validity of the assumptions we made.

EDIT: To PsychometStats' point about residuals versus error, I found this post clarifying for my own understanding. I think you'll find it enlightening too.

Demetri Pananos
  • 24,380
  • 1
  • 36
  • 94
  • 1
    +1 but I think it is worth-clarifying to the OP the difference between errors and residuals – PsychometStats Dec 07 '19 at 17:56
  • 1
    I'll add to your pedanticicity (?) (+1) and point out that we don't care much about Normality in OLS, it's nice but not a requirement in the same way linearity and homogeneity are. – jbowman Dec 07 '19 at 17:57
  • 1
    @Demetri Pananos, that's a good one thank you for adding! I actually had it saved in my bookmarks for a while. – PsychometStats Dec 07 '19 at 18:23
  • @Demetri Pananos, thank you for your explanation! But I am wondering we starts with making assumptions on the characteristic of ϵ, but different models with different betas will have different ϵ. Without the pre-specified value of betas, how can we get the value of ϵ? – Yuan Dec 07 '19 at 20:02
  • @Yuan the residuals are interpreted with respect to the betas. It doesn't make sense to talk about the residuals (the $\varepsilon$) without first specifying the coefficients. – Demetri Pananos Dec 07 '19 at 20:22
  • @Demetri Pananos Yes so can I take it this way-- we assume that there is a set of coefficient in the population and the error (unobservable) associated with them satisfy the assumptions. We leave some of the assumptions be assumptions (such as normality) but use some techniques to test the validity of others. And we use residuals as the proxy to the error in this process. If we believe the error doesn't satisfy the assumptions, then there is no such set of coefficient in the model we established that can meet all the assumptions of OLS and we have to reform the model. – Yuan Dec 07 '19 at 23:49