9

It seems that OLS residuals autocorrelation is not always an issue, depending on the problem at hand. But why residuals autocorrelation would affect the coefficient standard errors? From the Wikipedia article on autocorrelation:

While it does not bias the OLS coefficient estimates, the standard errors tend to be underestimated (and the t-scores overestimated) when the autocorrelations of the errors at low lags are positive.

Community
  • 1
Robert Kubrick
  • 4,078
  • 8
  • 38
  • 55
  • 2
    Consider an extreme case of correlation. Suppose all the errors were perfectly positively correlated. In other words, somebody had generated a *single* random number and added it to all the response values. How certain would you be of (say) the intercept in the regression? Would you have any clues at all concerning the size of the random value that was added? – whuber Sep 06 '14 at 22:55
  • Yes, but that is true of any missing predictor that could explain 99% of the variance and we just ignore. Why are making a specific case for $Y_{t-1}$? – Robert Kubrick Sep 06 '14 at 22:59
  • My example is not missing any predictors at all: it is only positing an extreme case of autocorrelation among the *residuals*. – whuber Sep 07 '14 at 13:34
  • ok, but how is this different than the case where we don't have *any* residuals autocorrelation, but we're not including another critical predictor? We can draw the same confidence conclusions because of that other critical predictor. The only difference is that in the case of a missing $Y_{t-1}$ critical predictor we can observe autocorrelation in the residuals. Other than that, I'm not clear what makes autocorrelation stand apart from other critical predictors. – Robert Kubrick Sep 07 '14 at 15:35
  • I'm afraid I cannot make any sense of that last comment. It's not even clear what you mean by "$Y_{t-1}$". If by that you mean a lagged variable in a time series, then you are no longer talking about OLS regression. – whuber Sep 07 '14 at 17:23
  • That would be the lagged DV. That is way we observe auto correlated residuals, because $Y$ and $Y_{t-1}$ are correlated. – Robert Kubrick Sep 07 '14 at 17:28

2 Answers2

9

Suppose your OLS regression is well specified and contains all the right explanatory variables, but you have an unspecified correlation structure of the residuals: $$ y_t = x_t' \beta + \epsilon_t, \mathbb{V}[\mathbf{\epsilon}]=\Omega $$ The OLS estimates are $$ \hat\beta = (X'X)^{-1} X'Y = \beta + (X'X)^{-1} X'\mathbf{\epsilon} $$ and their variance is $$ \mathbb{V}[\hat\beta] = \mathbb{E} [ (X'X)^{-1} X'\mathbf{\epsilon}\mathbf{\epsilon}'X (X'X)^{-1} ] $$ Typically, at this stage, we'd have to assume something like existence of the probability limit of $\frac1T (X'X) \to \Sigma$, so that $$ T \mathbb{V}[\hat\beta] \to \Sigma^{-1} {\rm plim} \bigl[ \frac1T X'\mathbf{\epsilon}\mathbf{\epsilon}'X \bigr] \Sigma^{-1} = \Sigma^{-1} {\rm plim} \bigl[ \frac1T X'\Omega X \bigr] \Sigma^{-1} $$ This expression is different from what the naive OLS standard errors produce, and so in general the OLS standard errors are wrong.

Of course, if $X$ can be considered fixed, then there is no need for asymptotic approximations, and $X$ can be carried through the expectations, so that $$ \mathbb{V}[\hat\beta] = (X'X)^{-1} X'\Omega X (X'X)^{-1} $$ to the same effect.

StasK
  • 29,235
  • 2
  • 80
  • 165
  • 1
    All I get from this is that the higher the residuals, the higher $V[\hat{\beta}]$. So that will increase the SE, not underestimate it. And how is this specific to $Y_{t-1}$? – Robert Kubrick Sep 06 '14 at 23:06
  • Take the last formula by @StasK and consider the univariate case with nonzero off-diagonal elements (autocorrelation). See what happens in the numerator and compare that to what you get in the numerator when you assume $\Omega=\sigma^2 I$. – ekvall Sep 07 '14 at 00:20
  • @KarlOskar I still don't see anything specific to $Y_{-1}$ in the equation above, only the residuals impact the results (which is true in any case, not just $Y$ autocorrelation). – Robert Kubrick Sep 07 '14 at 16:47
  • @RobertKubrick, not sure what you mean? Autocorrelation usually refers to $\mathrm{cor}(\epsilon_t,\epsilon_{t-h})\neq 0$ for some $h$. – ekvall Sep 07 '14 at 20:51
  • 3
    @RobertKubrick that's exactly the point: having any unmodeled correlation is bad, autocorrelation is just a common special case of having unmodeled correlation. – Aniko Sep 07 '14 at 22:03
  • 1
    @Aniko thank you to confirm! That's what I was trying to understand in my question comments as well. – Robert Kubrick Sep 07 '14 at 22:06
0

Auto correlation (Exogeneity) : If the observation of residual in t1 is dependent on the observation of residual in t0, then it basically violates the underlying assumption of OLS which states "Error term is independently distributed and not correlated". This can bias the estimation of Beta coefficient of OLS term.

When such trend is observed in the residuals, the residuals of regression model picks up the influence of those variables that are affecting the dependent variables that aren't part of the regression equation.

The persistence in excluded variables is most of the times a cause for autocorrelation. This is more prevalent in Time series data.

This can be mitigated by using some of the transformations techniques like,

  1. Paris Winston Transformation of Data
  2. Differencing the dependent variables
  3. Differencing all variables

One simpler approach would be,

  1. estimating the linear model using OLS.
  2. Compute the residuals.
  3. Regress the residuals of all Independent variables and lagged variables.
  4. Use t-test, if the coefficient of lagged residual is significant, we can reject the null of independent errors.

Test to detect Auto correlation: **

  1. Durbin - Watson
  2. Bruesch - Godfrey

This assumption/issue is violated/exempted if the explanatory variables are stochastic, for instance when they are measured with error, or are endogenous.

Suriya Kumar J S
  • 33
  • 8