2

I understand that the Maximum Likelihood Estimator for variance, in general, is biased (the average calculated from the sample itself reduces the degree of freedom by 1 e.t.c):

MLE <- sum((x - mean(x))^2) / n

But in single linear regression, it's assumed that the errors are independent and identically distributed as N(0, sigma^2), then the MLE for sigma^2 becomes

s^2 <- sum(error^2) / n

Is it still a biased estimator? (According to the textbook, yes; but I don't see why) Can somebody be so kind to explain this to me, please? I'm so confused.

PsychometStats
  • 2,147
  • 1
  • 11
  • 27
Paw in Data
  • 143
  • 4
  • 2
    Possible duplicate of [Why divide by $n-2$ for residual standard errors](https://stats.stackexchange.com/questions/201352/why-divide-by-n-2-for-residual-standard-errors) and https://stats.stackexchange.com/q/20227/77222 – Jarle Tufto Nov 28 '19 at 19:21

1 Answers1

3

  1. Yes, it is biased because, unlike the case of the OLS, the ML variance estimator does not include a correction. That is, the correction in the denominator should be n - k where n is your sample size, and k is the number of independent variables.

  2. Note that although the ML estimator of variance is biased, it is usually only the issue in smaller samples, where the OLS variance estimator is preferred.

  3. As the sample size is getting very large and eventually approaches infinity, both OLS and ML variance estimates converge. They converge because both are asymptotically equivalent.

Please read the estimation section of this page for more details. It explains this case well

PsychometStats
  • 2,147
  • 1
  • 11
  • 27