Role of Gauss-Markov Theorem in Linear Regression

Question

How does being BLUE matter in Linear Regression for the coefficients? What does Heteroscedasticity Consistent & Auto-correlation Consistent Dispersion matrix take care off in this regard?

Answer 1

"How does being BLUE matter in Linear Regression for the coefficients?" (I leave the other question about robust standard errors for others)

BLUE is best linear unbiased estimates. The gauss-Markov theorem gives that for linear models with uncorrelated errors and constant variance, the BLUE estimator is given by ordinary least squares, among the class of all linear estimators. That might have been comforting in times where limited computation power made computing some non-linear estimators close to impossibe, even least squares estimators could be a significant effort (read man-hours)!

The ordinary least squares estimator also happen to be the maximum likelihood estimator when the errors (in addition to suppositions above) have gaussian distributions. So knowing that OLS are BLUE, then, makes it comforting to think that the gaussian assumption is of little import (even if the errors are clearly non-normal).

The situation today is different, computing non-linear estimators are computationally cheap. One of the commenters above ask what is this non-linear estimators of $\beta$ in the linear model. They are many. One traditional way could be using least squares after analysing residuals & influence, and removing outliers. That will be a non-linear function of the original, complete, data. Other examples are many, specially modern robust estimators give many examples, see for instance the good book https://www.amazon.com/Robust-Statistics-Methods-Ricardo-Maronna/dp/0470010924/ref=sr_1_1?s=books&ie=UTF8&qid=1489942791&sr=1-1&keywords=maronna+robust or see my answer at Why should we use t errors instead of normal errors?

Also, while variance is easy to interpret in the context of gaussian distributions, it is not at all clear that variance is that meaningful for non-gaussian distributions, so in such cases it is less clear that minimizing variance is a good criterion. See What is the difference between finite and infinite variance

Quoting Tukey (referenced in the book by Maronna above):

It is perfectly proper to use both classical and robust/resistant methods routinely, and only worry when they differ enough to matter. But when they differ, you should think hard.

These robust/resistant methods will usually be non-linear. One good reference (shorter than the book above) is http://web.archive.org/web/20160611192739/http://www.stats.ox.ac.uk/pub/StatMeth/Robust.pdf document by Brian Ripley. A quote from its first paragraph:

The classical books on this subject are Hampel et al. (1986); Huber (1981), with somewhat simpler (but partial) introductions by Rousseeuw & Leroy (1987); Staudte & Sheather (1990). The dates reflect the development of the subject: it had tremendous growth for about two decades from 1964, but failed to win over the mainstream. I think it is an important area that is used a lot less than it ought to be.

I hope above justifies my conclusion that the Gauss-Markov theorem today is mostly of historical interest. What holds back more routine use of robust/non-linear methods is probably that implementations often lack post-estimation inferential machinery, ready implemented to use as easily as one can use least squares in, say, R. (Maybe that is changing, did'nt look into this a long time).