12

Examples of this page show that simple regression is markedly affected by outliers and this can be overcome by techniques of robust regression: http://www.alastairsanderson.com/R/tutorials/robust-regression-in-R/ . I believe lmrob and ltsReg are other robust regression techniques.

Why should one not do robust regression (like rlm or rq) every time rather than performing simple regression (lm)? Are there any drawbacks of these robust regression techniques? Thanks for your insight.

rnso
  • 8,893
  • 14
  • 50
  • 94
  • 2
    [This](http://stats.stackexchange.com/questions/1164/why-havent-robust-and-resistant-statistics-replaced-classical-techniques) might be helpful. – conjugateprior Apr 13 '15 at 08:22

1 Answers1

6

The Gauss-Markov theorem:

In a linear model with spherical errors (which along the way includes an assumption of no outliers, via a finite error variance), OLS is efficient in a class of linear unbiased estimators - there are (restrictive, to be sure) conditions under which "you can't do better than OLS".

Christoph Hanck
  • 25,948
  • 3
  • 57
  • 106
  • So if there are no outliers,linear regression would be the best. But if there are, or if other assumptions are being violated, then only one should perform robust regressions. Is that correct? – rnso Apr 13 '15 at 12:29
  • 2
    If there are outliers, other techniques are better, yes. I would not jump to the conclusion that "if other assumptions are being violated, then [...] one should perform robust regressions" - it is not a cure-all for all violations. F.x., when errors are correlated with regressors and you are after causal effects, instrumental variables techniques are called for. – Christoph Hanck Apr 13 '15 at 13:52
  • The Gauss-Markov theorem does not make a normality assumption. We still get the BLUE if the error is heavy-tailed. – Dave Oct 14 '20 at 14:21
  • Correct about normality; nor do I say it does. It does make an assumption of finite error variances, though, so does that square with "heavy tails" to you? – Christoph Hanck Oct 14 '20 at 15:10