1

I find following commonly mentioned linear regression methods:

OLS: ordinary least squares

GLS: generalized least squares

WLS: weighted least squaes

RLM: robust linear model

OLS is usually the default. I believe robust model is to be used to correctly handle outliers, but I am not clear about others.

What are the criteria to choose one over the other?

Edit: It is mentioned in the comments that it is a very broad question (I did not know that!). However, I would like to have a one or two lines on each of above to know the "indications" or when to use them.

OLS: default

RLM: if outliers are important and cannot be ignored.

That leaves only GLS and WLS. What would be most important reasons to use them?

rnso
  • 8,893
  • 14
  • 50
  • 94
  • This is very general, and whole books are needed for an answer. Could you limit the scope somewhat? Linear regression methods for ... ? – kjetil b halvorsen Jul 12 '20 at 17:35
  • I have tried to specify that I need only a general indication on when to use each of these. – rnso Jul 12 '20 at 17:40
  • 2
    OLS is useful as *pedagogically* to generally introduce regression and it's concepts and assumptions, but I am not sure if or when I actually have used OLS in research. Regression methods permitting [creative violation of OLS regression's assumptions](https://stats.stackexchange.com/a/100222/44269) permit us to model the complexities with which the world behaves with more fidelity. – Alexis Jul 12 '20 at 17:49

1 Answers1

2

Your links goes to statsmodels program web pages, a software I do not know. I will assume their use of terms is the standard. A very general indication, just as a starter, what you really need is a book on regression.

  • OLS is the starting point, many other models can be seen as extensions or generalization. Assumptions is continuous response, linear effects and constant variance + independence (of residuals.)

  • GLS weakens assumption, do not assume constant variance nor independence. So you will need somehow to model the variance and covariances.

  • WLS is GLS but with covariances zero, so really an assumption of independence of residuals.

  • RLM is really a huge class of models and methods. Especially think about this for routine or automatized analyses.

kjetil b halvorsen
  • 63,378
  • 26
  • 142
  • 467
  • I had asked earlier https://stats.stackexchange.com/questions/146077/why-not-robust-regression-everytime . But I cannot understand how to adjust options in RLM since you say that RLM is "a huge class and methods". – rnso Jul 14 '20 at 03:11
  • It is suggested at https://stats.stackexchange.com/questions/473603/log-transform-in-linear-regression that WLS is useful if there is a problem of Heteroscedasticity. – rnso Jul 15 '20 at 00:55
  • Well, that is what I tried to say, *do not assume constant variance*. – kjetil b halvorsen Jul 15 '20 at 01:21