13

When is it preferable to use Maximum Likelihood Estimation instead of Ordinary Least Squares? What are the strengths and limitations of each? I am trying to gather practical knowledge on where to use each in common situations.

user38133
  • 457
  • 3
  • 5
  • 8
  • See also https://stats.stackexchange.com/questions/143705/maximum-likelihood-method-vs-least-squares-method – gboeing Oct 13 '20 at 00:15

1 Answers1

2

As explained here, OLS is just a particular instance of MLE. Here is closely related question, with a derivation of OLS in terms of MLE.

The conditional distribution corresponds to your noise model (for OLS: Gaussian and the same distribution for all inputs). There are other options (t-Student to deal with outliers, or allow the noise distribution to depend on the input)

Community
  • 1
jpmuc
  • 12,986
  • 1
  • 34
  • 64
  • (-1) It's not true that the student $t$ can be depended upon to *deal with the outliers*. See the example [here](http://stats.stackexchange.com/a/86447/603) or chapter 2 of [this](http://onlinelibrary.wiley.com/book/10.1002/0470010940) textbook for more explanations. – user603 Feb 16 '14 at 10:58
  • 9
    The OLS is a distance-minimizing approximation/estimation method, while ML is a "likelihood" maximization method. OLS needs no stochastic assumptions to provide his distance-minimizing solution, while ML _starts_ by assuming a joint probability density/mass function. The fact that in some circumstances the two provide the same solution, in no way does it make the one a particular instance of the other. – Alecos Papadopoulos Feb 16 '14 at 14:35