1

I have a decent understand of OLS regression. Now, what if my observation isn't normally distributed anymore, how can I estimate my parameter of the regression model?

I was trying to estimate the parameter by minimize the negative log-likelihood. I have $$ l(\theta,\gamma) = n\log(\gamma) - \sum \log(\gamma^2 + (x-\theta)^2 ) $$ where $\gamma$ and $\theta$ is scale and location parameter of cauchy distribution. Then, let $ \frac{dl}{d\theta} = 0 , \frac{dl}{d\gamma} = 0 $, I got $$ \sum\frac{x-\theta}{\gamma^2 + (x - \theta)^2} =0 $$ and $$ \sum\frac{\gamma^2}{\gamma^2 + (x - \theta)^2} = n/2 $$

How do you go from $\theta$ and $\gamma$ to $\beta$? Am I on the right track at all?

kjetil b halvorsen
  • 63,378
  • 26
  • 142
  • 467
Phume
  • 75
  • 6
  • 1
    You have the location one correct, but the scale one looks (very) slightly wrong (the numerator in the sum is not quite right). See some related questions [here](http://stats.stackexchange.com/questions/174117/maximum-likelihood-estimator-of-location-parameter-of-cauchy-distribution) and [here](http://stats.stackexchange.com/questions/98971/mle-of-the-location-parameter-in-a-cauchy-distribution) which discuss the location parameter for a unit scale and may help focus your question – Glen_b Oct 10 '16 at 10:15
  • @Glen_b, thanks for the link!. Since I have two variables to solve, I assume Newton-Raphson Method wouldn't work here. – Phume Oct 10 '16 at 17:21
  • 1
    Newton Raphson works with multiple variables, but derivatives are vector-valued and second derivatives (and their inverses) become matrices. However if you start with good estimates you should be able to alternate updating your estimate of one and then the other though it may be slower. You can get a good (highly efficient) estimate of $\gamma$ (for $n>2$) by using the approach in Kravchuk&Pollett (there's a working version here https://people.smp.uq.edu.au/PhilipPollett/papers/ScaleRankCauchy.pdf), but see the section in wikipedia on estimation of parameters ... ctd – Glen_b Oct 10 '16 at 19:02
  • 1
    ctd... [here](https://en.wikipedia.org/wiki/Cauchy_distribution#Estimation_of_parameters); in particular the likelihood when estimating the location can sometimes have multiple local modes and MLE may not be efficient at small sample sizes. – Glen_b Oct 10 '16 at 19:02
  • You use irls, see: https://stats.stackexchange.com/questions/236676/can-you-give-a-simple-intuitive-explanation-of-irls-method-to-find-the-mle-of-a for a thourough discussion (maybe a duplicate) – kjetil b halvorsen Sep 08 '17 at 15:02
  • 2
    Possible duplicate of [can you give a simple intuitive explanation of IRLS method to find the MLE of a GLM?](https://stats.stackexchange.com/questions/236676/can-you-give-a-simple-intuitive-explanation-of-irls-method-to-find-the-mle-of-a) – kjetil b halvorsen Sep 08 '17 at 15:02

0 Answers0