1

I would like to minimise the function $l(\theta|Y)$.

Given the Newton's method below

$$\theta^{(t+1)} = \theta^{(t)} - \left[l''(\theta\;|\;Y)\right]^{-1} l'(\theta^{(t)}\; | Y)\quad t = 0,1,...$$

where $l''(\theta^{(t)} | Y)$ is the observed Fisher Information matrix.

The observed fisher information matrix (including the minus sign in front of it) would ideally be negative definite, in order to find the minimum. Correct me if I am wrong.

My question is: what if the observed Fisher Information I obtained is negative semidefinite. What may happen is this case ? Would my algorithm fail miserably ?

mynameisJEFF
  • 1,583
  • 4
  • 24
  • 29
  • See [this discussion](http://stats.stackexchange.com/questions/7308/can-the-empirical-hessian-of-an-m-estimator-be-indefinite). Short summary is that positive definitess (not negative definite think minimum of $x^2$ and its second derivative) is the required condition for solution to be a local minimum. If this condition is violated then the solution is not the local minimum. – mpiktas Mar 13 '14 at 12:01
  • Sorry. I am a little bit confused here. Which one needs to be positive definite ? $- l''[\theta |Y]$ or $l''[\theta | Y]$ ? – mynameisJEFF Mar 13 '14 at 17:24

0 Answers0