7

The Hessian of the log likelihood function is

$$\frac{\partial^2 \ln(\beta \mid x)}{\partial \beta \partial \beta'} = -\sum_{i=1}^n \underbrace{\Lambda(\beta'x_i)}_{\in(0,1)}\underbrace{\left[1-\Lambda(\beta'x_i)\right]}_{\in(0,1)}\underbrace{x_ix_i'}_{\geq 0}$$

where I have marked my understanding of the ranges of the different factors with underbraces. The Hessian would be zero if $x_i=\mathbf{0}$ for all $i$. Thus, I would conclude that the Hessian was negative semi-definite.

Yet in Greene (p. 691-692)---which specify the Hessian just as I do---it says

Note that the Hessian is always negative definite, so the log-likelihood is globally concave.

What am I missing here?

Fredrik P
  • 436
  • 3
  • 12

1 Answers1

5

When they say the Hessian is always negative definite, they are assuming that $X$ is a full rank matrix, which is a very typical assumption for regression models.

So it is true that in your example, the Hessian would not be negative definite. However, they assume that you have informative data: you can't expect to estimate the effect of $x$ if you don't observe different levels of $x$.

Cliff AB
  • 17,741
  • 1
  • 39
  • 84
  • I'll also note that statisticians tend to be a little lazier about stating such things than mathematicians! – Cliff AB May 27 '15 at 03:35