5

It appears to me that if I move the mode $u$ more to the negative and increase the scale parameter $\alpha$, one can get always a higher likelihood. If this is true, is there a limit of the likelihood?

The Gumbel likelihood is given by

$$\log(L) = \sum_{i=1}^{N}\log \left[\dfrac{1}{\alpha}\exp(-y_i - e^{-y_i}) \right]$$

where $y_i = \dfrac{x_i-u}{\alpha}$

As User Xi'an points out, there appears to be a (local) maximum somewhere. However, I'm more asking about the global maximum and whether one exists.

Yves
  • 4,313
  • 1
  • 13
  • 34
Harald Thomson
  • 384
  • 1
  • 9

2 Answers2

4

I will prove a more general result: if a density is log-concave, then the log-likelihood of the corresponding location-scale family has a global maximum. The wanted result then follows, since the Gumbel density is log-concave.

Consider a univariate density $f^\star(y)$ which is log-concave and smooth on the real line; the parameterized density using the location $\mu$ and the scale $\alpha >0$ is $$ f(x;\,\mu,\,\alpha) := \frac{1}{\alpha} \, f^\star\left(\frac{x - \mu}{\alpha}\right). $$ We can use the following alternative parameter vector $[\nu,\,\beta]$ with $\beta >0$ $$ \beta := 1 / \alpha, \qquad \nu := - \mu / \alpha. $$ We have a one-to-one smooth transformation $[\mu,\,\alpha] \mapsto [\nu,\,\beta]$. Using the parameter vector $[\nu,\,\beta]$, the density at $x$ writes as $\beta\,f^\star(\beta x + \nu)$ and the log-likelihood for a sample $X_i$ is $$ \log L = \sum_{i=1}^N \log\{ \beta \, f^\star(\beta X_i + \nu) \}. $$ It is clear that this is a concave function of the vector $[\nu,\,\beta]$ and hence that a global maximum exists (possibly for infinite $\nu$ or $\beta$). But this implies that a global maximum exists as well for the location-scale parameterization $[\mu,\,\alpha]$.

Yves
  • 4,313
  • 1
  • 13
  • 34
  • Very nice result. Well done. With your answer however, I realized the question I was asking was the wrong one, because from a practitioners perspective the result is a bit disappointing as $\alpha$ and $\mu$ still could be arbitrary large/small. The question I wanted to ask is whether there are values $\alpha$ and $\mu$ where the global max is attained. At least I know now that the Gumbel-Fit with ML has a global maximum. – Harald Thomson Oct 25 '18 at 10:51
  • Thank you. In practice you only need to find a local maximum using the new parameters: it will be the global maximum. Moreover, a workable alternative can rely on a one-dimensional optimization, see [my answer here](https://stats.stackexchange.com/a/233176/10479). – Yves Oct 25 '18 at 12:16
3

Here is the (log-)likelihood surface in $(u,\alpha)$ for a sample of 100 points from a standard Gumbel:

enter image description here

As you can see, the mode is located near $(0,1)$ which is the true value of the parameter. The graph was made by the following R code

library(VGAM)
obs=rgumbel(1e3)
loca=seq(min(obs),max(obs),le=1e2)
scala=seq(.1*sd(obs),10*sd(obs),le=1e2)
like=matrix(0,1e2,1e2)
for (i in 1:1e2)
  for (j in 1:1e2)
   like[i,j]=sum(dgumbel(x=obs,loc=loca[i],scal=scala[j],log=TRUE))
Taylor
  • 18,278
  • 2
  • 31
  • 66
Xi'an
  • 90,397
  • 9
  • 157
  • 575
  • I know that there is a at least a local maxima somewhere, but the question deals more with the case whether there is a global maxima. I'll update the question – Harald Thomson Jun 13 '18 at 12:38