Is likelihood function always unique?

Question

I am reading the following part of a book. What I do not understand in Sample 3 is that why $F_M(m)$ is not the likelihood function. As in sample 2, we produced the probability that $0.9< X < 4$. Similarly, $F_M(m)$ (can be expressed as $F_M(m,\theta)$ can provide the probability of $X_i\le3.5,i = 1,...,5$ for $m = 3.5.$

In other words, if I consider all the individual samples independent from each other, I can get likelihood of each sample as follows: $G_i(\theta) = \Phi(3.5-\theta)$ as $-\infty\le X_i\le 3.5$ After this, I can extract the likelihood as follows: $L_3(\theta) = \prod\limits_{i = 1}^5 {{G_i(\theta)}} = (\Phi(3.5-\theta))^5$.

What do I miss? Why is my result different than the book? Should the likelihood $L_3(\theta)$ be unique or my likelihood in above equation is also correct?

See https://stats.stackexchange.com/questions/97515/what-does-likelihood-is-only-defined-up-to-a-multiplicative-constant-of-proport/97522#97522 — kjetil b halvorsen, Jul 11 '18 at 13:23
@kjetilbhalvorsen Thanks. You pointed out a very very interesting example. Moreover, do you think that the likelihood function I made and the one in the book are both correct? Thanks a ton! — Ammar Ahmed, Jul 11 '18 at 19:18
The one in the book is correct. If yours is different it is wrong — kjetil b halvorsen, Jul 11 '18 at 19:34

Sextus Empiricus · Accepted Answer · 2018-07-11T13:22:54.173

2

Is likelihood function always unique?

The likelihood function is as unique as the probability distribution function. Different variables may have different distribution functions, but there is only one distribution function for each variable.

why $F_M(m)$ is not the likelihood function

$F_M(m)$ is the cumulative distribution/density function (CDF).

$$\begin{array}\\ F_M(m) &=& \int_{-\infty}^{m} f(t) \text{d}t\\ &=& Pr(M \leq m) \end{array}$$

The likelihood function is related to the probability density distribution (PDF), not to the cumulative density/distribution function.

$$\mathcal{L}(\theta \vert m) = f_M(m \vert \theta) \neq F_M(m \vert \theta)$$

As in sample 2, we produced the probability that $0.9< X < 4$. Similarly, $F_M(m)$ (can be expressed as $F_M(m,\theta)$ can provide the probability of $X_i\le3.5,i = 1,...,5$ for $m = 3.5.$

The event that $M=m$ does not equal the event that each of the five $X_i\leq m$. (so you are coming up with the likelihood function for a different observation set)

The difference is that at least one of the five inequalities $X_i\leq m$ must be an equality.

$M=m$ implies $X_i\leq m$, but the other way around is not true.

So you get $5\phi(m-\theta)\Phi(m-\theta)^4$ instead of $\Phi(m-\theta)^5$.

The term $5\phi(m-\theta)$ refers to the equality relation, and the second part $\Phi(m-\theta)^4$ relates to the inequality relations.

edited Jul 11 '18 at 13:22

answered Jul 11 '18 at 12:40

Sextus Empiricus

43,080
1
72
161

I completely understood your explanation; however, we can also have 2 observations equal to 3.5 giving rise to $5\phi(m-\theta)^2$ as the equality relation and $\Phi(m-\theta)^3$ as equality relation. In fact, in this case our likelihood should be as follows: $L_3(\theta) = \Phi(m-\theta)^4\phi(m-\theta) + \Phi(m-\theta)^3\phi(m-\theta)^2 + \Phi(m-\theta)^2\phi(m-\theta)^3 + \Phi(m-\theta)^1\phi(m-\theta)^4 + \phi(m-\theta)^5$ This is because the maximum of all 5 observations = 3.5 does not mean that only one observation is equal to 3.5. – Ammar Ahmed Jul 11 '18 at 22:04
@AmmarAhmed you only need one of the five terms being an equality (which indeed you can do in five ways). The situation $X_i=m$ for one of the $i$ and $X_i \leq m$ for the other four covers everything (since there are five different $i$ that you can set equal you get this factor 5). Note that for the other four it says less than *or equal*, so you can have more than only one observation equal to 3.5. – Sextus Empiricus Jul 12 '18 at 05:50
$$\begin{array}{c} ((X_1=m) \wedge (X_2 \leq m) \wedge (X_3 \leq m) \wedge (X_4 \leq m) \wedge (X_5\leq m)) \\ \lor\\ ((X_1 \leq m) \wedge (X_2 = m) \wedge (X_3 \leq m) \wedge (X_4 \leq m) \wedge (X_5\leq m)) \\ \lor\\ ((X_1 \leq m) \wedge (X_2 \leq m) \wedge (X_3 = m) \wedge (X_4 \leq m) \wedge (X_5\leq m)) \\ \lor\\ ((X_1 \leq m) \wedge (X_2 \leq m) \wedge (X_3 \leq m) \wedge (X_4 = m) \wedge (X_5\leq m)) \\ \lor\\ ((X_1 \leq m) \wedge (X_2 \leq m) \wedge (X_3 \leq m) \wedge (X_4 \leq m) \wedge (X_5 = m)) \\ \Leftrightarrow \\ M = m \end{array}$$ – Sextus Empiricus Jul 12 '18 at 06:06

Xi'an · Answer 2 · 2018-07-11T14:22:22.247

1

The likelihood is unique for each of the three experiments, the difference is between the three experiments in that one gets three different summaries of the 5 Normal random variables:

$x^\text{obs}=x_1=2.3$
$x^\text{obs}=\mathbb{I}_{(.9,4)}(x_1)=1$
$x^\text{obs}=\max\{x_1,\ldots,x_5\}=3.5$

The three summaries have different distributions hence the likelihood differs across the three cases.

In the last one, observing that $$X_{(5)}\stackrel{\text{def}}{=}\max\{x_1,\ldots,x_5\}=3.5$$which has density$$5\mathbb{P}_\theta(X_1\le 3.5)^4f_\theta(3.5)$$ is not equivalent to observing that all five Normal realisations are less than $3.5$, which has probability$$\mathbb{P}_\theta(X_1\le 3.5)^5$$ since in the former setting one of these realisations took the value $3.5$.

edited Jul 11 '18 at 14:22

answered Jul 11 '18 at 06:23

Xi'an

90,397
9
157
575

Thank you. I understand that the likelihood will be unique for the three cases as they have different observation set. My question is: Is it possible to get two likelihood functions for Sample 3. I understand that the ${\text{max}} \{x_1, x_2,...,x_5\} = 3.5$ which means at least one of the observation in Sample 3 is equal to $3.5$. But the authors also exploited the independence assumption between the 5 Normal realizations while calculating the second and third step of $F_M(m)$. – Ammar Ahmed Jul 11 '18 at 09:06
No, there is a single likelihood function in Case #3. – Xi'an Jul 11 '18 at 10:04

Is likelihood function always unique?

2 Answers2