3

I read in the likelihood function Wikipedia article that:

This (the likelihood function) is not the same as the probability that those parameters are the right ones, given the observed sample. Attempting to interpret the likelihood of a hypothesis given observed evidence as the probability of the hypothesis is a common error, with potentially disastrous consequences in medicine, engineering or jurisprudence. See prosecutor's fallacy for an example of this.

My question is, what does it mean that the likelihood function is not the same as the probability that the parameters are the "right ones"? In this case, what IS the probability of the parameters being the right ones given an observed sample? Is it the posterior probability? Thanks.

user321627
  • 2,511
  • 3
  • 13
  • 49

2 Answers2

6

The likelihood is not restricted to be between 0 and 1, so it could hardly be a probability of anything: notice that in order to compute the likelihood associated to a sample of a continuous variable, you multiply the densities which can have any (non-negative) value.

Aside from that, asking for the probability of parameters being such and such only makes sense in a Bayesian setting. In a classical setting parameters are fixed rather than random variables, so it makes no sense to assign them any probability.

F. Tusell
  • 7,733
  • 19
  • 34
5

Call $\theta$ the parameter and $X$ the observed variable.

The likelihood is $P(X|\theta)$ : the probability to observe something, depending on (given) the parameter.

not the same as the probability that those parameters are the right ones

This instead is called the posterior and is formally $P(\theta|X)$ : the probability that the parameter is something, depending on (given) the observation.

Bayesian inference says : $$P(\theta|X)\propto P(X|\theta)P(\theta)$$

When the prior $P(\theta)$ is uniform then the likelihood and the posterior are proportional to each other and can be meaningfully identified. Note that the posterior has no meaning in the frequetist framework.

Yet, it is a reasoning we sometimes unwillingly do, simply because :

  • we are not aware of reversing the conditioning in our heads
  • there is an intuition this "tends to be rather true" practically

This intuition is rather justified in many simple practical situations. You can see my answer for confidence intervals : What, precisely, is a confidence interval?. But it is false generally.

Benoit Sanchez
  • 7,377
  • 21
  • 43