3

For continuous random variables, evaluating p(x) for a specific value of x is always 0 as show here, here and here. So when we're calculating the likelihood for a random variable X that is represented by a continuous distribution we have:

$$ P( \mid X) = P(x_1, x_2 ..., x_n \mid ) = \prod\limits_{i=1}^n = p(x_i \mid ) $$

There are two ways I consider thinking about.

1) The first is the intuitive definition of likelihood, e.g. how probable is a set of n draws for random variable X. Intuitively the likelihood seems like it calculates the probability of n draws occuring which, for any continuous random variable would be zero. This does not make much sense to me.

2) If we consider a gaussian distribution of µ = 0 and = 1 then p(0) = 0.4. This number doesn't represent anything concrete, but when used in the likelihood calculation, can serve as a relative reference for which model parameters fit our data better. In this scenario, likelihood is a bit of a misnomer as it isn't actually the probability of getting our observed data, but is a number that only draws meaning in reference to other likelihood calculations.

This answer makes some really good points about interpreting a density function as density and not probability, which suggests that likelihood is indeed a bit of a misnomer. It's also somewhat confusing that the likelihood is expressed in terms of p(x), which I read as probability of x occurring.

jvans
  • 131
  • 2
  • "which suggests that likelihood is indeed a bit of a misnomer" not really. densities, probabilities and likelihoods are all different things. This question is probably a duplicate. – Taylor May 01 '18 at 15:33
  • 3
    "Evaluating $p(x)$ for a specific value of $x$ is always $0$"... only if you define $p(x)$ to be the *probability* of $x$. In regular usage, it's the *probability density* of $x$, i.e., the derivative of the cumulative density function at $x$, and is not always $0$. "Likelihood" is not a misnomer, it refers to a function for which the parameter(s) of $p$ are the arguments and the observed data $x$ are fixed quantities, which is conceptually quite different than the pdf, for which $x$ is the argument and the parameters are... well... parameters. – jbowman May 01 '18 at 16:03
  • @jbowman defining things that way makes it much clearer. Is it fair to say that the likelihood L( | X) also cannot be interpreted as probability since the function p(x) evaluates to a density? How should you interpret the value of a likelihood function other than the MLE represents the most probable parameters for a given model? – jvans May 01 '18 at 19:29
  • 1
    Yes, it is fair to say that about the likelihood function. The value of the likelihood function (well the log of the likelihood function) is also useful for constructing asymptotically correct confidence intervals, tests, etc., all of which are based on varying the value of $\theta$ while holding the data $x$ fixed and seeing how much the value of the likelihood function changes as a result. – jbowman May 01 '18 at 19:53

0 Answers0