4

I have a mixture data. I used EM to estimate the model parameters. When I calculate the log likelihood function, I found that the values is positive. So, is that ok. Can the log likelihood function be positive?

To be more clear:

$ll = \sum_{n=1}^{N}\log(\sum_{k=1}^{K} \pi_{k} f(x_n;\theta_{k}))$

Alice
  • 225
  • 2
  • 18
  • To my understanding, likelihood function is usually a product of probabilities and log-likelihood is a sum of logs. Because probabilities are less than 1, theirs logs should be less than 0. And because you sum those logs, you should get a negative number. – Celdor Dec 21 '17 at 10:32
  • 5
    The comment above is wrong. The likelihood function of continuous parameters (such as your case, I guess) is based on probability *densities*, which can be greater than 1 depending on the domain and the density, so the log likelihood can occasionally be positive. Still, you should check that everything else is correct. – lacerbi Dec 21 '17 at 10:35
  • yes, my density is continuous. ' – Alice Dec 21 '17 at 10:37

1 Answers1

12

Simply (just summarizing the comments):

  • when using probabilities (discrete outcome), the log likelihood is the sum of logs of probabilities all smaller than 1, thus it is always negative
  • when using probability densities (continuous outcome), the log likelihood is the sum of logs of densities that can be greater than 1, thus is can be positive.
Benoit Sanchez
  • 7,377
  • 21
  • 43
  • If all the discrete probability is in a single outcome, log-likelihood would be 0 rather than negative. – Glen_b Sep 03 '20 at 05:41