1

I have a score of some feature, $F_1$ and $F_2$ where this score is the logarithm of probability.

This score is very low and very sparse, for example i have:

$F_1$ with score $-800$

$F_2$ with score $-3000$

I need to get the probability with the exponential function and make the sum of probability like this:

$e^{-800} + e^{-3000}$

But if I make the exponential of this number i get 0 for the underflow problem. With log-sum-exp trick i can do:

$\log(e^{-800}+e^{-3000})$

$= \log(e^{-800+800}+e^{-3000+800})-800$

$= \log(e^{0}+e^{-2200})-800$

But now i have $e^{-2200}$ that is too small.

What's wrong? how I can make my $e^{-800} + e^{-3000}$ sum ?

Ben
  • 91,027
  • 3
  • 150
  • 376
Neptune
  • 579
  • 3
  • 15
  • 5
    Please explain why underflow is any problem at all. To a very high precision, it is indeed the case that $\exp(-800) + \exp(-3000) = \exp(-800)$, as your analysis shows. Do you really need $955+$ decimal digits of precision in your numbers? – whuber Jul 09 '14 at 16:20
  • Yes because my probability is like 1E-95, if i decrease the precision i get 0. – Neptune Jul 09 '14 at 16:28
  • 3
    I don't think you follow my question: you are asking how to compute $\log(1 + \exp(-2200))$. This value is approximately $10^{-955}$. Do you need the kind of precision that distinguishes this from $0$? If so, why? – whuber Jul 09 '14 at 17:15
  • 1
    You are right, I need to search the value with maximum probability, then I can approximate to zero all the probability away from the maximum and take only the value near the maximum. – Neptune Jul 10 '14 at 18:19

0 Answers0