How do you handle probability zero for states due to rounding error in an HMM?

Question

Say I have a hidden Markov model (HMM), and due to the emission distribution and our observations, we get probability 0 for certain states due to computer precision, which causes singularities (divide by 0) when using the forward algorithm.

What are some methods to handle this? One method is to use a distribution with fatter tails, i.e. replace a Gaussian distribution with a t-distribution. Are there others?

It may be helpful to provide some math or code showing where the division by zero occurs? — Matthew Gunn, Sep 02 '16 at 05:33

score 1 · Answer 1 · answered Jun 30 '17 at 12:20

1

I can think of two ways to avoid numerical underflow:

1) Use the scaling method described in Rabiner's paper (Rabiner, L. R. (1989). A tutorial on hidden Markov models and selected applications in speech recognition. Proceedings of the IEEE, 77(2), 257-286.)

2) Conduct your computations in log space. (use the logsumexp trick). I don't have a reference right now, but I remember there was a paper on this.

answered Jun 30 '17 at 12:20

discretetimeisnice

427
3
12

Sure it should be a log space. I made the same mistake once, log trick helps. – German Demidov Nov 11 '19 at 19:06

How do you handle probability zero for states due to rounding error in an HMM?

1 Answers1