Probability of a measurement with uncertainty covariance being generated by a normal distribution

Question

I have the following situation:

A set of Kalman filters with the same model, each with its own current estimated state and state covariance.
A measurement with a covariance matrix expressing its uncertainty.

And the following problem: I want to decide to which KF to assign the measurement based on the likelihood of the measurement being generated by the random distribution, represented by the current state and state covariance of the KF.

I know about the Mahalanobis distance, but that doesn't take into account uncertainty covariance of the measurement, only covariance of the random distribution. I also know about the Bhattacharyya distance, but if I understand it correctly, that measures the similarity of two random distributions, and I'm not sure if that's the same thing as what I want to do.

Is there a way to calculate the likelihood of a measurement with a known uncertainty covariance matrix being generated by a random distribution? If possible, can you suggest a better metric to decide to which KF to assign the measurement?

Thank you.

score 0 · Answer 1 · answered Oct 18 '18 at 18:47

0

You might want to consider the Kullback-Leibler (KL) "distance" metric. It is often used to compare distributions. So if the signal has come from one of KFs, one would expect the it's KL distance to be the least among all.

answered Oct 18 '18 at 18:47

saipk

95
1
8

Your suggestion sounds very close to what I'm looking for (from the wikipedia description). But I'm not sure what is the difference from the Bhattacharyya metric - they seem very similar. Could you please explain the difference? – Matouš Vrba Oct 18 '18 at 19:15
Here's an interesting link - https://stats.stackexchange.com/questions/130432/differences-between-bhattacharyya-distance-and-kl-divergence From the inequality between KL and Bhattacharya distance, it seems KL might be easier to work with. Also, it has some useful properties. For example, convergence in KL implies convergence in distribution etc. – saipk Oct 19 '18 at 03:46
I guess noone else wants to contribute and your answer is technically correct (I was just hoping to get a few more different suggestions :)), so I'll accept it. – Matouš Vrba Oct 19 '18 at 14:33
I am sorry, but I have found out that your answer is incorrect (see my new answer). I would still like to upvote it because you set me in the correct direction, but unfortunately I can't (too low SE score). – Matouš Vrba Dec 12 '18 at 11:56

score 0 · Accepted Answer · answered Dec 12 '18 at 11:55

After some more research regarding this topic, I have found the correct answer. It turns out that what I was trying to describe and implement is a variant of the Multiple Hypothesis Tracker (MHT) [1, 2].

Answer to my question:

The correct way to calculate likelihood of a measurement $\mathbf{z}_k$ at time step $k$ given a set of previous measurements $\textbf{Z}_{k-1} = \{ \textbf{z}_{0}, \ldots , \textbf{z}_{k-1} \}$ is (taken from wikipedia [1])

$$ p(\textbf{z}) = \prod^T_{k=0}\left( p(\textbf{z}_k), | p(\textbf{z}_{0}), \ldots , p(\textbf{z}_{k-1}) \right) = \prod^T_{k=0} \mathcal{N}\left( \textbf{z}_k; \textbf{H}_k \hat{\textbf{x}}_{k|k-1}, \textbf{S}_k \right), $$

where $\hat{\textbf{x}}_{k|k-1}$ is the predicted state vector of the Kalman filter and $\textbf{H}_k$ is the matrix, mapping states of the KF to measurements (part of the system model). $\mathbf{S}_k$ is the innovation covariance, which is calculated as part of the Kalman filter update step.

This can be calculated iteratively like the Kalman filter, but it is not very numerically stable, which is why a log-likelihood $l_k = \log{p(\mathbf{z}_k)}$ is usually used. The iterative update equation for the log-likelihood is (again from wikipedia [1])

$$ l_k = l_{k-1} - \frac{1}{2}\left( \tilde{\mathbf{y}}_k^T \mathbf{S}_k^{-1} \tilde{\mathbf{y}}_k + \log{|\mathbf{S}_k|} + d_y\log{2\pi} \right), $$

where $\tilde{\mathbf{y}}_k$ is the innovation vector, which is calculated as part of the KF update step, and $d_y$ is number of dimensions of the measurement.

Notes regarding using other metrics:

The metrics for measuring a distance of distributions (such as the Mahalanobis distance or Kullback-Leibler divergence) are not well suitable for this problem, since they usually describe the 'similarity' of two random distributions, whereas in this case it is desired to express the likelihood of a measurement being generated by a stochastic system. They do not take into account the measurement model, but only the two random distributions.

Probability of a measurement with uncertainty covariance being generated by a normal distribution

2 Answers2

Answer to my question:

Notes regarding using other metrics:

Further reading