8

The Wikipedia entry on Mahalanobis Distance contains this note:

Another intuitive description of Mahalanobis distance is that it is square root of the negative log likelihood. That is, the exponential of the negative square of the Mahalanobis distance will give you the likelihood of your data point belonging to (a presumed normal) distribution of the sample points you already have.

How/where is this shown? What does it mean (I guess it's not so intuitive...)?

Jacob
  • 235
  • 1
  • 2
  • 9
Adi Shavit
  • 255
  • 1
  • 10
  • 1
    This is the topic from Bayes classification. You can find a number of good answers here covering it. But let me mention this time just [one of mine](http://stats.stackexchange.com/a/31384/3277). The "likelihood" of a class given the data point, `k|x`, there is expressed pdf function, where the exponent of the squared Mahalanobis d you may notice in the numerator. – ttnphns May 12 '14 at 18:01
  • 2
    The quotation is only partly correct: the log likelihood is given by *negative one-half* the square root of the Mahalanobis distance *plus* one-half the log of the determinant of the inverse covariance matrix, $\det(S^{-1})$, *minus* $(n/2)\log(2\pi)$ (in $n$ dimensions). For the purpose of comparing likelihoods with the same covariance matrix the additive factors can be dropped, but the initial factor of $1/2$ is crucial for implementing and interpreting certain tests. – whuber May 12 '14 at 18:17
  • @whuber, ttnphns: Can you write out the exact equations as an answer? What does it mean? – Adi Shavit May 13 '14 at 09:06
  • @whuber: I just found this equation on Wikipedia: http://en.wikipedia.org/wiki/Multivariate_normal_distribution#Likelihood_function. It is a bit different from your comment. Which is correct? – Adi Shavit May 13 '14 at 19:15
  • 1
    How do the two differ? It looks the same to me. :-) It might help to know that $\Sigma=S^{-1}$, which flips the sign of the logarithm of the determinant in the formula. – whuber May 13 '14 at 19:18
  • If the Mahalanobis distance can be defined in terms of the log likelihood of the normal distribution, can we derive other distances for other distributions by just plugging in their respective log-likelihood functions? Would there be a name for this possible class of distances? I guess I will post a new question for this. – Cesar Sep 23 '15 at 10:41

1 Answers1

7

For completeness, I'll answer myself.
I eventually found the equation here, and as @whuber writes in his comment:

Define:

  • $k$: the multivariate dimension
  • $\mu$: the multivariate mean (a $k$-dimensional vector);
  • $\Sigma$: the $k\times k$ covariance matrix;

Then:

  • The squared Mahalanobis Distance is: $D^2=(x-\mu)^T\Sigma^{-1}(x-\mu)$
  • The log-likelihood is: $ln(L)=-\frac12ln(|\Sigma|)-\frac12(x-\mu)^T\Sigma^{-1}(x-\mu)-\frac k2 ln(2\pi)$
    or: $ln(L)=-\frac12ln(|\Sigma|)-\frac12D^2-\frac k2 ln(2\pi)$

Thus, define $c\equiv\frac12(ln(|\Sigma|) + k\cdot ln(2\pi))$ then:

  • $ln(L)=-\frac12D^2-c$ and;
  • $D=\sqrt{-2(ln(L)+c)} $
Adi Shavit
  • 255
  • 1
  • 10