1

Given a univariate Gaussian with mean $\mu_1$ and variance $\sigma_1$ and a second univariate Gaussian with $\mu_2, \sigma_2$. Compare the two using the likelihood in order to find out how similar they are.

This task was given to me by my supervisor. He said "Google it, probably top hit". Now after two hours I didn't found it. I know how to compute the likelihood for some data that it was generated by some Gaussian, but not how to compare two Gaussians and he explicitly said that I should use mean and variance for both.

Edit: Not the Kullback-Leibler divergence

MLmuchAmaze
  • 31
  • 5
  • 1
    I am not sure it is well-defined. 'Similar' in what sense? one could argue you can take the two cdfs, let's say $F_1=\Phi((x-\mu_1)/\sigma_1)$ and $F_2=\Phi((x-\mu_2)/\sigma_2)$, where $\Phi$ is the standard normal cdf, and calculate $||F_1-F_2||$. But then you need to think about the norm in which you calculate it (L2 norm, TV-norm, max-norm, etc.), and also whether you want to calculate it on the cdf or on the pdf, etc. – yoki Jul 30 '15 at 11:32
  • 1
    The likelihood is a function of the parameters *contingent upon having data.* (See, for instance, our thread at http://stats.stackexchange.com/questions/2641 .) In the absence of any data, suggesting that Gaussians (or any other distributions) could be compared on the basis of a "likelihood" alone is therefore meaningless. – whuber Jul 30 '15 at 13:16

1 Answers1

2

If you want compare the two probability density functions, your prof is probably talking about measuring the Kullback Leibler (KL) divergence, which is one good way to measure the similarity between two probability distributions. The KL divergence between gaussians has a simple formula which you can find on your own and then check with a simple google search

There are other measures of distance between probability distributions which all have different properties. The total-variation distance is a very useful one, but harder to compute in general (also has a simple formula for gaussians), the wassertein distances (there are several), and many more

Guillaume Dehaene
  • 2,137
  • 1
  • 10
  • 18