0

In a single dimension Gaussian, the variance $\sigma$ denotes the expected value of the squared deviation from the mean $\mu$.

I am trying to understand why in the multivariate case of modeling variable $\mathbf{x}$ we end up having a matrix $\Sigma^{-1}$. Why not instead of a vector which in each dimension shows the variance of the input variable $\mathbf{x}$.

From Wikipedia the 2d Gaussian function is represented as:

$f(x,y) = A \exp\left(- \left(\frac{(x-x_o)^2}{2\sigma_X^2} + \frac{(y-y_o)^2}{2\sigma_Y^2} \right)\right)$

Why not use a form like that for the multivariate Gaussian with $\mathbf{\sigma} = [\sigma_{X} \ \sigma_{Y}]^{T}$? Given than my vector $\mathbf{x} = [x \ y]^{T}$.

How this is interpreted in the following example:

enter image description here

Jose Ramon
  • 75
  • 1
  • 12

1 Answers1

3

The probability density function for Bivariate Gaussian is

$$ f(x,y) = \frac{1}{2 \pi \sigma_X \sigma_Y \sqrt{1-\rho^2}} \mathrm{e}^{ -\frac{1}{2(1-\rho^2)}\left[ \left(\frac{x-\mu_X}{\sigma_X}\right)^2 - 2\rho\left(\frac{x-\mu_X}{\sigma_X}\right)\left(\frac{y-\mu_Y}{\sigma_Y}\right) + \left(\frac{y-\mu_Y}{\sigma_Y}\right)^2 \right] } $$

notice that apart from $\mu_X,\mu_Y$ and $\sigma_X,\sigma_Y$, it has the $\rho$ parameter for the correlation between the $X$ and $Y$ variables. If they are uncorrelated, i.e. $\rho=0$, the pdf reduced to what you described.

The same applies to multivariate normal, you could use a covariance matrix that is all-zeros, with the $\sigma$'s on the diagonal. In such a case, the individual variables are assumed to be uncorrelated.

Tim
  • 108,699
  • 20
  • 212
  • 390
  • IIs there a way to explain this visually? How the correlation affects the Gaussian? – Jose Ramon Jun 17 '21 at 10:05
  • @JoseRamon are you familiar with [correlation](https://en.wikipedia.org/wiki/Correlation_and_dependence)? – Tim Jun 17 '21 at 10:12
  • yes, I am trying to grasp the reason that why make use of it in our case. – Jose Ramon Jun 17 '21 at 10:14
  • @JoseRamon to have a distribution for variables that are Gaussian and correlated. If you don't need that, you don't need the distribution. – Tim Jun 17 '21 at 10:16
  • I understand what is the covariance matrix, but I am not sure why it is necessary and what it represents in the multivariate case. – Jose Ramon Jun 17 '21 at 10:18
  • @JoseRamon it represents the covariance matrix between the variables that together form the multivariate distribution. – Tim Jun 17 '21 at 10:20
  • that does not really answer my question. Visually in the example, I have posted, why the variance in the two dims $\sigma_{X}$ and $\sigma_{Y}$ isn't enough to be used? – Jose Ramon Jun 17 '21 at 10:33
  • @JoseRamon because it tells you nothing about [covariance](https://en.wikipedia.org/wiki/Covariance) between the variables. – Tim Jun 17 '21 at 10:36