2

I was going through Chuong B Do' notes on The Multivariate Gaussian Distribution. The multivariate Gaussian distribution according to it, is given as follows:

$p(x; \mu, \Sigma) = \frac{1}{(2\pi)^{n/2}|\Sigma|^{1/2}} \exp(-\frac{1}{2}(x − \mu)^{T}\Sigma^{−1}(x − \mu))$

While attempting to come up to this conclusion myself I got stuck at,

$P(x_1,x_2) = P(x_1)P(x_2) = \frac{1}{2\pi\sigma^{2}}\exp(-\frac{1}{2\sigma^{2}}((x_1 - \mu_{X_1})^{2} + (x_2 - \mu_{X_2})^{2}))$

How can I relate the two equation or rather generalize the second one to reach to first?

kjetil b halvorsen
  • 63,378
  • 26
  • 142
  • 467
  • 2
    What principles are you using to "come up to" this result, given this is a *definition* and not a conclusion? There are many principles that could be used and it's interesting to read about them: see https://stats.stackexchange.com/questions/4364 and focus on the answers that refer to multivariate distributions. – whuber Jan 17 '20 at 14:28

1 Answers1

1

There are errors in your second expression in particular terms involving $\sigma$.

Assuming $X_1, X_2$ are independent normal with variance $\sigma^2$.

Here $\Sigma = \begin{bmatrix} \sigma^2 & 0 \\ 0 & \sigma^2\ \end{bmatrix}=\sigma^2I_2$, then $|\Sigma|=\sigma^4$ and $|\Sigma|^\frac12=\sigma^2$

\begin{align} P(x_1, x_2) &= \prod_{i=1}^2 P(x_i) \\ &=\prod_{i=1}^2 \frac1{\sqrt{2\pi}\sigma} \exp\left(-\frac1{2\sigma^2} (x_i-\mu_i)^2\right) \\ &= \frac1{2\pi\sigma^2} \exp\left(-\frac12\sum_{i=1}^2\frac1{\sigma^2} (x_i-\mu_i)^2\right) \\ &= \frac1{2\pi\sigma^2} \exp\left(-\frac12\left(\frac1{\sigma^2} (x_1-\mu_1)^2+\frac1{\sigma^2} (x_2-\mu_2)^2\right)\right) \\ &= \frac1{2\pi\sigma^2} \exp\left(-\frac12\begin{bmatrix} x_1-\mu_1, & x_2-\mu_2\end{bmatrix}\begin{bmatrix} \frac1{\sigma^2} & 0 \\ 0& \frac1{\sigma^2}\end{bmatrix}\begin{bmatrix} x_1-\mu_1 \\ x_2-\mu_2\end{bmatrix}\right) \\ &= \frac1{(2\pi)^{2/2}\sigma^2}\exp \left(-\frac12(x-\mu)^T\Sigma^{-1} (x-\mu) \right) \end{align}

Siong Thye Goh
  • 6,431
  • 3
  • 17
  • 28
  • Thanks a lot for the response. My question is how were you able to map from univariate to multivariate? As in, how was the $ \Sigma^{-1} $ get introduced in that particular location between $(x - \mu)^{T}$ and $(x - \mu)$ ? –  Jan 17 '20 at 14:19
  • 1
    I added a few more lines of details. Typically we have to be familiar with the expansion of $x^TAx$. Notice that there is no mixed terms in the form of $(x_1-\mu_1) (x_2-\mu_2)$, hence the off diagonal parts are $0$ and the diagonal entries are given by the coefficeint of the square terms. – Siong Thye Goh Jan 17 '20 at 14:52