PDF of a degenerate multivariable gaussian distribution

Question

The pdf of a multivariate gaussian distribution with mean $\mu\in\mathbb{R}^n$ and variance $\Sigma\in\mathbb{S}^n_{++}$ ($\Sigma$ is positive definite) is given as $$ p(x) = \frac{1}{\sqrt{|\Sigma|(2\pi)^n}} \operatorname{exp}\left( (x-\mu)^\top \Sigma^{-1} (x-\mu) \right).$$

Now, consider the case of a degenerate multivariate gaussian such that $\Sigma \in \mathbb{S}^n_{+}$, i.e. one or more eigenvalues of $\Sigma$ are zero. For the univariate case ($\Sigma$ is a scalar), it has been established in this question if the variance is 0 then the pdf does not exist. I am interested in the implications for the multivariate case.

Suppose, for simplicity, $n=3$, $\mu = [1,2,3]^\top$, and $$ \Sigma = \begin{bmatrix} 1&0&0\\0&0&0\\0&0&1 \end{bmatrix}.$$ Furthermore, let $e_1,e_2,e_3$ be the canonical bases in $\mathbb{R}^3$. It is evident that the multivariate distribution $\mathcal{N}(\mu,\Sigma)$ for $x\in\operatorname{span}\{e_2\}$ stipulates a deterministic value of $x = [0,2,0]^\top$. However, for $x\in\operatorname{span}\{ e_1,e_3 \}$, $\mathcal{N}(\mu,\Sigma)$ specifies a non-degenrate distribution. My question is, is there any way to 'write down' the pdf for $x\in\operatorname{span}\{ e_1,e_3 \}$?

Answer 1

When $\Sigma$ is singular, you don't define a density over random vectors. Perfect correlation implies the $n$-vectors lie in a $n$-plane, which has volume 0, i.e. its Lebesgue measure $\mathcal{L}^n$ is 0, so it's not a density on $\mathbb{R}^n$. This is true of all densities, not only multivariate normal densities.

For simplicity, suppose we have a bivariate normal distribution and the covariance matrix has perfect correlation between the two vectors. All of the probability mass falls in a line on the plane. A line has 0 area. However, by the Kolmogorov axioms, all probability distributions have total measure 1. Due to this contradiction, this distribution is not a probability density.

Sometimes, software will cheat and ignore this fact. Some discussion: Why Multivariate Distribution with Covariance Matrix with Rank 1 Works?