Does every semi-positive definite matrix correspond to a covariance matrix?

Question

It is well-known that a covariance matrix must be semi-positive definite, however, is the converse true?

That is, does every semi-positive definite matrix correspond to a covariance matrix?

Answer 1

Going by the definitions of PD and PSD here, yes, I think so, since we can do this by construction. I'll assume for a slightly simpler argument that you mean for matrices with real elements, but with appropriate changes it would extend to complex matrices.

Let $A$ be some real PSD matrix; from the definition I linked to, it will be symmetric. Any real symmetric positive definite matrix $A$ can be written as $A = LL^T$. This can be done by $L=Q\sqrt{D}Q^T$ if $A=QDQ^T$ with orthogonal $Q$ and diagonal $D$ and $\sqrt{D}$ as matrix of component wise square roots of $D$. Thus, it needn't be full rank.

Let $Z$ be some vector random variable, of the appropriate dimension, with covariance matrix $I$ (which is easy to create).

Then $LZ$ has covariance matrix $A$.

[At least that's in theory. In practice there'd be various numerical issues to deal with if you wanted good results, and - because of the usual problems with floating point calculation - you'd only approximately get what you need; that is, the population variance of a computed $LZ$ usually wouldn't be exactly $A$. But this sort of thing is always an issue when we come to actually calculate things]