2

If a covariance matrix is non-singular, does this implies that correlation matrix is also non-singular.

My guess is it depends on mean vector in $K_{X} = R_{X} - m_X.{m_X}^H$

Not sure though.

kjetil b halvorsen
  • 63,378
  • 26
  • 142
  • 467
urvah shabbir
  • 233
  • 3
  • 7
  • 2
    The answer to the question in the first sentence is Yes; covariances and correlation coefficients do not depend on the values of the means. – Dilip Sarwate May 22 '16 at 14:56
  • 1
    @DilipSarwate. If covariances and correlation coefficients do not depend on the values of the means, does this not imply that singularity of either of them is independent of the other? Therefore giving the answer of No for the given question. i.e I can have a singular Kx but non-singular Rx? – urvah shabbir May 22 '16 at 17:06
  • 1
    $R_X$ is _not_ called the correlation matrix in statistical circles. The standard definition of correlation matrix is $\rho_X = \sigma^{-1} K_X \sigma^{-1}$ where $\sigma^{-1} = \operatorname{diag}(\sigma_1^{-1},\sigma_2^{-1},\cdots,\sigma_n^{-1})$, that is, we get $\rho_X$ from $K_X$ by dividing each $i$-$j$-th entry of $K_X$ by $\sigma_i\sigma_j$. – Dilip Sarwate May 22 '16 at 23:10

1 Answers1

2

If $X$ is a random vector with mean (vector) $\mu$, then the covariance matrix is given by $\DeclareMathOperator{\E}{\mathbb{E}} \Sigma = \E (X-\mu)(X-\mu)^T$. The variances is on the diagonal matrix $D= \operatorname{Diag}(\Sigma)$. If $\Sigma$ is nonsingular then so is $D$, that is, all the variances are positive.

Now the correlation matrix can be written $$ R= D^{-1/2}\Sigma D^{-1/2} $$ and it follows that $R$ and $\Sigma$ is jointly singular or non-singular.

To the extra question in comment: If the covariance (and then correlation matrix$^\dagger$) is singular, then there is some linear subspace (of dimension equal to the range of the covariance matrix) such that (with probability 1) the random variables $X$ "lives" in that subspace. So there is some deterministic structure, but still, $X$ are random variables. Even a constant is a random variable! random variables are just functions, see What is meant by a "random variable"?. Just as in calculus, constants can be seen as functions. Some details here.

$^\dagger$If there is zeros on the diagonal of $D$, then replace the inverse by the Moore-Penrose generalized inverse.

kjetil b halvorsen
  • 63,378
  • 26
  • 142
  • 467