Does the correlation matrix always have a smaller condition number than the covariance matrix?

Question

In my experience and the experience of others (for example: https://stats.stackexchange.com/a/287737/193216), a covariance matrix

$\textrm{COV}=\langle {\bf x}_t{\bf x}_t'\rangle$

has a larger condition number $\kappa$ than the corresponding correlation matrix with elements

$\textrm{COR}_{ij} := \frac{\textrm{COV}_{ij}}{\sqrt{\textrm{COV}_{ii}\textrm{COV}_{jj}}}$.

Is it possible to show that this is always the case? Or maybe at least if we assume a multivariate Gaussian distribution for $\bf x$?

The general definition of condition number [depends on the matrix norm you use.](https://en.wikipedia.org/wiki/Condition_number#Matrices) Which one do you have in mind? BTW, this is purely a question of linear algebra: Normal distributions are irrelevant. — whuber, Aug 05 '21 at 22:13
I was thinking of using singular values to define condition numbers (which I think corresponds to using the L2 norm). However, results for any definition of condition number would be interesting to me — Alice Schwarze, Aug 05 '21 at 22:16
With two variables this is always true. Otherwise, consider this covariance matrix for instance: $$\pmatrix{0.27 &-0.16& 0.12\\ -0.16 &0.19& -0.23\\ 0.12 &-0.23& 0.37}$$ — whuber, Aug 05 '21 at 22:19
@whuber: When I compute the condition number via `numpy.linalg.cond` I get 56.86 for the covariance matrix you suggested and 48.80 for the corresponding correlation matrix. So the correlation matrix has a smaller condition number than the covariance matrix. How did you design this example? — Alice Schwarze, Aug 05 '21 at 22:30
*What* condition number is that software computing, then? If you would let us know, then we can provide appropriate answers. In the meantime, because you mentioned using the singular values, why not compute those and see what they tell you? — whuber, Aug 06 '21 at 12:43
It is computing the condition number based on the L2 norm. I get the same result when I use singular values to compute the condition numbers myself. What did you expect me to discover? — Alice Schwarze, Aug 06 '21 at 15:09
Sorry; I erred when rounding the matrix entries for convenient display. Let's try again. The matrix $$\pmatrix{30&-26&-30\\-26&57&56\\-30&56&61}$$ has eigenvalues ranging from $2.63$ to $130.7$ for a condition number of $130.7/2.63=49.7$ The corresponding correlation matrix has eigenvalues ranging from $0.0449$ to $2.529$ for a condition number of $56.3,$ which is larger than $49.7.$ — whuber, Aug 06 '21 at 15:30

Does the correlation matrix always have a smaller condition number than the covariance matrix?

0 Answers0