1

I'm looking at the bivariate relationships among a set of time-series data that have the same units. I have computed the covariance matrix and am working on interpreting it. My hunch is that it would make sense to take the square root of the off-diagonals, yielding a result in the same units as the data, just like converting variance to standard deviation (on the diagonal).

Does the square root of covariance have a name? And is this approach reasonable?

Thank you.

edit: I am interested in the correlations between pairs of channels and I am trying to get a metric in the most physically interpretable units.

Neuromancer
  • 53
  • 5
  • It looks to me like you are treating the data as multivariate when you consider the covariance matrix.but it ignores the time dependence. It is the autocovariance and autocorrelation functions that I think you should be dealing with. It doesn't make sense to me that you would take square roots of off diagonal elements of a covariance matrix. It is unclear to me what you want to do. – Michael R. Chernick Dec 06 '16 at 21:09
  • 1
    There is no reason to suppose a square root of a covariance makes sense--after all, covariances can be negative. Indeed, [every covariance is a difference of variances.](http://stats.stackexchange.com/a/142472). Variances are additive, so subtracting two of them does make sense. These nice mathematical properties suggest that the most *physically interpretable units* indeed are the variances and covariances themselves, rather than anything derived from them. – whuber Dec 06 '16 at 23:10

2 Answers2

3

Covariance matrices are positive semi-definite or positive definite (bundle both terms under "positive") A matrix $V$ is positive if and only if there exists a positive matrix $C$ such that

$$V = CC$$

If $V$ is positive definite, so will be $C$. $C$ is unique.

$C$ is called "the square root of $V$", and it can be considered as the standard deviation analog in the multivariate setting.

SOURCE
Bhatia, R. (2009). Positive definite matrices. Princeton university press. Chapter 1, page 2, result (v).

Alecos Papadopoulos
  • 52,923
  • 5
  • 131
  • 241
  • I'm not sure you're quoting this theorem correctly: could you indicate your source? – whuber Dec 06 '16 at 22:10
  • @whuber Good idea. Done. – Alecos Papadopoulos Dec 06 '16 at 22:21
  • Could you indicate whether Bhatia's matrices must be symmetric or not? (In many applications the matrix represents a quadratic form and therefore automatically is symmetric). – whuber Dec 06 '16 at 22:26
  • Let's take the simplest case of $$V=\pmatrix{1&\rho\\ \rho&1}$$ for which $$C=\pmatrix{a&b\\ b&a}$$ with $$2a=\sqrt{1+\rho}+\sqrt{1-\rho},\ 2b=\sqrt{1+\rho}-\sqrt{1-\rho}.$$ Those terms are interesting, but they aren't simple or conventional functions of the correlation coefficient $\rho$ and the *marginal* standard deviations (which are both $1$). How are we supposed to interpret the components of $C$, then? – whuber Dec 06 '16 at 22:35
  • @whuber He deals with matrices $A$ such us the inner product between $x$ and $Ax$ is non-negative for all vectors $x$, and strictly positive for all non-zero vectors $x$. – Alecos Papadopoulos Dec 06 '16 at 22:36
  • Thank you--that's what I was supposing. But regardless, I am curious about the *meaning* of your proposal. How are we supposed to relate this $C$ to the usual ways of understanding and interpreting variances, covariances, and standard deviations? – whuber Dec 06 '16 at 22:39
  • @whuber Well, what if we look at a random vector and see a two-dimensional single object, and not a two-dimensional container into which we have placed two uni-dimensional rv's? Still its two dimensions can be related... I would start like that somehow to try to understand $C$ – Alecos Papadopoulos Dec 06 '16 at 22:53
  • OK, that's a good start. But what would be the advantage of using $C$ compared to the (equivalent) approach of studying the PCA? After all, $V=QD^2Q^\prime$ for an othogonal $Q$ and non-negative diagonal $D$, whence $C=QDQ^\prime$. What is the *statistical* meaning of the entries of $C$? – whuber Dec 06 '16 at 23:04
  • related: https://en.m.wikipedia.org/wiki/Cholesky_decomposition It looks like these decompositions are mostly for numerical computations rather than interpretable counterpart of standard deviation. – Paul Dec 07 '16 at 18:44
1

I think it might be simpler than all that. In the case of normally distributed data, the standard deviation is the square root of the 2nd central moment of a distribution. The 2nd central moment is sometimes called the variance. Thus, for nomral variates, the variance is the square of the standard deviation.

Covariances are the central 2nd moments of multivariate data. I don't think I need to write the formula. Since, by definition, the covariance matrix of the multivariate normal distribution is positive definite, you can take the "square root" as described by @Alecos. This would be your analog of a standard deviation.

To get a bit more practical, one of the main uses of the s.d. is to use it as a measure for a range under which some percent of the data lie. To accomplish this in the multivariate case, you could use principal component analysis (eigenvalue decomposition) to find an orthogonal set of vectors and find the range for the desired percent along each of these independent variables. Then you could transform back to the original basis, and you would have an ellipse with the desired range.

abalter
  • 770
  • 6
  • 18