11

What are the "best" metrics for covariance matrices, and why? It is clear to me that Frobenius&c are not appropriate, and angle parametrizations have their issues too. Intuitively one might want a compromise between these two, but I would also like to know if there are other aspects to keep in mind and maybe well-established standards.

Common metrics have various drawbacks since they're not natural for covariance matrices, e.g. they often don't especially penalize non PSD matrices or don't behave well w.r.t rank (consider two rotated low-rank covariance ellipsoids: I'd like the same-rank intermediate rotation to have lower distances than the componentwise average, which is not the case with $L_1$ and maybe Frobenius, please correct me here). Also convexity is not always guaranteed. It'd be good seeing these and other issues addressed by a "good" metric.

Here is a good discussion of some issues, one example from network optimization and one from computer vision. And here's a similar question getting some other metrics but without discussion.

Community
  • 1
Quartz
  • 878
  • 8
  • 18
  • 2
    What is the purpose of the metric you seek? For what is the Frobenius metric inappropriate? – whuber Jul 01 '13 at 17:32
  • 1
    @whuber: I'd like to get an overview in general before imposing too many constraints. My field is quant finance where most people stick to Frobenius for simplicity. Common metrics have various drawbacks since they're not natural for covariance matrices, e.g. they dont especially penalize non PSD matrices and dont behave well w.r.t rank (think of two rotated low-rank covariance ellipsoids: I'd like the same-rank intermediate rotation to have lower distances than the componentwise average, which is not the case with $L_1$ and maybe Frobenius if I'm not mistaken). Added a few links. – Quartz Jul 02 '13 at 10:15
  • How is that last question you reference "more restricted"? After all, all covariance matrices are symmetric. It seems to be a perfect duplicate. – whuber Jul 02 '13 at 11:53
  • That's a funny title yeah, and sure, "restricted" is bad wording on my part (on meta it was being discussed how minor editing is discouraged, shall I correct nevertheless?). Anyway what the poster said was an excellent answer doesnt seem to address my question at all, hence my dissatisfaction. I'm looking for explicit discussion of the drawbacks/strengths of each metric and for methods avoiding those problems mentioned in the comment here (where the former could be intended implicit in that question, but answers didn't go into it, they just list different metrics). – Quartz Jul 02 '13 at 12:34
  • 1
    That's a good criticism of the other question. May I suggest you edit your question (and title) to reflect the content of your last comment? That will clearly distinguish it from the apparent duplicate and help respondents give you more appropriate replies. (And don't worry about edits to your own question: that is expected; the meta thread is primarily about *community* editing.) – whuber Jul 02 '13 at 12:36
  • @whuber: I've expanded the question; thanks for helping me clarifying, conciseness was indeed excessive. – Quartz Jul 02 '13 at 14:51
  • @user603 I'm asking for their properties to compare them and choose one; that question was linked as an example not answering the question, see the above comments. – Quartz Feb 03 '14 at 16:27
  • The set of covariance matrices can be organized as a Riemannian geometry, and then we can find a natural distance: The riemannian distance. – kjetil b halvorsen Jun 30 '14 at 19:47
  • 1
    @kjetilbhalvorsen That's a provocative sentence! Could you expand in an answer? Or provide an article reference? – Sycorax Aug 02 '14 at 00:05

2 Answers2

2

Well, I don't think there is a good metric or 'the best way' to analyze Covariance matrices. The analysis should be always aligned to your goal. Let's say C is my covariance matrix. The diagonal contains the variance for each computed parameter. So if you're interested in parameter significance then trace(C) is a good start since it's your overall performance.

If you plot your parameter and their significance you can see something like this:

x1 =  1.0 ±  0.1 
x2 = 10.0 ±  5.0
x3 =  5.0 ± 15.0 <-- non-significant parameter

If you're interested in their mutual correlation then such a table might yield something interesting:

x1  1.0
x2  0.9  1.0
x3 -0.3 -0.1  1.0
    x1    x2   x3

Each element is the correlation coefficient between the parameter xi and xj. From the example it's visible that parameter x1 and x2 are highly correlated.

nali
  • 424
  • 3
  • 12
1

Interesting question, I'm grappling with the same issue at the moment! It depends on how you define 'best', i.e., are you looking for some average single value for the spread, or for the correlation between the data, etc. I found in Press, S.J. (1972): Applied Multivariate Analysis, p. 108 that the generalized variance, defined as the determinant of the covariance matrix, is useful as a single measure for spread. But if it's correlation that you are after, I will need to think futher. Let me know.

Lucozade
  • 619
  • 1
  • 6
  • 7