8

enter image description here

I have a correlation matrix of 8,854 * 8,854 size. These are Pearson correlation coefficient values in the matrix. I want to perform Hierarchical clustering and create good resolution images like I have attached. A step by step explanation would be a great help.

enter image description here

enter image description here

gung - Reinstate Monica
  • 132,789
  • 81
  • 357
  • 650
bsoni
  • 81
  • 1
  • 1
  • 3
  • What's the point of this? From the information given, you have data for lots of pairs of times. You should get get more out of an autocorrelation function or (if your times are irregularly spaced) a variogram than what you are asking for (even assuming it's computable in reasonable time). Do you think an image 8000 pixels square will be intelligible? – Nick Cox Apr 24 '14 at 11:47
  • `A step by step explanation would be a great help` What is it? Maybe you can work out a more specific question? What are the troubles on your way, for instance. – ttnphns Apr 24 '14 at 12:44
  • Thanks for the clarification: the appearance of dates is pure illusion, so the suggestion of autocorrelation or variogram is quite wrong here. But 8000+ square sounds awful big to me. – Nick Cox Apr 24 '14 at 13:29

1 Answers1

10

To apply most hierarchical clustering/heatmap tools you'll need to convert your correlation matrix into a distance matrix (ie 0 is close together, higher is further apart). This blog post covers some simple methods with R code.

However, a more computationally efficient method is to convert the correlation matrix to a graph, apply a cutoff so that it is sparse and apply graph partitioning methods. A colleague and I actually have a project, Inspectra, that uses spectral graph analysis to compare graphs derived from correlation matrixes. We are developing it for cancer data and it is still a prototype but won best poster at biovis last year and you are welcome to try it and contact one of us via github if you have questions.

Ryan Bressler
  • 2,147
  • 10
  • 15
  • Thank you so much. Instructions given in the first link you provided answers almost all my questions. – bsoni Apr 25 '14 at 18:21
  • I would be very much interested in the graph conversion method you have suggested. – bsoni Apr 25 '14 at 18:28
  • @Ryan If I understand correctly Inspectra can be used to eliminate "low" correlations from the correlation network? – Andrej Aug 02 '15 at 11:40
  • @Andrej, inspectra is for comparing two correlation networks (ie from two disease subtypes or something). It does include a parameter for filtering out edge weights close to zero but it does lots of other things too. – Ryan Bressler Aug 03 '15 at 18:20