4

I have 2 results for the same dataset. One is hierarchical clustering using Ward's method and I got 0.75 cophenetic correlation coefficient. The second is average method and I got 0.91 cophenetic correlation coefficient. I used "euclidean distance" for both. When I plot the dendrograms, Ward's dendrogram seems excellent while the one with average method is bad. So, Can I use the Ward's one since it has a good dendrogram and easy to cut the tree, or I have to use the average one?

average

ward

Emrah Bilgiç
  • 289
  • 2
  • 7
  • 14
  • 1
    Let me first of all to recommend you to read [this](http://stats.stackexchange.com/a/63549/3277) answer, especially 2 first points. Also, please explain what do you mean by cophenetic correlation: do you compute it for the whole dendrogram or for the selected solution (the specific cut level giving you k clusters), in which the latter case it will be point-biserial correlation. – ttnphns Mar 04 '16 at 07:30
  • Thank you, I read your post. I computed the cophenetic correlation for the whole dendrogram; between the distance matrix and the clustering like: d1 – Emrah Bilgiç Mar 04 '16 at 08:07
  • 1
    So, if you mean that you are computig "overall dendrogram" correlation between the distance and the step level and not for a specific partition (solution) - then the question why do you need so. Are you not going to arrive at a partition? Method X can easily lose to method Y by the "overall" cophenetic correlation and still give the best partition which is much better than method's Y best partition, as tested by the "local" point-biserial cophenetic correlation. – ttnphns Mar 04 '16 at 08:16
  • Thank you, so I should compute cophenetic correlation for the specific cut level. – Emrah Bilgiç Mar 04 '16 at 08:22
  • Yes, try it. Instead of overal-dendrogram cophenetic corr. value (such as explained [here](http://stats.stackexchange.com/a/166861/3277)) compute point biserial correlation (such as mentioned [here](http://stats.stackexchange.com/a/22867/3277)) for any cut level you you see a good candidate for partition - one value for average method, the other value for Ward,s method. Compare and select the best partition and method. – ttnphns Mar 04 '16 at 08:29
  • I think you should also consider not using pearson's correlation (but maybe spearman) - since your first split is very high - it means it would give you very extreme values which pearson would be very sensitive to. – Tal Galili Mar 04 '16 at 09:56

0 Answers0