1

I have some data which I wish to cluster. I would like to see how these clusters compare to the categories that have already been assigned. Is there some kind of metric or visualisation that will tell me how well these line up?

  • Any suggestions with each item assigned to just one category is fine, but each data element can actually be assigned multiple categories. Perhaps there's a neat way to deal with this?
Casebash
  • 429
  • 1
  • 3
  • 13
  • 2
    So, your task is to assess how much the clusters have reproduced the existent classes, yes? Then search for "external clustering validation". – ttnphns Oct 08 '13 at 06:12

1 Answers1

1

Yes, this is known as external evaluation (and IMHO makes more sense than using internal measures). Read the Wikipedia article on Cluster Analysis!

One of the most popular is probably the ARI.

However, I'm not aware of any method that makes sense when you have overlapping labels. Most measures have the implicit assumption that you are comparing strict partitionings. Also, they can't really deal with "noise", as produced by e.g. DBSCAN clustering. Therefore, you still can't compare results of different algorithms very well.

Has QUIT--Anony-Mousse
  • 39,639
  • 7
  • 61
  • 96
  • Some measures listed, such as the Rand measure, deal with items pairwise only and so could easily be adapted to handle multiple clusters – Casebash Oct 08 '13 at 07:48
  • @Casebash have you tried it? It's not that easy to get *meaningful* and *unbiased* results. You can compute some number, but it may be biased e.g. towards or against results that have a deep/shallow hierarchy etc. – Has QUIT--Anony-Mousse Oct 08 '13 at 09:13
  • I think that'll more be an issue of the measure itself, rather than an issue caused by the adaption – Casebash Oct 08 '13 at 23:44
  • Well, it definitely is an issue; and the methods don't "just" work with such clusterings. They are too biased to be useful when you don't have a strict partitioning IMHO. In fact, with many you may end up with scores higher than 1 or lower than 0 if you try to use them on a non-strict partitioning. – Has QUIT--Anony-Mousse Oct 09 '13 at 07:40