1

I am trying to cluster textual data using fastText vectors with different clustering algorithms, mainly K-Means and DBSCAN.

I would like to know which internal evaluation metric works best with K-means and DBSCAN (ex: silhouette coefficient).

More specifically, I am looking for a metric that does not give higher values to convex-shaped like the silhouette coefficient in order to be able to compare clustering obtained with K-Means and DBSCAN.

My understanding is that internal clustering metrics are not only to be used for comparing a different number of clusters or clustering methods. What decision should I make if different metrics contradict each other i.e if one metric indicates an improvement and another, a decrease in clustering quality?

ryuzakinho
  • 163
  • 8
  • 1
    Some info on internal clustering criteria https://stats.stackexchange.com/a/358937/3277. You always may compare any partitions of the same dataset (partitions produced by any, different methods) by any selected clustering validity index. – ttnphns Aug 20 '18 at 14:16
  • We need more info about your data. – Mensch Aug 20 '18 at 17:15
  • My data consists of 300-dimension real-valued vectors representing short chat messages. The vectors were obtained using the FastText library (https://fasttext.cc/) – ryuzakinho Aug 21 '18 at 07:50
  • 1
    It sounds like your data are counts of words in the messages that match a dictionary. k-means is unlikely to be a good choice for that a-priori (regardless of what any metric shows, see: [How to understand the drawbacks of K-means](https://stats.stackexchange.com/q/133656/), & [Why is Euclidean distance not a good metric in high dimensions?](https://stats.stackexchange.com/q/99171/)). DBSCAN may or may not be a good choice; I would focus on understanding your data & picking the most appropriate distance metric. – gung - Reinstate Monica Aug 22 '18 at 01:55
  • @gung For finding similar vectors, I have been using cosine similarity between vectors. To be able to do this, I have started using spherical kmeans. Again, I tried using spherical kmeans out of intuition more than anything else i.e I did not have any proper way to assess the quality of either method. – ryuzakinho Aug 22 '18 at 07:00

1 Answers1

3

Actually almost no internal metric can handle DBSCAN results properly.

The problem is that noise is not a cluster and almost all metrics assume the data is strictly partitioned into disjoint clusters.

Most implementations will then evaluate noise like a cluster, and the result will appear much worse than it is.

Pretty much the only metric I know (but haven't used) for this is DBCV, it supposedly is designed for density based cluster evaluation.

In general the only way to choose an evaluation metric is to understand what it does. Pick there meric whose formal approach is most closely related to your desire of a "good" cluster. Because everybody seems to have a slightly different understanding of when a cluster is "good".

Has QUIT--Anony-Mousse
  • 39,639
  • 7
  • 61
  • 96
  • Found this implementation for DBCV: https://github.com/christopherjenness/DBCV – ryuzakinho Aug 22 '18 at 11:35
  • Would it be possible to use DBCV as an internal metric for KMEANS in theory? – ryuzakinho Aug 22 '18 at 13:04
  • 1
    I guess so. Ideally, KMeans clusters are also density separated. A bad choice of k tend to split a cluster, with no separation, so you could possibly detect that. But likely KMeans will always score bad in DBCV because it puts noise into the nearest cluster... Check the paper, try to **understand** what it measures, and how that relates to k-means. And double check any code you "find". You can find a lot of flawed code, too. – Has QUIT--Anony-Mousse Aug 22 '18 at 13:22