Visualize Analysis of clustering after pca

Question

I am using kmeans for clustering and if I read the topics around here and somewhere else it is always recommended to do a graphical check-up for the number of clusters.

The problem is my orginal dataframe has 80 features. So I did a pca in python

pca = PCA(n_components=2)
X_pca = pca.fit_transform(data)

Is it ok to use this as analysis now? The output looks like this (only two dimensions):

Or do I miss something when I did the pca?

Answer 1

To determine the number of clusters is a hard problem. Let $K$ be the number of clusters.

Larger K means smaller "within cluster variation" (W), which is good.

Larger K means smaller "Between cluster variation" (B), which is bad.

One solution found in the literature is to use the ration between the two metrics and it is called the CH index=B/W. See for instance Tibshirani slides.

Use the CH-index in Sklearn.