0

The publication introducing Isomap compares PCA and Isomap by means of

$$\text{residual variance} = 1 - R^2(\hat D_M, D_y)$$

where $R$ is the standard linear correlation coefficient over all entries of $\hat D_M$ and $D_Y$. $\hat D_M$ is the euclidean distance matrix for PCA and the geodesic distance matrix for Isomap. $D_Y$ is the euclidean distance matrix of the low dimensional embedding, this matrix changes with the number of dimensions you use for the embedding.

Doing some testing reveals that "residual variance" is numerically different from $$ 1 - \text{explained variance}$$ derived from the eigenvalue spectrum.

Is there a link between "residual variance" and "explained variance" for PCA and possibly Isomap?

gdkrmr
  • 131
  • 4
  • That's presumably correlation coefficient across all entries of $D$ matrices, i.e. these $n\times n$ matrices are "vectorized" as $n^2$ datapoints, right? – amoeba May 25 '18 at 09:22
  • PCA is not even trying to match $D_Y$ with $D_M$, that's the task of multi-dimensional scaling. There is no relation between PCA explained variance and this correlation coefficient. – amoeba May 25 '18 at 09:23
  • Regarding your first comment: I think so, this is how I read the footnote in the original paper. – gdkrmr May 25 '18 at 13:00
  • Regarding your second comment: MDS gives the same result as PCA when used with Euclidean distances, so there should be some connection. – gdkrmr May 25 '18 at 13:06
  • It's so called "Torgerson" (aka "classical") MDS that is equivalent to PCA, but it does not match distances. The version of MDS that actually tries to match distances is not equivalent to PCA. – amoeba May 25 '18 at 13:09
  • yes, I was thinking about classical scaling, sorry. – gdkrmr May 25 '18 at 13:10
  • pca minimizes $\sum_{i,j} d(x_i, x_j)^2 - d(y_i, y_j)^2$ – gdkrmr May 25 '18 at 13:27
  • I don't think so. – amoeba May 25 '18 at 14:01
  • checkout https://doi.org/10.1080/03610927808827707 – gdkrmr May 25 '18 at 14:28

0 Answers0