7

I am using R princomp function (from stats package) to run a PCA on a data set and I want to compare its output to that of the nonlinear dimensionality reduction method ISOMAP, which I am using under matlab through this toolbox: http://isomap.stanford.edu on the same dataset.

What I am interested in is the intrinsic dimensionality of the dataset as determined according to PCA and to ISOMAP separately, the ultimate goal being to check whether nonlinear dimensionality reduction works better on this dataset than PCA.

With princomp I get the standard deviations associated to each component, while the ISOMAP package returns residual variances as a function of the manifold dimensionality. How do I compare these two quantities? In other words, how is the residual variance defined in ISOMAP?

This should be irrelevant, but the dataset is 54 points in 5 dimensions.

amoeba
  • 93,463
  • 28
  • 275
  • 317
user3358740
  • 71
  • 3
  • Add up squared standard deviations of PCA components, subtract from the total variance of the dataset, and you get residual variance that can be directly compared to ISOMAP values. – amoeba Dec 29 '14 at 15:32
  • @amoeba I know this is an old question, but you should post that as an answer. – shadowtalker Apr 09 '15 at 02:08
  • @ssdecontrol: My comment only answers one part of the question. The other part is about how the residual variance is defined in ISOMAP (and why). Seeing your interest, I might write an answer, but it will require some research. I think I know approximately how it is supposed to work, but I need to check the mathematical details. – amoeba Apr 09 '15 at 13:25
  • @amoeba I just stumbled on this question by following related links. OP seems long gone and you probably have other things to do, and more important questions to spend your time on. – shadowtalker Apr 09 '15 at 15:03

1 Answers1

3

The original Isomap paper defined "residual variance" as follows (reference 42):

$$\text{residual variance} = 1 - R^2(\hat D_M, D_y)$$

where $R$ is the Pearson correlation coefficient over all entries of $\hat D_M$ and $D_Y$. $\hat D_M$ is the euclidean distance matrix for PCA and the geodesic distance matrix for Isomap. $D_Y$ is the euclidean distance matrix of the low dimensional embedding, this matrix changes with the number of dimensions you use for the embedding.

EDIT: This is numerically different than the explained variance of PCA derived from the eigenvalues and I don't know if there is a direct connection between the two of them.

EDIT: I asked here for the connection between residual and explained variance.

gdkrmr
  • 131
  • 4