2

Classic MDS (cMDS or PCoA) preserves global distances, characteristic of linear techniques. However, metric MDS seeks to minimize a cost function (stress), while non-metric MDS (nMDS) preserves only the ranking of dissimilarities between points. It seems to me these techniques produce a kind of embedding, which would be nonlinear, but both cMDS and nMDS are listed as linear techniques in this article. Conversely, Wikipedia describes MDS in general as a form of nonlinear dimensionality reduction.

It is possible to use a nonlinear kernel in MDS to preserve smaller distances, as in the case of a Sammon mapping. This is definitely a nonlinear technique.

So: are multidimensional scaling and its variants considered linear or nonlinear dimensionality reduction techniques, and why?

sara-es
  • 21
  • 4
  • "but both cMDS and nMDS are listed as linear techniques in this article" -- where exactly? – amoeba Jul 30 '19 at 11:05
  • It clearly lists nMDS as nonlinear in "Tip 1". – amoeba Jul 30 '19 at 11:06
  • The article itself seems to be contradictory. NMDS is not classified as nonlinear in [table 1](https://doi.org/10.1371/journal.pcbi.1006907.t001) in the same paper. – sara-es Jul 31 '19 at 13:22
  • Oh I see. So it's just an unfortunate typo in the Table 1. – amoeba Jul 31 '19 at 13:33
  • 1
    I might write-up a proper answer later, but I wouldn't call t-SNE/nMDS/etc "nonlinear", I think it's sloppy. They should rather be called "nonparametric". See here https://stats.stackexchange.com/questions/142960/ about this distinction. cMDS is called "linear" because it's equivalent to PCA but it's actually an abuse of terminology. – amoeba Jul 31 '19 at 13:39

1 Answers1

1

According to the information I know, I think Metric-MDS just construct the distance matrix by using Euclidian distance, which is a linear transformation. But Non-Metric-MDS redefines the distance between the data using rank order of distances, which is a nonlinear transformation.

JJ weng
  • 11
  • 1