Statistical Analysis Stack Exchange
Statistical Analysis Stack Exchange

Questions tagged [manifold-learning]

Manifold learning subsumes techniques conceived for problems where data of interest are assumed to lie on an embedded non-linear manifold within a higher-dimensional space.

Manifold Learning can be thought of as an attempt to generalize linear frameworks like PCA to be sensitive to non-linear structure in data. Though supervised variants exist, the typical manifold learning problem is unsupervised: it learns the high-dimensional structure of the data from the data itself, without the use of predetermined classifications.

Examples of Manifold Learning algorithms include:

  • Isomap
  • Locally Linear Embedding
  • Hessian Eigenmapping
  • Laplacian Eigenmaps
  • Multi-dimensional Scaling (MDS)
  • t-distributed Stochastic Neighbor Embedding (t-SNE)

Reference: http://scikit-learn.org/stable/modules/manifold.html and https://en.wikipedia.org/wiki/Nonlinear_dimensionality_reduction#Manifold_learning_algorithms

81 questions
33
votes
4 answers

What is a manifold?

In dimensionality reduction technique such as Principal Component Analysis, LDA etc often the term manifold is used. What is a manifold in non-technical term? If a point $x$ belongs to a sphere whose dimension I want to reduce, and if there is a…
Ria George
  • 1,375
  • 2
  • 14
  • 31
28
votes
2 answers

How to understand "nonlinear" as in "nonlinear dimensionality reduction"?

I am trying to understand the differences between the linear dimensionality reduction methods (e.g., PCA) and the nonlinear ones (e.g., Isomap). I cannot quite understand what the (non)linearity implies in this context. I read from Wikipedia…
Sibbs Gambling
  • 2,208
  • 5
  • 20
  • 42
23
votes
2 answers

What is the manifold assumption in semi-supervised learning?

I am trying to figure out what the manifold assumption means in semi-supervised learning. Can anyone explain in a simple way? I cannot get the intuition behind it. It says that your data lie on a low-dimensional manifold embedded in a…
user34790
  • 6,049
  • 6
  • 42
  • 64
15
votes
2 answers

How to choose a kernel for kernel PCA?

What are the ways to choose what kernel would result in good data separation in the final data output by kernel PCA (principal component analysis), and what are the ways to optimize parameters of the kernel? Layman's terms if possible would be…
Chives
  • 253
  • 3
  • 9
13
votes
2 answers

Graphical intuition of statistics on a manifold

On this post, you can read the statement: Models are usually represented by points $\theta$ on a finite dimensional manifold. On Differential Geometry and Statistics by Michael K Murray and John W Rice these concepts are explained in prose…
Antoni Parellada
  • 23,430
  • 15
  • 100
  • 197
11
votes
4 answers

Dimension reduction using space filling curve to avoid "Curse of dimensionality"?

In machine learning, we want to train a model. While training, if the dimension of data is high, we have a problem (Curse of Dimensionality), so we want to reduce the dimension of our data. Since we know $\mathbb{R}^n$ and $\mathbb{R}$ have the same…
Shriman Keshri
  • 111
  • 4
10
votes
1 answer

What is the difference between manifold learning and non-linear dimensionality reduction?

What is the difference between manifold learning and non-linear dimensionality reduction? I have seen these two terms being used interchangeably. For example: http://www.cs.cornell.edu/~kilian/research/manifold/manifold.html : Manifold Learning…
Franck Dernoncourt
  • 42,093
  • 30
  • 155
  • 271
10
votes
1 answer

What exactly is the procedure to compute principal components in kernel PCA?

In kernel PCA (principal component analysis) you first choose a desired kernel, use it to find your $K$ matrix, center the feature space via the $K$ matrix, find its eigenvalues and eigenvectors, then multiply the centered kernel matrix by the…
Chives
  • 253
  • 3
  • 9
9
votes
2 answers

How to prove that the manifold assumption is correct?

In machine learning, it is often assumed that a data set lies on a smooth low-dimensional manifold (the manifold assumption), but is there any way to prove that assuming certain conditions are satisfied, then the data set is indeed (approximately)…
thinkbear
  • 141
  • 5
8
votes
1 answer

Nonlinear Dimensionality Reduction: geometric/topologic algorithms vs. autoencoders

As I understand there are three main approaches to nonlinear dimensionality reduction: Manifold learning (geometric/topologic algorithms like ISOMAP, LLE, LTSA) Autoencoders things that do not fit into first 2 categories (probability inspired…
Sengiley
  • 326
  • 3
  • 10
7
votes
1 answer

Comparing ISOMAP residual variance to PCA explained variance

I am using R princomp function (from stats package) to run a PCA on a data set and I want to compare its output to that of the nonlinear dimensionality reduction method ISOMAP, which I am using under matlab through this toolbox:…
user3358740
  • 71
  • 3
6
votes
3 answers

How to apply Isomap to test data?

I implemented Isomap and I plan to use it as a feature extraction technique for a classification task. My problem is that although I can map the training data into a lower dimensional space, how can I use the same mapping for a separate test…
Zoran
  • 522
  • 4
  • 11
6
votes
1 answer

How to know when to use linear dimensionality reduction vs non-linear dimensionality reduction?

I am trying to decide whether to use linear dimensionality reduction methods (eg. PCA) vs. non-linear dimensionality reduction methods (eg. t-SNE) for my high-dimensional data set. However, I know nothing about the underlying structure of the data.…
M. T.
  • 61
  • 3
6
votes
1 answer

What is the value of linear dimensionality reduction in the presence of nonlinear alternatives?

From the results I've seen, manifold learning methods seem to generally outperform PCA for complicated, very high-dimensional datasets like images or videos. This makes sense to me, since nonlinear dimensionality reduction (like manifold learning)…
KFox
  • 235
  • 2
  • 6
6
votes
1 answer

Two broad categories of dimensionality reduction: with and without an explicit mapping function

I think there are two very broad categories of dimensionality reduction (DR) techniques: We can compute an analytic form of mapping from the training data, say $x\mapsto y: y=f(x)$, where $f(\cdot)$ can be linear or nonlinear. In this case, given a…
Sibbs Gambling
  • 2,208
  • 5
  • 20
  • 42
1
2 3 4 5 6