Statistical Analysis Stack Exchange
Statistical Analysis Stack Exchange

Questions tagged [autoencoders]

Feedforward neural networks trained to reconstruct their own input. Usually one of the hidden layers is a "bottleneck", leading to encoder->decoder interpretation.

Autoencoders can be applied to unlabeled data to learn features that best represent the variations in the data distribution. This process is known as unsupervised feature learning or representation learning.

Here is an example of a set of filters learned by an autoencoder on a face recognition dataset:

569 questions
136
votes
4 answers

What is the difference between convolutional neural networks, restricted Boltzmann machines, and auto-encoders?

Recently I have been reading about deep learning and I am confused about the terms (or say technologies). What is the difference between Convolutional neural networks (CNN), Restricted Boltzmann machines (RBM) and Auto-encoders?
RockTheStar
  • 11,277
  • 31
  • 63
  • 89
101
votes
7 answers

How does the reparameterization trick for VAEs work and why is it important?

How does the reparameterization trick for variational autoencoders (VAE) work? Is there an intuitive and easy explanation without simplifying the underlying math? And why do we need the 'trick'?
David Dao
  • 2,474
  • 3
  • 12
  • 16
84
votes
4 answers

What're the differences between PCA and autoencoder?

Both PCA and autoencoder can do demension reduction, so what are the difference between them? In what situation I should use one over another?
RockTheStar
  • 11,277
  • 31
  • 63
  • 89
45
votes
4 answers

how to weight KLD loss vs reconstruction loss in variational auto-encoder

in nearly all code examples I've seen of a VAE, the loss functions are defined as follows (this is tensorflow code, but I've seen similar for theano, torch etc. It's also for a convnet, but that's also not too relevant, just affects the axes the…
memo
  • 859
  • 1
  • 7
  • 10
40
votes
4 answers

When should I use a variational autoencoder as opposed to an autoencoder?

I understand the basic structure of variational autoencoder and normal (deterministic) autoencoder and the math behind them, but when and why would I prefer one type of autoencoder to the other? All I can think about is the prior distribution of…
DiveIntoML
  • 1,583
  • 1
  • 11
  • 21
38
votes
4 answers

What are the differences between sparse coding and autoencoder?

Sparse coding is defined as learning an over-complete set of basis vectors to represent input vectors (<-- why do we want this) . What are the differences between sparse coding and autoencoder? When will we use sparse coding and autoencoder?
RockTheStar
  • 11,277
  • 31
  • 63
  • 89
37
votes
3 answers

Building an autoencoder in Tensorflow to surpass PCA

Hinton and Salakhutdinov in Reducing the Dimensionality of Data with Neural Networks, Science 2006 proposed a non-linear PCA through the use of a deep autoencoder. I have tried to build and train a PCA autoencoder with Tensorflow several times but I…
Donbeo
  • 3,001
  • 5
  • 31
  • 48
36
votes
3 answers

Pre-training in deep convolutional neural network?

Have anyone seen any literature on pre-training in deep convolutional neural network? I have only seen unsupervised pre-training in autoencoder or restricted boltzman machines.
RockTheStar
  • 11,277
  • 31
  • 63
  • 89
31
votes
1 answer

What are variational autoencoders and to what learning tasks are they used?

As per this and this answer, autoencoders seem to be a technique that uses neural networks for dimension reduction. I would like to additionally know what is a variational autoencoder (its main differences/benefits over a "traditional" autoencoders)…
user188529
30
votes
2 answers

What is the "capacity" of a machine learning model?

I'm studying this Tutorial on Variational Autoencoders by Carl Doersch. In the second page it states: One of the most popular such frameworks is the Variational Autoencoder [1, 3], the subject of this tutorial. The assumptions of this model are…
Andrés Marafioti
  • 513
  • 1
  • 4
  • 5
28
votes
2 answers

Loss function for autoencoders

I am experimenting a bit autoencoders, and with tensorflow I created a model that tries to reconstruct the MNIST dataset. My network is very simple: X, e1, e2, d1, Y, where e1 and e2 are encoding layers, d2 and Y are decoding layers (and Y is the…
AkiRoss
  • 465
  • 1
  • 4
  • 11
25
votes
2 answers

Autoencoders can't learn meaningful features

I have 50,000 images such as these two: They depict graphs of data. I wanted to extract features from these images so I used autoencoder code provided by Theano (deeplearning.net). The problem is, these autoencoders don't seem to learn any…
b93dh44
  • 253
  • 1
  • 3
  • 6
24
votes
3 answers

What is the architecture of a stacked convolutional autoencoder?

So I am trying to do pretraining on images of humans using convolutional nets. I read the papers (Paper1 and Paper2) and this stackoverflow link, but I am not sure I am understand the structure of the nets (it is not well defined in the…
Run2
  • 453
  • 1
  • 3
  • 9
22
votes
4 answers

Why do we need autoencoders?

Recently, I have been studying autoencoders. If I understood correctly, an autoencoder is a neural network where the input layer is identical to the output layer. So, the neural network tries to predict the output using the input as golden…
larry
  • 321
  • 2
  • 5
21
votes
3 answers

Loss function autoencoder vs variational-autoencoder or MSE-loss vs binary-cross-entropy-loss

When having real valued entries (e.g. floats between 0 and 1 as normalized representation for greyscale values from 0 to 256) in our label vector, I always thought that we use MSE(R2-loss) if we want to measure the distance/error between input and…
SolingerMuc
  • 674
  • 1
  • 4
  • 12
1
2 3
…
37 38