Statistical Analysis Stack Exchange
Statistical Analysis Stack Exchange

Questions tagged [generative-models]

A probabilistic or statistical model thought about as describing how the values in a sample is actually generated, and not only as a description or approximation.

Wikipedia has an article https://en.wikipedia.org/wiki/Generative_model with further references.

235 questions
197
votes
3 answers

Generative vs. discriminative

I know that generative means "based on $P(x,y)$" and discriminative means "based on $P(y|x)$," but I'm confused on several points: Wikipedia (+ many other hits on the web) classify things like SVMs and decision trees as being discriminative. But…
Yang
  • 2,981
  • 3
  • 20
  • 18
101
votes
7 answers

How does the reparameterization trick for VAEs work and why is it important?

How does the reparameterization trick for variational autoencoders (VAE) work? Is there an intuitive and easy explanation without simplifying the underlying math? And why do we need the 'trick'?
David Dao
  • 2,474
  • 3
  • 12
  • 16
19
votes
2 answers

Modern Use Cases of Restricted Boltzmann Machines (RBM's)?

Background: A lot of the modern research in the past ~4 years (post alexnet) seems to have moved away from using generative pretraining for neural networks to achieve state of the art classification results. For example, the top results for mnist…
user27886
  • 585
  • 5
  • 14
16
votes
2 answers

The connection between Bayesian statistics and generative modeling

Can someone refer me to a good reference that explains the connection between Bayesian statistics and generative modeling techniques? Why do we usually use generative models with Bayesian techniques? Why it is especially appealing to use Bayesian…
bayesian
  • 799
  • 1
  • 8
  • 9
14
votes
4 answers

Likelihood-free inference - what does it mean?

Recently I have become aware of 'likelihood-free' methods being bandied about in literature. However I am not clear on what it means for an inference or optimization method to be likelihood-free. In machine learning the goal is usually to maximise…
piccolo
  • 797
  • 5
  • 13
13
votes
2 answers

Why are Gaussian "discriminant" analysis models called so?

Gaussian discriminant analysis models learn $P(x|y)$ and then apply Bayes rule to evaluate $$P(y|x) = \frac{P(x|y)P_{prior}(y)}{\Sigma_{g \in Y} P(x|g) P_{prior}(g) }.$$ Hence, they are generative models. Why then is it called discriminant…
highBandWidth
  • 2,092
  • 2
  • 21
  • 34
12
votes
1 answer

How to understand Generative Adversarial Networks Discriminative distribution?

So I am currently studying Generative Adversarial Network and I read the paper by Goodfellow a few times now Generative Adversarial Nets and a few other papers in this field (DCGAN, CycleGAN, pix2pix, and a few others). But I have been struggling…
Kalle
  • 225
  • 1
  • 5
12
votes
1 answer

Variational Autoencoder − Dimension of the latent space

I've done some experiments to understand the influence of the dimension of the latent space in a VAE, and it seems that the higher the space, the harder it is to generate realistic images. I might have an intuition of the reason, and I wanted to…
Arthur Pesah
  • 223
  • 1
  • 2
  • 10
11
votes
2 answers

What are the current methods to check for GAN overfitting?

In generative modeling, the goal is to find a way for a model to output samples of some distribution $p_X$ given a lot of samples $x_1, \ldots, x_n$. In particular, we want sampling from our model $G$ to satisfy $G(z)$ is a new example $G(z)$ looks…
Anon
  • 567
  • 3
  • 16
11
votes
3 answers

Is the optimization of the Gaussian VAE well-posed?

In a Variational Autoencoder (VAE), given some data $x$ and latent variables $t$ with prior distribution $p(t) = \mathcal{N}(t \mid 0, I)$, the encoder aims to learn a distribution $q_{\phi}(t)$ that approximates the true posterior $p(t \mid x)$ and…
learner
  • 637
  • 6
  • 14
11
votes
1 answer

Why in Variational Auto Encoder (Gaussian variational family) we model $\log\sigma^2$ and not $\sigma^2$ (or $\sigma$) itself?

In theory the encoder in VAE (assuming that variational family is Gaussian) generates the $\mu$ and $\sigma$ (or $\sigma^2$). But, in practice, I have seen people assuming the output is $\log\sigma^2$. Why this is necessary or useful?
user3639557
  • 1,042
  • 10
  • 21
11
votes
3 answers

Why use Binary Cross Entropy for Generator in Adversarial Networks

I'm trying to work with General Adversarial Networks and there's something I'm seeing everywhere but can't explain why... the GANs are usually constructed from a Generator (which usually generates an image and is connected to the discriminator) and…
Cypher
  • 446
  • 1
  • 4
  • 14
11
votes
1 answer

Beyond Fisher kernels

For a while, it seemed like Fisher Kernels might become popular, as they seemed to be a way to construct kernels from probabilistic models. However, I've rarely seen them used in practice, and I have it on good authority that they tend not to work…
tdc
  • 7,289
  • 5
  • 32
  • 62
10
votes
1 answer

Choosing a Generative Models for time series data

I'm looking to try and generate economic time series data (GDP, Inflation, Unemployment etc.) with some generative models. I am thinking of using them with LSTMs as well as more AR(p) style models so I would prefer models that generate long strings…
www3
  • 601
  • 8
  • 16
8
votes
1 answer

Structure of Generative Adversarial Networks (GAN) for mapping a simulation model

There is a simulation model of a system that I want to map as a neural network to test if a better execution time can be achieved with similar accuracy. The simulation model receives real-valued measurement data of its environment and generates a…
Emma
  • 93
  • 6
1
2 3
…
15 16