0

I just started reading about GAN theory properly for the first time and I have a question about a comment in the original GAN paper.

On page two there's a paragraph that states the following:

... had developed more general stochastic backpropagation rules, allowing one to backpropagate through Gaussian distributions with finite variance, and to backpropagate to the covariance parameter as well as the mean.

Can anyone explain what backpropagating through a distribution means? In other words, what's the concrete meaning of this?

Jan Kukacka
  • 10,121
  • 1
  • 36
  • 62
del
  • 171
  • 3
  • Actually, this paragraph is not in the (arxiv) version of the paper you linked. It is in the NIPS version: https://papers.nips.cc/paper/5423-generative-adversarial-nets.pdf – Jan Kukacka Apr 10 '19 at 14:38

1 Answers1

1

The quoted paragraph (actually appearing in the NIPS version of GAN paper) is dealing with variational autoencoders (VAE). Particularly, it refers to their reparametrization trick, which allows efficient computation of gradients of expected values over distributions parametrized by the differentiated parameters.

How exactly this reparametrization works is described nicely in: How does the reparameterization trick for VAEs work and why is it important?

Another great, easy to understand explanation is in the blog post The reparametrization trick by Gregory Gundersen.

Jan Kukacka
  • 10,121
  • 1
  • 36
  • 62