9

Reading here, I found:

If the discriminator wins by too big a margin, then the generator can’t learn as the discriminator error is too small.

This is something I read somewhere else as well, but I can't really get it. If the discriminator has a low loss, it means that when I give it a fake sample (with "fake" label) it gives me a low score (assuming its output is "probability of real") with high certainty, so I can imagine that the gradient of the error will be small.

When I train the generator, I pass the same fake image, but with the "real" label. In this case, I expect that the gradient of the error should be high, since we are basically telling the discriminator that it's making a mistake (and a big one, if the discriminator loss was low), so the error gradient should be high, and this gradient will be the one going to the generator for training.

rand
  • 427
  • 1
  • 5
  • 10

1 Answers1

2

You might find the answer in this paper "Towards principled methods for training generative adversarial networks" (https://arxiv.org/pdf/1701.04862.pdf). It has a part explaining why the generator's gradient vanishes as the discriminator gets stronger.

kangzheng
  • 36
  • 2
  • 4
    Thanks! I'm accepting the answer, although I think you should actually explain it rather than just linking the paper. – rand Apr 11 '18 at 13:33
  • In fact, I was looking for the answer myself and found this paper. I haven't finished reading it yet. lol. – kangzheng Apr 13 '18 at 03:07