1

I'm trying to implement drop connect. Am I supposed to use the same drop masks during back propagation?

piRSquared
  • 251
  • 1
  • 10

1 Answers1

1

Yes, because back propagation is for computing gradients.

If some connection is blocked by the mask, it contributes nothing to the loss, so its associated gradient should be zero.

dontloo
  • 13,692
  • 7
  • 51
  • 80