4

Is it really necessary to use a validation set to avoide overfitting while we are using Dropout ?

Richard Hardy
  • 54,375
  • 10
  • 95
  • 219
Ka_
  • 55
  • 4
  • Without comparison to a validation set, how would you know that you've successfully prevented overfitting? The probability of masking a unit can be anything between 0 and 1. Is masking masking probability $10^{-10}$ too small (model overfits)? Or too large (model underfits)? How can you know? – Sycorax Aug 11 '20 at 15:23
  • I did not get the idea of masking units can u please expalin more – Ka_ Aug 12 '20 at 19:28
  • masking units at random is how dropout works. more information: https://stats.stackexchange.com/questions/241645/how-to-explain-dropout-regularization-in-simple-terms/241668#241668 – Sycorax Aug 12 '20 at 19:29
  • So did u mean the validation set will help us to find the suitable dropout probability for our model – Ka_ Aug 12 '20 at 19:53

1 Answers1

4

Like typical regularization, dropout is a mechanism to fight overfitting. It doesn't detect it. On the contrary, using a validation set helps you detect it. So, the two have different uses.

gunes
  • 49,700
  • 3
  • 39
  • 75
  • So if we are using Dropout we expect not having an over-fitted model.in this case do we really need to use a validation set ? – Ka_ Aug 11 '20 at 14:27
  • 2
    Dropout fights over-fitting, but it may not prevent it (it's not a silver bullet). Validation set is used for detecting/deciding the overfitting. So, you should use it nevertheless. – gunes Aug 11 '20 at 14:40