Can anyone provide me the mathematical details to why we shouldn't initialize a neural network (single layer ANN) weights to zero? In multi-layer we don't initialize it to zero because of loss of symmetry but why can't we initialize a single layer network weights to zero ?
Asked
Active
Viewed 493 times
1
-
Papers like https://arxiv.org/abs/1812.03425 shows that weight initialization to zero could lead to better results. – Vedanshu Dec 17 '18 at 12:50
-
Single layer meaning single hidden layer? Or single layer meaning logistic regression? – Jan Kukacka Dec 17 '18 at 14:19
-
@JanKukacka single hidden layer or may be a dense single layer at the output of a model. – Vedanshu Dec 17 '18 at 14:20