What should be examined when feedforward neural network does not fit the training set well?

Asked Jun 26 '17 at 15:36

Active Jun 26 '17 at 15:47

Viewed 27 times

In theory, feed forward neural network should be able to approximate any smooth functions given enough nodes in the hidden layer. In other words, it could significantly overfit the training data set. I was expecting feed forward NNets to fit the training set with a sign accuracy close to 100% without any penalization, but for some reason, the accuracy gets quite low from time to time. Can anyone provide some guidance why training accuracy can be very low, even when there are enough hidden layers?

Edit: As @ThomasW pointed out, one of the possible causes is local minima. what are some other causes? Is there a way to tell whether a specific cause is indeed the problem?

edited Jun 26 '17 at 15:47

asked Jun 26 '17 at 15:36

denizen of the north

Pls give more details, such as target and model structure – danche354 Jun 26 '17 at 15:39
@danche i dont have a specific target function in mind. This is more of a conceptual question. – denizen of the north Jun 26 '17 at 15:41
@user1559897 it doesn't seem like a conceptual question. `the accuracy gets quite low from time to time. ` can be very problem specific (local minima) – Thomas Wagenaar Jun 26 '17 at 15:42
@ThomasW Well, you just mentioned one of the possible causes I am asking for: local minima. What are some other causes? Is there a way to tell whether a specific cause is indeed the problem? – denizen of the north Jun 26 '17 at 15:45

What should be examined when feedforward neural network does not fit the training set well?

0 Answers0