2

From what I understand, a one layer network (no hidden layers) is exactly the same as logistic regression.

Now suppose I have a function $f(x)$ that is exactly a logistic regression function with say d features. If I insist on running a neural network with say 3 layers with d/3 neurons each, is this a bad idea? That is, will it:

  1. Take much longer to train such a network?
  2. The final optimum weights produce a function that is not a great approximation to the original function? (Note the regression model reproduces f(x) exactly).

This would be a hypothetical example of a situation where naively using deep learning is worse than shallow learning so to say.

gung - Reinstate Monica
  • 132,789
  • 81
  • 357
  • 650
Femto
  • 21
  • 1
  • With small sample sizes use logistic regression, neural network needs huge sample size! – kjetil b halvorsen Apr 12 '17 at 12:06
  • So if I have less amount of data, you are saying it is better to use logistic regression (i.e. a shallow neural network) with few features than deep neural networks with the same number of features ? Is this to avoid overfitting perhaps ? – Femto Apr 12 '17 at 12:10
  • a) It won't unless $d$ is large ($10^3$ and more) b) It is a much more complex model so it may overfit. (But only if you add noise to your data, otherwise it will give same result) Also i think you meant to say weights, or parameters (of neural network) rather than features - features are your inputs – Łukasz Grad Apr 12 '17 at 12:25
  • Related: http://stats.stackexchange.com/questions/43538/difference-between-logistic-regression-and-neural-networks – Ferdi Apr 12 '17 at 13:08

0 Answers0