0

I have a NN that I would like to square a number. This is a learning exercise for me.

My input is the number to be squared, the output is the square.

Two questions: 1) How can this possibly work? The weights and nodes of the NN need to square to a number that isn't fixed.

2) Assuming I am wrong, what is a strategy for choosing the numbers of nodes and layers for a NN?

Tony Ennis
  • 103
  • 4
  • 1
    As an example: https://stats.stackexchange.com/questions/299915/how-does-the-rectified-linear-unit-relu-activation-function-produce-non-linear/299933#299933 but a necessary unstated component t to your question is what amount of precision you want in the result; the *universal **approximation** theorem* lays out technical criteria for NNs to approximate specific functions. – Sycorax Jun 24 '19 at 18:29

1 Answers1

1

The ReLU activation function should take care of this.

ReLU works by fitting short, straight lines to approximate curves. That should be able to create a parabola. You will have performance suffer for inputs with very large absolute values, but we know that models won't be perfect.

I was thinking that one hidden layer could take care of this, but reading about the universal approximation theorem (which I suggest doing), we can be more efficient by having fewer nodes in multiple hidden layers than tons of nodes in one hidden layer.

Dave
  • 28,473
  • 4
  • 52
  • 104