2

Using the model from theano's tutorial, I'm training a 3-layers perceptron with log returns over a very large dataset (~55,000 points). The output's layer contains two neurons, one for each of the classes (either the returns are + or -) in which I am trying to cluster my inputs (returns for the past n days).

The machine always predicts the same value, ie the output which has the highest probability over the entire dataset. Has anyone faced a similar problem before? How is this even possible?

Inputs: 100d-array of log returns (quasi normally distributed).

Target Output: sign of consecutive log return (1 or 0).

kenorb
  • 549
  • 2
  • 7
  • 19
Rackham Le Rouge
  • 21
  • 4
  • can you post a snapshot of what your data looks like, preferably with a few examples from each class? – Alex R. Jun 22 '16 at 21:28
  • @AlexR. a typical point in the data set would be the tuple: (array([-0.072, -0.207, ...,0.086]),1), meaning that after the given series of log returns, the given asset went up. – Rackham Le Rouge Jun 22 '16 at 21:51
  • Does that mean you sum the first array, and see if it's positive or negative? – Alex R. Jun 22 '16 at 22:03
  • How balanced is your dataset? Maybe one of classes dominates over the other and the network exploits this. – Dr. Snoopy Jun 23 '16 at 00:44
  • @MatiasValdenegro my set is large enough, log returns are normally distributed around 0 – Rackham Le Rouge Jun 23 '16 at 00:58
  • @RackhamLeRouge I am not talking about dataset size, but the distribution of your labels. – Dr. Snoopy Jun 23 '16 at 08:28
  • One neuron with softmax should be enough for a 2-class problem. What loss function are you using? Also, sometimes normalizing the input solves tjis issue. – mbiron Jun 23 '16 at 13:37
  • @mbiron input set is already almost normally distributed. My loss function is the negative log likelihood, although I also tried to just take 1-softmax(actual output) to get the same results. If I only take one output neuron, how would I distinguish between the two classes? Simply by looking at the magnitude of the softmax? – Rackham Le Rouge Jun 23 '16 at 22:21
  • Please clarify: you are using [this exact code](http://deeplearning.net/tutorial/mlp.html), but in a different but similar dataset, and the results are very different. Is this a fair summary of what is going on? – Ricardo Magalhães Cruz Jun 29 '16 at 02:00

0 Answers0