0

I'm attempting to create a Neural Network to use as a rain model to generate representative precipitation levels for a given location.

The problem I'm facing is that the network always converges to the average amount of rain. For example, the trained network predicts that it will be constantly raining a small amount all the time, instead of predicting that at some times it will be raining and sometimes it won't.

Convergence on average through training: (green - actual, blue - predicted) Neural Network Precipitation Training

Is it reasonable to expect a neural network to be able to predict like this? If so, what can be done to allow the network to learn in this way?

(I'm using DL4J, one hidden layer with 5 nodes, squared loss function, month and hour as inputs (normalised))

DL4J NeuralNetConfiguration

MultiLayerConfiguration modelConfiguration = new NeuralNetConfiguration.Builder()
    .seed(seed)
    .iterations(1)
    .optimizationAlgo(OptimizationAlgorithm.STOCHASTIC_GRADIENT_DESCENT)
    .learningRate(0.0001)
    .updater(Updater.NESTEROVS)
    .list()
    .layer(0, new DenseLayer.Builder()
        .nIn(4)
        .nOut(5)
        .weightInit(WeightInit.XAVIER)
        .activation(Activation.LEAKYRELU)
        .build())
    .layer(1, new OutputLayer.Builder(LossFunctions.LossFunction.SQUARED_LOSS)
        .weightInit(WeightInit.XAVIER)
        .activation(Activation.IDENTITY)
        .nIn(5)
        .nOut(1)
        .build())
    .pretrain(false)
    .backprop(true)
    .build();
parrowdice
  • 121
  • 2
  • What type of neural network are you using? What features is it based on? – tagoma Jan 23 '18 at 09:07
  • Sounds like you are using a simple fully connected neural network. This does not use your prior knowledge about the distribution of rain over time. Perhaps you should try a recurrent neural network. – elliotp Jan 23 '18 at 09:20
  • @tagoma To be honest, I'm not sure what 'type' of neural network you'd call it, I'm fairly new to this. I've included the code I'm using though. The features are month and hour normalised in a cyclic way using sin/cos. – parrowdice Jan 23 '18 at 11:34
  • Don't start with a neural network, try a a simpler times series model like SARIMAX. Honestly, it's much easier to debug and you can interpret the model results. – MachineEpsilon Jan 23 '18 at 13:23

0 Answers0