0

I am training an LSTM network using Tensorflow 2, is there a way to debug it to see if its learning or to know what areas should be adjusted ?
Is there a way to debug to know if its the data, the model or the hyper-parameters ?
Is there a way to have more informative approach to what to work on in LSTM to make it better ?
Simple model:

single_step_model = tf.keras.models.Sequential()
single_step_model.add(tf.keras.layers.LSTM(32,
                                           input_shape=x_train_single.shape[-2:]))
single_step_model.add(tf.keras.layers.Dense(1))

single_step_model.compile(optimizer=tf.keras.optimizers.RMSprop(), loss='mae')

And the loss:

Train for 200 steps, validate for 50 steps
Epoch 1/10
200/200 [==============================] - 1s 7ms/step - loss: 0.0332 - val_loss: 0.0766
Epoch 2/10
200/200 [==============================] - 1s 6ms/step - loss: 0.0290 - val_loss: 0.0746
Epoch 3/10
200/200 [==============================] - 1s 6ms/step - loss: 0.0271 - val_loss: 0.0328
Epoch 4/10
200/200 [==============================] - 1s 6ms/step - loss: 0.0280 - val_loss: 0.0270
Epoch 5/10
200/200 [==============================] - 1s 6ms/step - loss: 0.0258 - val_loss: 0.0328
Epoch 6/10
200/200 [==============================] - 1s 6ms/step - loss: 0.0249 - val_loss: 0.0274
Epoch 7/10
200/200 [==============================] - 1s 6ms/step - loss: 0.0345 - val_loss: 0.0364
Epoch 8/10
200/200 [==============================] - 1s 6ms/step - loss: 0.0276 - val_loss: 0.0445
Epoch 9/10
200/200 [==============================] - 1s 5ms/step - loss: 0.0273 - val_loss: 0.0430
Epoch 10/10
200/200 [==============================] - 1s 6ms/step - loss: 0.0273 - val_loss: 0.0428
```
Ramzy
  • 43
  • 1
  • 7
  • that is a very broad question. How does your data look like? Also, it seems like your loss function is going down this each epoch – aaaaa says reinstate Monica Feb 04 '20 at 00:14
  • Yes, broad question because i am asking about guidelines as i am working on based on this tutorial https://www.tensorflow.org/tutorials/structured_data/time_series – Ramzy Feb 04 '20 at 00:17
  • I used 10 epochs only, because i went for 25 but after the 10 epoch the loss stays the same – Ramzy Feb 04 '20 at 00:20
  • i have ran into it already but i was hoping for something to be looked into. more practical than a literature. like if there is a way to debug or investigate if you'd say. – Ramzy Feb 04 '20 at 01:25
  • The thread contains numerous suggestions and examples of things to try to improve a network. – Sycorax Feb 04 '20 at 01:35

0 Answers0