0

I have taken text input then converted to a sequence of values and fed it to LSTM model where my loss is not reducing and accuracy is abnormal. enter image description here

The above image is about training and validation accuracy. Here I have taken 10 epochs but even for 100 epochs or 1000 epochs the graph is similar.

enter image description here

The above image is about training and validation loss, there is no change in it. No reduction in loss.

How can I make my model learn.

The considered learning rate is 0.05

Model Architecture is

model = tf.keras.Sequential([tf.keras.layers.Embedding(vocab_size, embedding_dim),tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(embedding_dim,dropout=0.3)),tf.keras.layers.Dense(embedding_dim, activation='relu'), tf.keras.layers.Dense(2, activation='softmax')])
SS Varshini
  • 159
  • 1
  • 8
  • Are u cleaning the gradient to avoid gradient clipping? –  Dec 17 '20 at 13:03
  • No, Not clearing gradients – SS Varshini Dec 17 '20 at 13:35
  • 1
    This can make your model unable to retro propagate through time, once it uses THAN and Sigmoid to forget, update and filter the input / output; which then causes your weights to vanish, i.e the impact on the learning of the model is decreased on each iteration time, through the loss and backpropagation. Try to clean it on each batch iteration. –  Dec 17 '20 at 13:41
  • See https://stats.stackexchange.com/questions/352036/what-should-i-do-when-my-neural-network-doesnt-learn – kjetil b halvorsen Dec 17 '20 at 14:42
  • I understood that weights are not updating, how I can train my model? – SS Varshini Dec 18 '20 at 08:07
  • 1
    Does this answer your question? [What should I do when my neural network doesn't learn?](https://stats.stackexchange.com/questions/352036/what-should-i-do-when-my-neural-network-doesnt-learn) – pdpino May 04 '21 at 21:46

0 Answers0