1

I am new to LSTM and deep learning. I have 3000 reviews which I am trying to train on gensim pretrained model via word embedding. I have the following model where max_sequence_size=564.

max_sequence_size = max_features
classes_num = 1
my_model=Sequential()

my_model.add(Embedding(max_sequence_size,300,input_length=trainDataVecs.shape[1],weights=[embedding_matrix]))
my_model.add(Bidirectional(LSTM(100,activation='tanh',recurrent_dropout = 0.2, dropout = 0.2)))

my_model.add(Dense(50,activation='sigmoid'))
my_model.add(Dropout(0.2))


my_model.add(Dense(classes_num, activation='softmax'))

print "Compiling..."
my_model.compile(optimizer='adam',
              loss='binary_crossentropy',
              metrics=['accuracy'])

print my_model.summary()

I saw the link in which he is saying to change the optimizer which doesn't help.

Training loss goes down and up again. What is happening?

Should I add more layer since this might be shallow? Is there any way in which I can print the gradients?

amy
  • 131
  • 3
  • The linked to question says to gradually lower the learning rate, not change the optimizer. What values did you try, and in what way did it not help? (Also see the comment on that answer, as I think you are also using keras?) – Darren Cook Oct 05 '18 at 07:26
  • @DarrenCook I tried learning rate from 0.0001 to 0.1. No effect. I am not sure if there is any issue with defining the problem but can't find anything. – amy Oct 06 '18 at 02:34
  • @amy: did you find a solution? I am facing a similar issue – Corel Sep 07 '20 at 21:28

0 Answers0