I'm training a neural machine translator using the encoder-decoder approach and I've got around 95% validation accuracy and 0.4 validation loss. The model is translating correctly for most of the sentences in the validation set but the problem is if I give sentences that are not in the dataset the predictions by the model are way too different from the actual translation.
model = Sequential()
model.add(Embedding(in_vocab, units, input_length=in_timesteps, mask_zero=True))
model.add(LSTM(units))
model.add(RepeatVector(out_timesteps))
model.add(LSTM(units, return_sequences=True))
model.add(Dense(out_vocab, activation='softmax'))
return model
This is the model I'm training for 100,000 sentence pairs.
I've changed the hyperparameters and tried different times but I'm getting same observation
