I'm trying to clasify author age group ('under 21','young adult', 'adult). I'm working on this example: https://colab.research.google.com/github/tensorflow/text/blob/master/docs/tutorials/classify_text_with_bert.ipynb#scrollTo=KiC1mzAnBR5Y.
I have provided my train, test and validation data:
- training: 31738 files belonging to 3 classes.
- validation: 3968 files belonging to 3 classes.
- test: 3966 files belonging to 3 classes.
I have changed:
- net = tf.keras.layers.Dense(3, activation=None, name='classifier')(net)
- classifier_model.compile(optimizer=optimizer, loss='sparse_categorical_crossentropy', metrics=['accuracy'])
I have tried:
- changing dropout rate
- changing batch size
- changing lerning rate
- different model (talking-heads_base)
- different optimizer (adam/adamw)
- different loss function(mae)
- addding activation function('relu')
**
The accuracy is always near 0.3333 while loss function is decreasing
** as you can see in chart:. validation loss/test loss/accuracy
