Training on cifar100

Question

I am building a model on training a cifar100 images

I try to follow others work on training cifar10 and they work very well(around 90% accuracy), so I add some more layers for training cifar100.But the val accuracy is stuck at around 50 to 55%, here is my code:

(x_train, y_train) , (x_test, y_test) = cifar100.load_data()
y_train = k.utils.to_categorical(y_train, number_class)
y_test = k.utils.to_categorical(y_test, number_class)
x_train = x_train.astype('float32')
x_test = x_test.astype('float32')
x_train /= 255.
x_test /= 255.

datagen = ImageDataGenerator(
    rotation_range=15,
    width_shift_range=0.1,
    height_shift_range=0.1,
    horizontal_flip=True,
    )
datagen.fit(x_train)

l2 = 1e-4
kernel_size = (3,3)
model = Sequential()
model.add(Conv2D(32,kernel_size,padding = 'same',kernel_regularizer=regularizers.l2(l2),input_shape=(32,32,3),name = 'block1_conv1'))
model.add(Activation('elu',name = 'block1_elu1'))
model.add(BatchNormalization())
model.add(Conv2D(32,kernel_size,padding = 'same',kernel_regularizer=regularizers.l2(l2),name = 'block1_conv2'))
model.add(Activation('elu',name = 'block1_elu2'))
model.add(BatchNormalization())
model.add(MaxPooling2D(2,name = 'block1_pool'))

model.add(Conv2D(64,kernel_size,padding = 'same',kernel_regularizer=regularizers.l2(l2),name = 'block2_conv1' ))
model.add(Activation('elu',name = 'block2_elu1'))
model.add(BatchNormalization())
model.add(Conv2D(64,kernel_size,padding = 'same',kernel_regularizer=regularizers.l2(l2),name = 'block2_conv2' ))
model.add(Activation('elu',name = 'block2_elu2'))
model.add(BatchNormalization())
model.add(MaxPooling2D(2,name = 'block2_pool'))

model.add(Conv2D(128,kernel_size,padding = 'same',kernel_regularizer=regularizers.l2(l2),name = 'block3_conv1' ))
model.add(Activation('elu',name = 'block3_elu1'))
model.add(BatchNormalization())
model.add(Conv2D(128,kernel_size,padding = 'same',kernel_regularizer=regularizers.l2(l2),name = 'block3_conv2' ))
model.add(Activation('elu',name = 'block3_elu2'))
model.add(BatchNormalization())
model.add(Conv2D(128,kernel_size,padding = 'same',kernel_regularizer=regularizers.l2(l2),name = 'block3_conv3' ))
model.add(Activation('elu',name = 'block3_elu3'))
model.add(BatchNormalization())
model.add(MaxPooling2D(2,name = 'block3_pool'))


model.add(Flatten(name = 'flatten'))
model.add(Dense(256, activation='elu', name = 'fc1'))
model.add(Dense(number_class, activation = 'softmax', name = 'pred'))
opt = k.optimizers.adam(lr=0.0001, decay=1e-6)
model.compile(loss='categorical_crossentropy',
              optimizer=opt,
              metrics=['accuracy'])
model.fit_generator(datagen.flow(x_train, y_train, batch_size=64),
                    epochs=40,
                    verbose=1,
                    validation_data=(x_test, y_test))

But the result is show below:

Epoch 31/40
782/782 [==============================] - 26s 34ms/step - loss: 1.2475 - acc: 0.6589 - val_loss: 1.8069 - val_acc: 0.5494
Epoch 32/40
782/782 [==============================] - 26s 34ms/step - loss: 1.2224 - acc: 0.6661 - val_loss: 1.8270 - val_acc: 0.5484
Epoch 33/40
782/782 [==============================] - 26s 34ms/step - loss: 1.2002 - acc: 0.6713 - val_loss: 1.9344 - val_acc: 0.5311
Epoch 34/40
782/782 [==============================] - 26s 34ms/step - loss: 1.1767 - acc: 0.6769 - val_loss: 1.8654 - val_acc: 0.5421
Epoch 35/40
782/782 [==============================] - 26s 34ms/step - loss: 1.1582 - acc: 0.6840 - val_loss: 1.8809 - val_acc: 0.5414
Epoch 36/40
782/782 [==============================] - 27s 34ms/step - loss: 1.1398 - acc: 0.6877 - val_loss: 1.8144 - val_acc: 0.5505
Epoch 37/40
782/782 [==============================] - 27s 34ms/step - loss: 1.1199 - acc: 0.6935 - val_loss: 1.8434 - val_acc: 0.5404
Epoch 38/40
782/782 [==============================] - 27s 34ms/step - loss: 1.1055 - acc: 0.6949 - val_loss: 1.8856 - val_acc: 0.5479
Epoch 39/40
782/782 [==============================] - 27s 34ms/step - loss: 1.0862 - acc: 0.7012 - val_loss: 1.8355 - val_acc: 0.5554
Epoch 40/40
782/782 [==============================] - 26s 33ms/step - loss: 1.0660 - acc: 0.7051 - val_loss: 1.8502 - val_acc: 0.5499

I tried to train more but the val_loss is increasing but val_accuracy remain unchanged

Is there any suggestion about this? Thank you!

@DeltaIV my problem probably relates to data augmentation issues or too many regularizers/normalisation, am I right? — cwl6, Jan 26 '19 at 18:26
Your question is not a good fit for this site, and you have been redirected to the Q&A that best suits your need, while remaining on-topic for CV. If you still can't solve your issue, check the rules of https://datascience.stackexchange.com/ o https://ai.stackexchange.com/, and verify if rewriting your question as a [MCVE](https://stackoverflow.com/help/mcve) would be acceptable for one of those sites. I'd rather not try to ask on SO because you claim the code is working. If the question, rewritten as a MCVE, is not acceptable also on DS and AI, then it's just not a good fit for Stack Exchange — DeltaIV, Jan 27 '19 at 11:33

Training on cifar100

0 Answers0