How to prevent the keras convolutional neural network model to over-fit?

Question

I want to build a convolutional neural network and train it to recognise whether the digit is 0 or 1.

Example of my training data is a 800 * 600 gray scale image containing a digit one: I have 22 such images, 11 containing zeroes, 11 containing ones.

I now build a convolutional nerual network:

def addConvolutionalLayer(layer, newDepth = None):
    print(layer.get_shape().as_list())
    if newDepth is None:
        newDepth = layer.get_shape().as_list()[3] * 2
    newLayer = Conv2D(newDepth, kernel_size = (3, 3),
                activation = 'relu', padding='same') (layer)
    newLayer = Dropout(0.2) (newLayer)
    return Conv2D(newDepth, kernel_size = (3, 3),
        activation = 'relu', padding='same') (newLayer)

def buildModel():
    s = Input((600, 800, 1))
    d1 = addConvolutionalLayer(s, 32)
    d2 = addConvolutionalLayer(MaxPooling2D(4)(d1))
    d3 = addConvolutionalLayer(MaxPooling2D(4)(d2))
    d4 = addConvolutionalLayer(MaxPooling2D(4)(d3))
    f = Flatten()(d4)
    dense = Dense(64, activation='relu') (f)
    o = Dense(1, activation='sigmoid') (dense)
    model = Model(inputs = [s], outputs = [o])
    model.compile(optimizer='adam', loss='binary_crossentropy',
              metrics=['accuracy'])
    return model

model = buildModel()
model.compile(optimizer='adam',
        loss='binary_crossentropy',
        metrics=['accuracy'])

In short, a network consists of multiple convolutional layers (with MaxPooling included), ending with a dense layer.

I then split the data into training and test sets:

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3)

X_train, X_test are arrays of float values, each value corresponding to the pixel in the image (1.0 for white, 0.0 for black).

y_train, y_test are lists containing 0 or 1 depending on the digit in the corresponding image.

Finally, I run the model to train:

model.fit(X_train, y_train, epochs = 50, validation_split=0.2)
model.save('prediction.h5')

The problem is that the model gets over-fitted: it reaches 100% accuracy on the training data, but does not perform well on the validation set:

Epoch 95/100
12/12 [==============================] - 0s 32ms/step - loss: 1.0640e-07 -
    acc: 1.0000 - val_loss: 10.7454 - val_acc: 0.3333

How to fix this issue?

score 0 · Answer 1 · answered Nov 06 '19 at 12:23

First thing you could try is that remove Dropout in ConvBase and add before dense layers (need to experiment a bit on dropout percentage value), then experiment with different pre-processing techniques to augment the data. Most importantly check no.of parameters by getting summary of network and try to reduce the no.of parameters in case if no.of parameters are high.

How to prevent the keras convolutional neural network model to over-fit?

1 Answers1