Neural network training better without MinMaxScaler()?

Question

Forum,

I have a multivariate time series problem; For my master thesis I am investigating whether it is possible to forecast the movement direction of stock price with machine learning. My model looks as follows:

def sentdex_model(X_train):
    model = Sequential()
    model.add(LSTM(33, input_shape=(X_train.shape[1:]), return_sequences=True))
    model.add(LSTM(33, input_shape=(X_train.shape[1:])))
    model.add(Dense(90, activation='relu'))
    model.add(Dense(1, activation='sigmoid'))
    model.compile(loss='binary_crossentropy', optimizer='rmsprop', metrics=['accuracy'])
    # model.summary()
    return model

The input data is in the form of [#samples, timesteps, features]. The features are OHLCV (Open-High-Low-Close-Volume) data of 6 different telecom companies. Im trying to predict whether a stock will rise (1) or fall (0). So it is basically a time series classification problem. I've always learned that, when data is presented on very different intervals - it is good practice to MinMaxScale the input data into the neural network.

However, when I do so, the train accuracy of the model keeps hovering around the baseline of 0.50 (There is an equal amount of 1s (price rise) and 0s (price fall). So, the model is not really learning. Now, when i dont MinMaxScale, accuracy slowly climbs to around 75% over 50 epochs.

Can anyone explain why the model without MinMaxScaling seems to learn better than the model with MinMaxScaling?

Answer 1

Training accuracy simply describes how well a model can fit the training data, but it doesn't explain how well your model will generalise to unseen data. After each epoch evaluate accuracy on a validation set which the model never trains on. You may very well find that without MinMaxScaler the validation accuracy is worse than with.

Additionally MinMaxScaler isn't always the best choice for scaling. Often standardization is preferable as it enforces that features have a mean of zero and unit variance. It looks like you're using Sklearn for the preprocessing, and the corresponding standardization scaler in Sklearn is StandardScaler.

If you discover that the validation accuracy is significantly worse than the training accuracy for all epochs it implies your model cannot fit your data well enough. This suggests you should try introducing more layers dense layers as well as BatchNormalization after all but the last dense layer.

Data scaling is always beneficial and I can't think of a case where it will worsen accuracy in a sufficiently expressive model. So to that extent I think the better accuracy on training without scaling is just an coincidental artefact of saturation in your nonlinear activations from unscaled inputs coupled with a network architecture that is not complex enough to capture your problem.

Also in a binary classification situation, an accuracy of 50% can imply that the network isn't actually learning anything.

To summarize: evaluate accuracy on the validation set, then try StandardScaler, and then try introducing more dense layers and regularization techniques like BatchNormalization.