1

I am currently working on a binary time-series classification problem using the keras deep learning library. The dataset that I am working with is heavily imbalanced as you can see in the table below:

Counts
Positive Class 3,615
Negative Class 38,757

I've read in many places of people using SMOTE or ADASYN to solve the problem of class imbalance in their datasets and to make the classifier be able to recognize the minority class. As such, I've tried to apply that using the imblearn Python library. The current approach to the data preprocessing that I am following can be seen in the diagram below:

Data preprocessing procedure

With this, a very simple 2D convolutional neural network is trained and the loss-epoch curve is as follows:

enter image description here

I have tried many solutions to this problem such as the addition of dropout, batch normalization, increasing, and decreasing the size of the network but despite that nothing seems to help.

Is there anything that I can do to solve the problem of overfitting when using ADASYN or SMOTE with this dataset that I have?

Joseph Anderson
  • 31
  • 1
  • Class imbalance isn’t actually a problem! https://stats.stackexchange.com/questions/357466/are-unbalanced-datasets-problematic-and-how-does-oversampling-purport-to-he – Dave Feb 11 '21 at 13:15
  • You tend to get these types of problems with synthetic data generating processes. Instead of using this package, have you tried: (1) altering your loss function with weights to make it more costly for making mistakes, and (2) sampling the actual data you have with replacement. Pytorch, for instance, has a sampler that will sample from your data (with replacement) to do its best to create even batches. – John Stud Feb 11 '21 at 13:23
  • @Dave Thank you for the thread. I've just read it, but my problem is in the fact that when performing the cross-validation, the performance of the model is quite poor especially when using the F1 score, FPR, and Recall metrics. What I believe happens is that entire batches end up only having instances of the negative class leading the model to get very little (if any at all) training on the minority class. – Joseph Anderson Feb 11 '21 at 13:23
  • @JohnStud I actually haven't tried these but thank you for the recommendation! I'm looking into them now and trying to see how it works on the dataset I have. – Joseph Anderson Feb 11 '21 at 13:24
  • F1, FPR, and recall all are threshold-based metrics, thus improper scoring rules like accuracy is. How is performance on Brier score or log loss? – Dave Feb 11 '21 at 13:34

0 Answers0