0

More of a curiosity, but I'm currently learning how to deal with imbalanced datasets and came across the SMOTE method to bias the minority class. The images below show before and after SMOTE was applied to the minority class (images taken from here: https://machinelearningmastery.com/smote-oversampling-for-imbalanced-classification/).

The idea seems to be that the algorithm tries to fill in the area/volume that the data occupies, but the way the algorithm works, it can only fill between the existing points.

My question is, would it be at all beneficial to oversample in an iterative manner using SMOTE? Where the subsequent iterations are applied to the original data and generated data, which would allow the algorithm to fill between purely generated data. Would this lead to strange unwanted biases? Or would it simply have no effect over the usual way to apply SMOTE?

Before SMOTE

After SMOTE

ryan132442
  • 361
  • 4
  • 2
    No, iteratively oversampling will not be beneficial. Even a single oversampling run is mistaken. Unbalanced classes are almost certainly not a problem, and oversampling will not solve a non-problem: [Are unbalanced datasets problematic, and (how) does oversampling (purport to) help?](https://stats.stackexchange.com/q/357466/1352) – Stephan Kolassa Nov 16 '21 at 14:51
  • @Stephan Kolassa Thanks for the link. I have had a read through and it still seems to me that oversampling does have a place? From what I understand, the issue does not stem from the fact that the data is simply imbalanced (which intuitively seems reasonable to me), but the issue is whether the algorithm used has a tendency to ignore the minority class if it improves model accuracy. So then following logical reasoning, if it was known that the model in use may ignore the minority class, then oversampling is needed to weight the minority class heavily so that the model cannot overlook it? – ryan132442 Nov 16 '21 at 15:56
  • No. The issue is that using accuracy will mislead you. Work with probabilistic classifications and aim for correct calibration. Then there is no issue whatsoever with "balance" or "minority classes" - one class quite simply has a lower probability of occurring, and that is all there is. And your probabilistic classifications will correctly reflect this. – Stephan Kolassa Nov 16 '21 at 16:33
  • @StephanKolassa I'll take a look at probabilistic classification now, thanks. The problem I had in mind was fraud detection practice project and my dataset is imbalanced 600:1. I can see that it would not be useful to use accuracy here as a model measure because the model would be very accurate without any predictive power. My plan was to evaluate the model with a confusion matrix, since I should care a lot about reducing false negatives specifically. I figured that oversampling here would allow me to bias the model towards the fraud examples and reduce the false negative rate. – ryan132442 Nov 16 '21 at 16:43
  • Fraud detection *especially* is a case where you need to distinguish [between the probabilistic classification and the subsequent decision](https://stats.stackexchange.com/a/312124/1352), because the costs are *highly* asymmetrical. Not detecting a fraudster is painful. But wrongly accusing an innocent person of committing fraud can ruin that person (and incidentally become extremely expensive if the victim has a good lawyer). ... – Stephan Kolassa Nov 16 '21 at 16:51
  • ... Plus, there are more than two possible actions here (classify as "fraud" or "non-fraud"), namely "investigate more deeply", "collect more information". In practice, a hard 0-1 classification makes even less sense in fraud detection than elsewhere. Yes, I understand that you appear to be doing a self-study project here - but these points are why I believe that fraud detection is an especially unfortunate choice for learning classification. – Stephan Kolassa Nov 16 '21 at 16:53

0 Answers0