We usually trained a model using balanced datasets. Even when we do not have a balanced datasets, we will use methods such as SMOTE to create a balanced dataset for training.
The question is - how reliable will be the trained model when it is implemented on an imbalanced datasets (e.g in the real world scenario, anomalies are usually rare)? Why can't we just train and test the model with an imbalanced dataset?
You can definitely train a model on an imbalanced dataset, however this often leads to deteriorated performance. Personally, I always do this so that I can measure the effect of applying SMOTE or related techniques.
In many cases, the minority class has too few representative samples to accurately model it leading to poor generalization performance of the resulting model. This is what methods such as SMOTE attempt to correct.
When testing your resulting model, you should not use your re-balanced set as this introduces bias and all sorts of other nasty issues. You want the test set to resemble the data your model will have to process in the wild as closely as possible
For more in-depth reading, I suggest the following paper:
