Tips for Unbalanced Datasets

[lwachowiak]

I am working with an unbalanced dataset (class A 84% of samples, class B 11%, class C 5%).
So far the accuracy of the trained model does not exceed that of a simple majority classifier by much. In your experience, what sampling techniques led improved results? My training set also has not a huge amount of samples (4k generated with a sliding window).

Did you have success with techniques like oversampling, under sampling, or SMOTE?
Did a certain model type perform best with highly unbalanced classes?

Thanks for any tips in advance!

[vrodriguezf]

I had better experiences with weighting the loss (weighting the samples themselves or the classes), or changing it to another loss such as the focal loss.