Training over-fitting after every epochs

[songshan0321]

I use train.py to train on my relabel ade20k dataset (150 to 5 classes) but couldn't get good result. mIoU is always at maximum 57%.

My training parameters:

• Batch Size: 32

After plotting out the train loss and val loss graph, I found that the loss pattern is suspicious, it is shown below. Model is overfitting regularly after every epoch and I assume that's the reason why my model can't hit a higher mIoU. I have no idea why.