How to add custom weights for each object for instance segmentation?

[agnim25]

🚀 Feature

I'd like to add weights for each object/class considered in training detectron2 to handle data imbalance. Is there a way to do this in the current detectron2 implementation?

[ppwwyyxx]

https://detectron2.readthedocs.io/en/latest/modules/data.html#detectron2.data.samplers.RepeatFactorTrainingSampler is used by all the LVIS configs https://github.com/facebookresearch/detectron2/tree/master/configs/LVISv1-InstanceSegmentation for this purpose. You can use other samplers to do other types of weighting.

[keithkctse]

I see another answer here,
https://stackoverflow.com/questions/64450632/how-set-custom-class-weights-in-detectron2

However, I am not sure how to do this in Instance Segmentation. I tried to modify the ROI_MASK_HEAD but that doesn't work similar to the SEM_SEG_HEADS. In ROI_MASK_HEAD, they use binary_cross_entropy_with_logits instead of CrossEntropyLoss in SEM_SEG_HEADS
putting the class weights throws me an error saying that the class weights does not fit the size of input and target.

[amroybd]

Hi, I can see that you posted a long time ago. But just to check whether did you find the solution for instance segmentation or not?

[amroybd]

Feature

I'd like to add weights for each object/class considered in training detectron2 to handle data imbalance. Is there a way to do this in the current detectron2 implementation?

Hi, did you find the solution yet? Thank you

[agnim25]

I ended up using this sampler on my dataset: https://detectron2.readthedocs.io/en/latest/modules/data.html#detectron2.data.samplers.RepeatFactorTrainingSampler

[amroybd]

Hi, Thank you for your reply. I have used something like below:

cfg = get_cfg()
# minimum image size for the train set
cfg.INPUT.MIN_SIZE_TRAIN = (1000,)
# maximum image size for the train set
cfg.INPUT.MAX_SIZE_TRAIN = 3000
# minimum image size for the test set
cfg.INPUT.MIN_SIZE_TEST = 1000
# maximum image size for the test set
cfg.INPUT.MAX_SIZE_TEST = 3000
cfg.merge_from_file(model_zoo.get_config_file("COCO-InstanceSegmentation/mask_rcnn_X_101_32x8d_FPN_3x.yaml")) #r_152
cfg.DATASETS.TRAIN = ("mydata_train",)
cfg.DATASETS.TEST = ("mydata_test",)
cfg.DATALOADER.SAMPLER_TRAIN = "RepeatFactorTrainingSampler "
cfg.DATALOADER.REPEAT_THRESHOLD = 0.001
cfg.DATALOADER.NUM_WORKERS = 4
cfg.TEST.EVAL_PERIOD = 500
cfg.MODEL.WEIGHTS = model_zoo.get_checkpoint_url("COCO-InstanceSegmentation/mask_rcnn_X_101_32x8d_FPN_3x.yaml")
cfg.SOLVER.IMS_PER_BATCH = 4
cfg.SOLVER.BASE_LR = 0.0005
cfg.SOLVER.STEPS = []
cfg.SOLVER.MAX_ITER = 7000
cfg.MODEL.ANCHOR_GENERATOR.SIZES = [[8, 16, 32, 64, 128, 256, 512, 1024]]
cfg.MODEL.ANCHOR_GENERATOR.ASPECT_RATIOS = [[0.25, 0.5, 1.0, 1.5, 2.0, 3]]
cfg.MODEL.ROI_HEADS.NUM_CLASSES = 51

os.makedirs(cfg.OUTPUT_DIR, exist_ok=True)
trainer = DefaultTrainer(cfg) 
trainer.resume_or_load(resume=False)
trainer.train()

After using the RepeatFactorTrainingSampler with "REPEAT_THRESHOLD = 0 . 001", I improved the class imbalance problem a little bit.
1- So the parameter "repeat_factors" in the document you have provided is the same as "REPEAT_THRESHOLD"? Could you please give me a coding example or how you have implemented it for your case? That would be very helpful.

2- Do you know any way to give each class a customized weight in detectron2?

Again thank you for your time and for replying to me.