Train Faster-RCNN on Cityscapes

[pinxau1000]

Hello all,

I have run Mask R-CNN on cityscapes dataset without any issue, however now I want to run Faster R-CNN on the same dataset.

I'm using as reference the paper On Intra Video Coding And In-Loop Filtering For Neural Object Detection Networks where the authors state:

The same (Mask R-CNN) backbone is taken for Faster R-CNN, but since there is no pre-trained Cityscapes model available in Detectron2, the already existing model trained on the COCO dataset is taken from Detectron2 as initial weights. Subsequently, this model is further trained on the Cityscapes classes and training images, where the GT pixel masks have been converted to bounding boxes before, for 42000 iterations (...)

I'm asking for ideas for implementing this approach, i.e. re-train Faster R-CNN on Cityscapes dataset.

Note: I tried to use train_net.py with Faster R-CNN model on Cityscapes dataset and the error below was been thrown. I also tried to implement a custom dataloader and a custom evaluator based on plain_train_net.py which produced the same exception.

File ".venv/lib/python3.8/site-packages/detectron2/modeling/proposal_generator/rpn.py", line 470, in forward
    assert gt_instances is not None, "RPN requires gt_instances in training!"
AssertionError: RPN requires gt_instances in training!

Best regards
pinxau1000

[pinxau1000]

@ppwwyyxx Can you help me out?

[Chris1nexus]

Have you set the default MODEL.MASK_ON = False
to MODEL.MASK_ON = True ?
I had the same issue some time ago due to this default setting on the detectron2 cfg file