Can test-time ROI head NMS be done on masks rather than bounding boxes?

[alexander-soare]

I'm using MaskRCNN-R50. After a little bit of digging I realised NMS is being done via FastRCNNOutputLayers at which point there are only bounding boxes not masks. If I understand correctly, the mask heads are not dependent on the outcome of the NMS, ie each mask is produced based on the ROI from the proposal network. So based on that understanding, it's not an obvious choice to me, to use the bounding boxes for NMS.

My questions are:

1. Would it make sense to do NMS using the masks rather than the bounding boxes?
2. If so, is this behaviour implemented somewhere in Detectron2 or should I do it myself?

My motivation for wanting to do so can be seen in this example. If I set a low NMS threshold, one of these boxes might be suppressed if I am calculating bbox IOU. But the mask IOU is 0, so if I use mask IOU instead of bbox IOU the result will be correct

[d12]

Hey @alexander-soare , I'm running into the exact same thing and found your post. Did you end up sorting this out?

[alexander-soare]

I ended up prioritizing other things, so unfortunately I don't have a solution for you. If I were facing the problem now, I'd probably just go ahead and do a custom implementation and patch it in. Cool that you identified it too though!

[d12]

No worries, I'm just keeping the NMS threshold fairly low for now and sorting out duplicates in post-processing. I'll update this thread if I ever try patching something in to Detectron.