About the evaluation on novel categories

[dragonlzm]

❓ Questions and Help

Dear author,
Hi! I wonder whether the evaluation of model performance on the novel categories will include the images that only have base categories annotations but without novel categories annotation? Since in the COCO evaluation set, there are many images with only base categories annotations but without novel categories annotations. Thank you so much for your help and looking forward to your reply!

[alirezazareian]

Thanks for your question. I may not fully remember the details, but I believe in the novel set evaluation, we exclude images that don't have any annotation for novel categories. However, there is also a generalized evaluation setting, where we include all images in COCO evaluation set, but we report base and novel mAPs separately.