Memory issues #248
Description
I am using Deformable DETR, I have a H100 server to experiment. However the memory usage is very high. The image resolution is 376p, the total training images is 6500 but the utilisation of the total VRAM during training is 95GB on H100 server. Is it expected? Any memory leak issues?