High mAP only with RESUME parameter, otherwise its almost zero

Hi @alcinos , @fmassa,

I am training DETR on my custom dataset. I did below changes in my code hubconf.py: if pretrained: checkpoint = torch.load("detr-r50-e632da11.pth",map_location="cpu") del checkpoint["model"]["class_embed.weight"] del checkpoint["model"]["class_embed.bias"] torch.save(checkpoint,"detr-r50_no-class-head.pth") main.py: model_without_ddp.load_state_dict(checkpoint['model'],strict=False)

I trained my model for 500 epochs but still mAP was only around zero. However, when I resumed training with RESUME parameters using "detr-r50_no-class-head.pth", then suddenly mAP jumped to 0.72. To check it again, I trained my model with 21 epochs and then resumed training with RESUME parameter. It then showed map again as 0.72. Please see below graphs

This means mAP is going up significantly only with RESUME parameter and its almost nothing without RESUME. Any reason why this is happening? Any way to make changes so that mAP will go up without RESUME?