Gradient Exploding & LossPredLoss - MultiBoxLoss Problems

[zuliani99]

Thank you for your code it has been very helpful for me, however I'm encountering some issues regarding the object detection task:

1. If I do not insert the gradient clipping the loss will explode causing it to be nan
2. The output of the MultiBoxLoss is a single loss of all the batch [1], whereas the output of the LossNet is [32,1], so once these input are fed to the LossPredLoss it will return always return 0.0 as final loss module

The change that I've made regarding the SSD module is that I've attached the LossNet on this class and called the forward pass on the return of the SSD forward, like so:

return output, self.loss_net(self.features), None

The None is for future work, for returning the embeddings

[zuliani99]

I end up with computing the multiboxloss for each image, by this code, however the loss for the prediction loss module do not decrease

images = images.clone().detach().requires_grad_(True).to(self.device, non_blocking=True)
targets = [ann.to(self.device, non_blocking=True) for ann in targets]
            
outputs, _ = self.model['backbone'](images)
features = self.model['backbone'].get_features()
out_weird = self.model['module'](features)
            
self.optimizer['backbone'].zero_grad()
self.optimizer['module'].zero_grad()
            
loc_data, conf_data, priors = outputs
for idx_image in range(images.shape[0]):
    output_tuple = (
        loc_data[idx_image,:,:].unsqueeze(0),
        conf_data[idx_image,:,:].unsqueeze(0),
        priors
    )
loss_l, loss_c = self.loss_fn(output_tuple, [targets[idx_image]])
losses = torch.cat((losses, torch.tensor([loss_l, loss_c], requires_grad=True).unsqueeze(0))) # [32,2]
                
target_loss = torch.sum(losses, dim = 1) # -> [32,1]
            
module_loss = self.loss_weird(out_weird, target_loss.squeeze())
backbone_loss = torch.mean(target_loss)
loss = backbone_loss + module_loss
               
loss.backward()

I do not understand how the following .reshape(1) on

module_loss = LossPredLoss(pred_loss, target_loss.reshape(1), margin=1.0)

could work if the two tensors have different shapes.