Bug while running pose only variant with default train_config

[umurotti]

Hey there,

I think there is a tiny bug while running the pose only variant with given default settings.

transformer_unimodal returns only action features.

Verb and noun features are missing.

This causes the following error in train_model.py:

local variable 'verb_output' referenced before assignment
File "./HandFormer/HandFormer/train_model.py", line 434, in train
verb_category_loss = self.loss_v(verb_output, batch_verb_label) if self.arg.add_verb_loss else 0
UnboundLocalError: local variable 'verb_output' referenced before assignment

Should the following lines to unimodal transformer:

    batch_out_verb_feature=torch.flatten(batch_seq_aout_feature[:,0],1,-1)
    batch_out_noun_feature=torch.flatten(batch_seq_aout_feature[:,1],1,-1)
    batch_out_action_feature=torch.flatten(batch_seq_aout_feature[:,2],1,-1)

plus additional classifier heads to hf_pose be added to solve the issue?

     # Classifiers
    self.classifier = nn.Linear(embedding_dim_final, num_classes)
    self.classifier_verb = nn.Linear(embedding_dim_final, num_verbs)
    self.classifier_noun = nn.Linear(embedding_dim_final, num_nouns)

[anastanescu]

Hi! I encountered the same issue and I think that just changing the flags 'add_verb_loss', 'add_noun_loss' and 'add_feat_loss' to False in train_config.yaml will fix this.