Densepose with keypoint detection

[benihime91]

Hi, I want to train a model that does both keypoint detection and densepose cse. I far I have tried adding the following to the densepose config.

  DENSEPOSE_ON: True
  KEYPOINT_ON: True

  ROI_HEADS:
    NAME: "DensePoseROIHeads"
    IN_FEATURES: ["p2", "p3", "p4", "p5"]
    NUM_CLASSES: 1

  ROI_BOX_HEAD:
    NAME: "FastRCNNConvFCHead"
    NUM_FC: 2
    POOLER_RESOLUTION: 7
    POOLER_SAMPLING_RATIO: 2
    POOLER_TYPE: "ROIAlign"
    SMOOTH_L1_BETA: 0.5

  ROI_MASK_HEAD:
    NAME: "MaskRCNNConvUpsampleHead"
    NUM_CONV: 4
    POOLER_RESOLUTION: 14

  ROI_DENSEPOSE_HEAD:
    NAME: "DensePoseV1ConvXHead"
    POOLER_TYPE: "ROIAlign"
    NUM_COARSE_SEGM_CHANNELS: 2
    CSE:
      EMBEDDERS:
        "smpl_27554":
          TYPE: vertex_feature
          NUM_VERTICES: 27554
          FEATURE_DIM: 256
          FEATURES_TRAINABLE: False
          IS_TRAINABLE: True
          INIT_FILE: "https://dl.fbaipublicfiles.com/densepose/data/cse/lbo/phi_smpl_27554_256.pkl"

While the model trains i have noticed that the loss for keypoint is very high. This is what i am getting after 20k iterations -
loss_cls: 0.1545 loss_box_reg: 0.1527 loss_keypoint: 6.79 loss_densepose_U: 0.4611 loss_densepose_V: 0.3841 loss_densepose_I: 1.776 loss_densepose_S: 1.79 loss_rpn_cls: 0.08757 loss_rpn_loc: 0.08099 time: 2.3350 last_time: 2.6769 data_time: 0.0619 last_data_time: 0.0984 lr: 0.00098672. Is this to be expected ? If not can I get some guidance here on how to achive the task ?

[benihime91]

I am using densepose_coco_2014_train_cse dataset for training

[Haozong-Zeng]

Have you solve the issue?