Reproduce the results of HRNet-W32 on CrowdPose

[leeyegy]

I followed the official config and trained HRNet-W32 on CrowdPose, however, I didn't get the results as good as the MMPose reported.
Mine(8*64) VS MMPose report : 66.2 AP VS 67.5 AP.
I was wondering that what are the official settings (e.g., how many GPUs used?)

Here comes the command I use:
tools/dist_train.sh configs/top_down/hrnet/crowdpose/hrnet_w32_crowdpose_256x192.py 8

The resulted log is as follows:

2021-05-24 23:27:48,483 - mmpose - INFO - Environment info:
------------------------------------------------------------
sys.platform: linux
Python: 3.6.2 |Anaconda, Inc.| (default, Sep 30 2017, 18:42:57) [GCC 7.2.0]
CUDA available: True
GPU 0,1,2,3,4,5,6,7: Tesla V100-PCIE-32GB
CUDA_HOME: /data/cuda/cuda-10.1/cuda
NVCC: Cuda compilation tools, release 10.1, V10.1.243
GCC: gcc (Ubuntu 7.4.0-1ubuntu1~18.04.1) 7.4.0
PyTorch: 1.6.0
PyTorch compiling details: PyTorch built with:
  - GCC 7.3
  - C++ Version: 201402
  - Intel(R) Math Kernel Library Version 2020.0.2 Product Build 20200624 for Intel(R) 64 architecture applications
  - Intel(R) MKL-DNN v1.5.0 (Git Hash e2ac1fac44c5078ca927cb9b90e1b3066a0b2ed0)
  - OpenMP 201511 (a.k.a. OpenMP 4.5)
  - NNPACK is enabled
  - CPU capability usage: AVX2
  - CUDA Runtime 10.2
  - NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_61,code=sm_61;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_37,code=compute_37
  - CuDNN 7.6.5
  - Magma 2.5.2
  - Build settings: BLAS=MKL, BUILD_TYPE=Release, CXX_FLAGS= -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DUSE_VULKAN_WRAPPER -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-sign-compare -Wno-unused-parameter -Wno-unused-variable -Wno-unused-function -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, USE_CUDA=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON, USE_STATIC_DISPATCH=OFF, 

TorchVision: 0.7.0
OpenCV: 3.4.1
MMCV: 1.3.0
MMCV Compiler: GCC 7.4
MMCV CUDA Compiler: 10.1
MMPose: 0.14.0+ebaede6
------------------------------------------------------------

2021-05-24 23:27:48,484 - mmpose - INFO - Distributed training: True
2021-05-24 23:27:50,583 - mmpose - INFO - Config:
log_level = 'INFO'
load_from = None
resume_from = None
dist_params = dict(backend='nccl')
workflow = [('train', 1)]
checkpoint_config = dict(interval=10)
evaluation = dict(interval=10, metric='mAP')
optimizer = dict(type='Adam', lr=0.0005)
optimizer_config = dict(grad_clip=None)
lr_config = dict(
    policy='step',
    warmup='linear',
    warmup_iters=500,
    warmup_ratio=0.001,
    step=[170, 200])
total_epochs = 210
log_config = dict(interval=50, hooks=[dict(type='TextLoggerHook')])
channel_cfg = dict(
    num_output_channels=14,
    dataset_joints=14,
    dataset_channel=[[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13]],
    inference_channel=[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13])
model = dict(
    type='TopDown',
    pretrained=
    'https://download.openmmlab.com/mmpose/pretrain_models/hrnet_w32-36af842e.pth',
    backbone=dict(
        type='HRNet',
        in_channels=3,
        extra=dict(
            stage1=dict(
                num_modules=1,
                num_branches=1,
                block='BOTTLENECK',
                num_blocks=(4, ),
                num_channels=(64, )),
            stage2=dict(
                num_modules=1,
                num_branches=2,
                block='BASIC',
                num_blocks=(4, 4),
                num_channels=(32, 64)),
            stage3=dict(
                num_modules=4,
                num_branches=3,
                block='BASIC',
                num_blocks=(4, 4, 4),
                num_channels=(32, 64, 128)),
            stage4=dict(
                num_modules=3,
                num_branches=4,
                block='BASIC',
                num_blocks=(4, 4, 4, 4),
                num_channels=(32, 64, 128, 256)))),
    keypoint_head=dict(
        type='TopDownSimpleHead',
        in_channels=32,
        out_channels=14,
        num_deconv_layers=0,
        extra=dict(final_conv_kernel=1),
        loss_keypoint=dict(type='JointsMSELoss', use_target_weight=True)),
    train_cfg=dict(),
    test_cfg=dict(
        flip_test=True,
        post_process='default',
        shift_heatmap=True,
        modulate_kernel=11))
data_cfg = dict(
    image_size=[192, 256],
    heatmap_size=[48, 64],
    num_output_channels=14,
    num_joints=14,
    dataset_channel=[[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13]],
    inference_channel=[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13],
    crowd_matching=False,
    soft_nms=False,
    nms_thr=1.0,
    oks_thr=0.9,
    vis_thr=0.2,
    use_gt_bbox=False,
    det_bbox_thr=0.0,
    bbox_file=
    '/data/dataset/crowdpose/crowdpose_annotations/det_for_crowd_test_0.1_0.5.json'
)
train_pipeline = [
    dict(type='LoadImageFromFile'),
    dict(type='TopDownRandomFlip', flip_prob=0.5),
    dict(
        type='TopDownHalfBodyTransform',
        num_joints_half_body=6,
        prob_half_body=0.3),
    dict(
        type='TopDownGetRandomScaleRotation', rot_factor=40, scale_factor=0.5),
    dict(type='TopDownAffine'),
    dict(type='ToTensor'),
    dict(
        type='NormalizeTensor',
        mean=[0.485, 0.456, 0.406],
        std=[0.229, 0.224, 0.225]),
    dict(type='TopDownGenerateTarget', sigma=2),
    dict(
        type='Collect',
        keys=['img', 'target', 'target_weight'],
        meta_keys=[
            'image_file', 'joints_3d', 'joints_3d_visible', 'center', 'scale',
            'rotation', 'bbox_score', 'flip_pairs'
        ])
]
val_pipeline = [
    dict(type='LoadImageFromFile'),
    dict(type='TopDownAffine'),
    dict(type='ToTensor'),
    dict(
        type='NormalizeTensor',
        mean=[0.485, 0.456, 0.406],
        std=[0.229, 0.224, 0.225]),
    dict(
        type='Collect',
        keys=['img'],
        meta_keys=[
            'image_file', 'center', 'scale', 'rotation', 'bbox_score',
            'flip_pairs'
        ])
]
test_pipeline = [
    dict(type='LoadImageFromFile'),
    dict(type='TopDownAffine'),
    dict(type='ToTensor'),
    dict(
        type='NormalizeTensor',
        mean=[0.485, 0.456, 0.406],
        std=[0.229, 0.224, 0.225]),
    dict(
        type='Collect',
        keys=['img'],
        meta_keys=[
            'image_file', 'center', 'scale', 'rotation', 'bbox_score',
            'flip_pairs'
        ])
]
data_root = '/data/dataset/crowdpose'
data = dict(
    samples_per_gpu=64,
    workers_per_gpu=2,
    train=dict(
        type='TopDownCrowdPoseDataset',
        ann_file=
        '/data/dataset/crowdpose/crowdpose_annotations/mmpose_crowdpose_trainval.json',
        img_prefix='/data/dataset/crowdpose/images/',
        data_cfg=dict(
            image_size=[192, 256],
            heatmap_size=[48, 64],
            num_output_channels=14,
            num_joints=14,
            dataset_channel=[[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13]],
            inference_channel=[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13],
            crowd_matching=False,
            soft_nms=False,
            nms_thr=1.0,
            oks_thr=0.9,
            vis_thr=0.2,
            use_gt_bbox=False,
            det_bbox_thr=0.0,
            bbox_file=
            '/data/dataset/crowdpose/crowdpose_annotations/det_for_crowd_test_0.1_0.5.json'
        ),
        pipeline=[
            dict(type='LoadImageFromFile'),
            dict(type='TopDownRandomFlip', flip_prob=0.5),
            dict(
                type='TopDownHalfBodyTransform',
                num_joints_half_body=6,
                prob_half_body=0.3),
            dict(
                type='TopDownGetRandomScaleRotation',
                rot_factor=40,
                scale_factor=0.5),
            dict(type='TopDownAffine'),
            dict(type='ToTensor'),
            dict(
                type='NormalizeTensor',
                mean=[0.485, 0.456, 0.406],
                std=[0.229, 0.224, 0.225]),
            dict(type='TopDownGenerateTarget', sigma=2),
            dict(
                type='Collect',
                keys=['img', 'target', 'target_weight'],
                meta_keys=[
                    'image_file', 'joints_3d', 'joints_3d_visible', 'center',
                    'scale', 'rotation', 'bbox_score', 'flip_pairs'
                ])
        ]),
    val=dict(
        type='TopDownCrowdPoseDataset',
        ann_file=
        '/data/dataset/crowdpose/crowdpose_annotations/mmpose_crowdpose_test.json',
        img_prefix='/data/dataset/crowdpose/images/',
        data_cfg=dict(
            image_size=[192, 256],
            heatmap_size=[48, 64],
            num_output_channels=14,
            num_joints=14,
            dataset_channel=[[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13]],
            inference_channel=[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13],
            crowd_matching=False,
            soft_nms=False,
            nms_thr=1.0,
            oks_thr=0.9,
            vis_thr=0.2,
            use_gt_bbox=False,
            det_bbox_thr=0.0,
            bbox_file=
            '/data/dataset/crowdpose/crowdpose_annotations/det_for_crowd_test_0.1_0.5.json'
        ),
        pipeline=[
            dict(type='LoadImageFromFile'),
            dict(type='TopDownAffine'),
            dict(type='ToTensor'),
            dict(
                type='NormalizeTensor',
                mean=[0.485, 0.456, 0.406],
                std=[0.229, 0.224, 0.225]),
            dict(
                type='Collect',
                keys=['img'],
                meta_keys=[
                    'image_file', 'center', 'scale', 'rotation', 'bbox_score',
                    'flip_pairs'
                ])
        ]),
    test=dict(
        type='TopDownCrowdPoseDataset',
        ann_file=
        '/data/dataset/crowdpose/crowdpose_annotations/mmpose_crowdpose_test.json',
        img_prefix='/data/dataset/crowdpose/images/',
        data_cfg=dict(
            image_size=[192, 256],
            heatmap_size=[48, 64],
            num_output_channels=14,
            num_joints=14,
            dataset_channel=[[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13]],
            inference_channel=[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13],
            crowd_matching=False,
            soft_nms=False,
            nms_thr=1.0,
            oks_thr=0.9,
            vis_thr=0.2,
            use_gt_bbox=False,
            det_bbox_thr=0.0,
            bbox_file=
            '/data/dataset/crowdpose/crowdpose_annotations/det_for_crowd_test_0.1_0.5.json'
        ),
        pipeline=[
            dict(type='LoadImageFromFile'),
            dict(type='TopDownAffine'),
            dict(type='ToTensor'),
            dict(
                type='NormalizeTensor',
                mean=[0.485, 0.456, 0.406],
                std=[0.229, 0.224, 0.225]),
            dict(
                type='Collect',
                keys=['img'],
                meta_keys=[
                    'image_file', 'center', 'scale', 'rotation', 'bbox_score',
                    'flip_pairs'
                ])
        ]))
work_dir = './work_dirs/opt_hrnet_w32_crowdpose_256x192'
gpu_ids = range(0, 1)

2021-05-24 23:27:55,564 - mmpose - WARNING - The model and loaded state dict do not match exactly

unexpected key in source state_dict: head.0.0.0.conv1.weight, head.0.0.0.bn1.weight, head.0.0.0.bn1.bias, head.0.0.0.bn1.running_mean, head.0.0.0.bn1.running_var, head.0.0.0.bn1.num_batches_tracked, head.0.0.0.conv2.weight, head.0.0.0.bn2.weight, head.0.0.0.bn2.bias, head.0.0.0.bn2.running_mean, head.0.0.0.bn2.running_var, head.0.0.0.bn2.num_batches_tracked, head.0.0.0.conv3.weight, head.0.0.0.bn3.weight, head.0.0.0.bn3.bias, head.0.0.0.bn3.running_mean, head.0.0.0.bn3.running_var, head.0.0.0.bn3.num_batches_tracked, head.0.0.0.downsample.0.weight, head.0.0.0.downsample.1.weight, head.0.0.0.downsample.1.bias, head.0.0.0.downsample.1.running_mean, head.0.0.0.downsample.1.running_var, head.0.0.0.downsample.1.num_batches_tracked, head.0.1.0.conv1.weight, head.0.1.0.bn1.weight, head.0.1.0.bn1.bias, head.0.1.0.bn1.running_mean, head.0.1.0.bn1.running_var, head.0.1.0.bn1.num_batches_tracked, head.0.1.0.conv2.weight, head.0.1.0.bn2.weight, head.0.1.0.bn2.bias, head.0.1.0.bn2.running_mean, head.0.1.0.bn2.running_var, head.0.1.0.bn2.num_batches_tracked, head.0.1.0.conv3.weight, head.0.1.0.bn3.weight, head.0.1.0.bn3.bias, head.0.1.0.bn3.running_mean, head.0.1.0.bn3.running_var, head.0.1.0.bn3.num_batches_tracked, head.0.1.0.downsample.0.weight, head.0.1.0.downsample.1.weight, head.0.1.0.downsample.1.bias, head.0.1.0.downsample.1.running_mean, head.0.1.0.downsample.1.running_var, head.0.1.0.downsample.1.num_batches_tracked, head.0.2.0.conv1.weight, head.0.2.0.bn1.weight, head.0.2.0.bn1.bias, head.0.2.0.bn1.running_mean, head.0.2.0.bn1.running_var, head.0.2.0.bn1.num_batches_tracked, head.0.2.0.conv2.weight, head.0.2.0.bn2.weight, head.0.2.0.bn2.bias, head.0.2.0.bn2.running_mean, head.0.2.0.bn2.running_var, head.0.2.0.bn2.num_batches_tracked, head.0.2.0.conv3.weight, head.0.2.0.bn3.weight, head.0.2.0.bn3.bias, head.0.2.0.bn3.running_mean, head.0.2.0.bn3.running_var, head.0.2.0.bn3.num_batches_tracked, head.0.2.0.downsample.0.weight, head.0.2.0.downsample.1.weight, head.0.2.0.downsample.1.bias, head.0.2.0.downsample.1.running_mean, head.0.2.0.downsample.1.running_var, head.0.2.0.downsample.1.num_batches_tracked, head.1.0.0.conv1.weight, head.1.0.0.bn1.weight, head.1.0.0.bn1.bias, head.1.0.0.bn1.running_mean, head.1.0.0.bn1.running_var, head.1.0.0.bn1.num_batches_tracked, head.1.0.0.conv2.weight, head.1.0.0.bn2.weight, head.1.0.0.bn2.bias, head.1.0.0.bn2.running_mean, head.1.0.0.bn2.running_var, head.1.0.0.bn2.num_batches_tracked, head.1.0.0.conv3.weight, head.1.0.0.bn3.weight, head.1.0.0.bn3.bias, head.1.0.0.bn3.running_mean, head.1.0.0.bn3.running_var, head.1.0.0.bn3.num_batches_tracked, head.1.0.0.downsample.0.weight, head.1.0.0.downsample.1.weight, head.1.0.0.downsample.1.bias, head.1.0.0.downsample.1.running_mean, head.1.0.0.downsample.1.running_var, head.1.0.0.downsample.1.num_batches_tracked, head.1.1.0.conv1.weight, head.1.1.0.bn1.weight, head.1.1.0.bn1.bias, head.1.1.0.bn1.running_mean, head.1.1.0.bn1.running_var, head.1.1.0.bn1.num_batches_tracked, head.1.1.0.conv2.weight, head.1.1.0.bn2.weight, head.1.1.0.bn2.bias, head.1.1.0.bn2.running_mean, head.1.1.0.bn2.running_var, head.1.1.0.bn2.num_batches_tracked, head.1.1.0.conv3.weight, head.1.1.0.bn3.weight, head.1.1.0.bn3.bias, head.1.1.0.bn3.running_mean, head.1.1.0.bn3.running_var, head.1.1.0.bn3.num_batches_tracked, head.1.1.0.downsample.0.weight, head.1.1.0.downsample.1.weight, head.1.1.0.downsample.1.bias, head.1.1.0.downsample.1.running_mean, head.1.1.0.downsample.1.running_var, head.1.1.0.downsample.1.num_batches_tracked, head.2.0.0.conv1.weight, head.2.0.0.bn1.weight, head.2.0.0.bn1.bias, head.2.0.0.bn1.running_mean, head.2.0.0.bn1.running_var, head.2.0.0.bn1.num_batches_tracked, head.2.0.0.conv2.weight, head.2.0.0.bn2.weight, head.2.0.0.bn2.bias, head.2.0.0.bn2.running_mean, head.2.0.0.bn2.running_var, head.2.0.0.bn2.num_batches_tracked, head.2.0.0.conv3.weight, head.2.0.0.bn3.weight, head.2.0.0.bn3.bias, head.2.0.0.bn3.running_mean, head.2.0.0.bn3.running_var, head.2.0.0.bn3.num_batches_tracked, head.2.0.0.downsample.0.weight, head.2.0.0.downsample.1.weight, head.2.0.0.downsample.1.bias, head.2.0.0.downsample.1.running_mean, head.2.0.0.downsample.1.running_var, head.2.0.0.downsample.1.num_batches_tracked, head.3.0.0.conv1.weight, head.3.0.0.bn1.weight, head.3.0.0.bn1.bias, head.3.0.0.bn1.running_mean, head.3.0.0.bn1.running_var, head.3.0.0.bn1.num_batches_tracked, head.3.0.0.conv2.weight, head.3.0.0.bn2.weight, head.3.0.0.bn2.bias, head.3.0.0.bn2.running_mean, head.3.0.0.bn2.running_var, head.3.0.0.bn2.num_batches_tracked, head.3.0.0.conv3.weight, head.3.0.0.bn3.weight, head.3.0.0.bn3.bias, head.3.0.0.bn3.running_mean, head.3.0.0.bn3.running_var, head.3.0.0.bn3.num_batches_tracked, head.3.0.0.downsample.0.weight, head.3.0.0.downsample.1.weight, head.3.0.0.downsample.1.bias, head.3.0.0.downsample.1.running_mean, head.3.0.0.downsample.1.running_var, head.3.0.0.downsample.1.num_batches_tracked, fc.weight, fc.bias, stage4.2.fuse_layers.1.0.0.0.weight, stage4.2.fuse_layers.1.0.0.1.weight, stage4.2.fuse_layers.1.0.0.1.bias, stage4.2.fuse_layers.1.0.0.1.running_mean, stage4.2.fuse_layers.1.0.0.1.running_var, stage4.2.fuse_layers.1.0.0.1.num_batches_tracked, stage4.2.fuse_layers.1.2.0.weight, stage4.2.fuse_layers.1.2.1.weight, stage4.2.fuse_layers.1.2.1.bias, stage4.2.fuse_layers.1.2.1.running_mean, stage4.2.fuse_layers.1.2.1.running_var, stage4.2.fuse_layers.1.2.1.num_batches_tracked, stage4.2.fuse_layers.1.3.0.weight, stage4.2.fuse_layers.1.3.1.weight, stage4.2.fuse_layers.1.3.1.bias, stage4.2.fuse_layers.1.3.1.running_mean, stage4.2.fuse_layers.1.3.1.running_var, stage4.2.fuse_layers.1.3.1.num_batches_tracked, stage4.2.fuse_layers.2.0.0.0.weight, stage4.2.fuse_layers.2.0.0.1.weight, stage4.2.fuse_layers.2.0.0.1.bias, stage4.2.fuse_layers.2.0.0.1.running_mean, stage4.2.fuse_layers.2.0.0.1.running_var, stage4.2.fuse_layers.2.0.0.1.num_batches_tracked, stage4.2.fuse_layers.2.0.1.0.weight, stage4.2.fuse_layers.2.0.1.1.weight, stage4.2.fuse_layers.2.0.1.1.bias, stage4.2.fuse_layers.2.0.1.1.running_mean, stage4.2.fuse_layers.2.0.1.1.running_var, stage4.2.fuse_layers.2.0.1.1.num_batches_tracked, stage4.2.fuse_layers.2.1.0.0.weight, stage4.2.fuse_layers.2.1.0.1.weight, stage4.2.fuse_layers.2.1.0.1.bias, stage4.2.fuse_layers.2.1.0.1.running_mean, stage4.2.fuse_layers.2.1.0.1.running_var, stage4.2.fuse_layers.2.1.0.1.num_batches_tracked, stage4.2.fuse_layers.2.3.0.weight, stage4.2.fuse_layers.2.3.1.weight, stage4.2.fuse_layers.2.3.1.bias, stage4.2.fuse_layers.2.3.1.running_mean, stage4.2.fuse_layers.2.3.1.running_var, stage4.2.fuse_layers.2.3.1.num_batches_tracked, stage4.2.fuse_layers.3.0.0.0.weight, stage4.2.fuse_layers.3.0.0.1.weight, stage4.2.fuse_layers.3.0.0.1.bias, stage4.2.fuse_layers.3.0.0.1.running_mean, stage4.2.fuse_layers.3.0.0.1.running_var, stage4.2.fuse_layers.3.0.0.1.num_batches_tracked, stage4.2.fuse_layers.3.0.1.0.weight, stage4.2.fuse_layers.3.0.1.1.weight, stage4.2.fuse_layers.3.0.1.1.bias, stage4.2.fuse_layers.3.0.1.1.running_mean, stage4.2.fuse_layers.3.0.1.1.running_var, stage4.2.fuse_layers.3.0.1.1.num_batches_tracked, stage4.2.fuse_layers.3.0.2.0.weight, stage4.2.fuse_layers.3.0.2.1.weight, stage4.2.fuse_layers.3.0.2.1.bias, stage4.2.fuse_layers.3.0.2.1.running_mean, stage4.2.fuse_layers.3.0.2.1.running_var, stage4.2.fuse_layers.3.0.2.1.num_batches_tracked, stage4.2.fuse_layers.3.1.0.0.weight, stage4.2.fuse_layers.3.1.0.1.weight, stage4.2.fuse_layers.3.1.0.1.bias, stage4.2.fuse_layers.3.1.0.1.running_mean, stage4.2.fuse_layers.3.1.0.1.running_var, stage4.2.fuse_layers.3.1.0.1.num_batches_tracked, stage4.2.fuse_layers.3.1.1.0.weight, stage4.2.fuse_layers.3.1.1.1.weight, stage4.2.fuse_layers.3.1.1.1.bias, stage4.2.fuse_layers.3.1.1.1.running_mean, stage4.2.fuse_layers.3.1.1.1.running_var, stage4.2.fuse_layers.3.1.1.1.num_batches_tracked, stage4.2.fuse_layers.3.2.0.0.weight, stage4.2.fuse_layers.3.2.0.1.weight, stage4.2.fuse_layers.3.2.0.1.bias, stage4.2.fuse_layers.3.2.0.1.running_mean, stage4.2.fuse_layers.3.2.0.1.running_var, stage4.2.fuse_layers.3.2.0.1.num_batches_tracked

2021-05-24 23:28:05,587 - mmpose - INFO - Start running, host: liyanjie@test-28F12706-worker-0, work_dir: /data/pose/mmpose/work_dirs/opt_hrnet_w32_crowdpose_256x192
2021-05-24 23:28:05,588 - mmpose - INFO - workflow: [('train', 1)], max: 210 epochs
2021-05-24 23:29:48,686 - mmpose - INFO - Epoch [1][50/83]	lr: 4.945e-05, eta: 9:56:58, time: 2.061, data_time: 0.673, memory: 9832, mse_loss: 0.0026, acc_pose: 0.0419, loss: 0.0026
2021-05-24 23:31:45,472 - mmpose - INFO - Epoch [2][50/83]	lr: 1.324e-04, eta: 6:31:40, time: 1.553, data_time: 0.384, memory: 9832, mse_loss: 0.0020, acc_pose: 0.3032, loss: 0.0020
2021-05-24 23:33:39,090 - mmpose - INFO - Epoch [3][50/83]	lr: 2.153e-04, eta: 5:40:09, time: 1.508, data_time: 0.333, memory: 9832, mse_loss: 0.0017, acc_pose: 0.4383, loss: 0.0017
2021-05-24 23:35:33,003 - mmpose - INFO - Epoch [4][50/83]	lr: 2.982e-04, eta: 5:17:05, time: 1.519, data_time: 0.414, memory: 9832, mse_loss: 0.0016, acc_pose: 0.5038, loss: 0.0016
2021-05-24 23:37:29,634 - mmpose - INFO - Epoch [5][50/83]	lr: 3.811e-04, eta: 5:04:52, time: 1.556, data_time: 0.308, memory: 9832, mse_loss: 0.0015, acc_pose: 0.5299, loss: 0.0015
2021-05-24 23:39:24,318 - mmpose - INFO - Epoch [6][50/83]	lr: 4.640e-04, eta: 4:54:54, time: 1.502, data_time: 0.324, memory: 9832, mse_loss: 0.0014, acc_pose: 0.5395, loss: 0.0014
2021-05-24 23:41:16,486 - mmpose - INFO - Epoch [7][50/83]	lr: 5.000e-04, eta: 4:47:15, time: 1.490, data_time: 0.684, memory: 9832, mse_loss: 0.0014, acc_pose: 0.5611, loss: 0.0014
2021-05-24 23:43:12,854 - mmpose - INFO - Epoch [8][50/83]	lr: 5.000e-04, eta: 4:42:47, time: 1.557, data_time: 0.390, memory: 9832, mse_loss: 0.0014, acc_pose: 0.5660, loss: 0.0014
2021-05-24 23:45:08,733 - mmpose - INFO - Epoch [9][50/83]	lr: 5.000e-04, eta: 4:38:18, time: 1.518, data_time: 0.542, memory: 9832, mse_loss: 0.0013, acc_pose: 0.5876, loss: 0.0013
2021-05-24 23:47:02,339 - mmpose - INFO - Epoch [10][50/83]	lr: 5.000e-04, eta: 4:33:47, time: 1.478, data_time: 0.293, memory: 9832, mse_loss: 0.0013, acc_pose: 0.6017, loss: 0.0013
2021-05-24 23:47:40,861 - mmpose - INFO - Saving checkpoint at 10 epochs
2021-05-24 23:59:12,582 - mmpose - INFO - Epoch(val) [10][83]	AP: 0.5078, AP .5: 0.7365, AP .75: 0.5444, AR: 0.6262, AR .5: 0.8589, AR .75: 0.6675, AP(E): 0.6103, AP(M): 0.5162, AP(H): 0.3883
2021-05-25 00:00:25,005 - mmpose - INFO - Epoch [11][50/83]	lr: 5.000e-04, eta: 4:29:24, time: 1.447, data_time: 0.320, memory: 9832, mse_loss: 0.0013, acc_pose: 0.6063, loss: 0.0013
2021-05-25 00:02:25,065 - mmpose - INFO - Epoch [12][50/83]	lr: 5.000e-04, eta: 4:28:00, time: 1.618, data_time: 0.337, memory: 9832, mse_loss: 0.0013, acc_pose: 0.6231, loss: 0.0013
2021-05-25 00:04:16,073 - mmpose - INFO - Epoch [13][50/83]	lr: 5.000e-04, eta: 4:24:49, time: 1.480, data_time: 0.524, memory: 9832, mse_loss: 0.0012, acc_pose: 0.6212, loss: 0.0012
2021-05-25 00:06:09,266 - mmpose - INFO - Epoch [14][50/83]	lr: 5.000e-04, eta: 4:22:18, time: 1.513, data_time: 0.298, memory: 9832, mse_loss: 0.0012, acc_pose: 0.6227, loss: 0.0012
2021-05-25 00:08:03,145 - mmpose - INFO - Epoch [15][50/83]	lr: 5.000e-04, eta: 4:20:06, time: 1.524, data_time: 0.358, memory: 9832, mse_loss: 0.0012, acc_pose: 0.6300, loss: 0.0012
2021-05-25 00:09:59,707 - mmpose - INFO - Epoch [16][50/83]	lr: 5.000e-04, eta: 4:18:08, time: 1.537, data_time: 0.312, memory: 9832, mse_loss: 0.0012, acc_pose: 0.6351, loss: 0.0012
2021-05-25 00:11:54,325 - mmpose - INFO - Epoch [17][50/83]	lr: 5.000e-04, eta: 4:16:13, time: 1.532, data_time: 0.302, memory: 9832, mse_loss: 0.0012, acc_pose: 0.6415, loss: 0.0012
2021-05-25 00:13:49,463 - mmpose - INFO - Epoch [18][50/83]	lr: 5.000e-04, eta: 4:14:10, time: 1.509, data_time: 0.316, memory: 9832, mse_loss: 0.0012, acc_pose: 0.6448, loss: 0.0012
2021-05-25 00:15:46,243 - mmpose - INFO - Epoch [19][50/83]	lr: 5.000e-04, eta: 4:12:38, time: 1.561, data_time: 0.306, memory: 9832, mse_loss: 0.0012, acc_pose: 0.6429, loss: 0.0012
2021-05-25 00:17:45,063 - mmpose - INFO - Epoch [20][50/83]	lr: 5.000e-04, eta: 4:11:32, time: 1.611, data_time: 0.966, memory: 9832, mse_loss: 0.0012, acc_pose: 0.6551, loss: 0.0012
2021-05-25 00:18:23,556 - mmpose - INFO - Saving checkpoint at 20 epochs
2021-05-25 00:31:25,218 - mmpose - INFO - Epoch(val) [20][83]	AP: 0.5703, AP .5: 0.7729, AP .75: 0.6186, AR: 0.6746, AR .5: 0.8782, AR .75: 0.7219, AP(E): 0.6610, AP(M): 0.5795, AP(H): 0.4526
2021-05-25 00:32:50,946 - mmpose - INFO - Epoch [21][50/83]	lr: 5.000e-04, eta: 4:11:11, time: 1.713, data_time: 0.305, memory: 9832, mse_loss: 0.0011, acc_pose: 0.6654, loss: 0.0011
2021-05-25 00:35:11,642 - mmpose - INFO - Epoch [22][50/83]	lr: 5.000e-04, eta: 4:12:01, time: 1.888, data_time: 0.364, memory: 9832, mse_loss: 0.0011, acc_pose: 0.6603, loss: 0.0011
2021-05-25 00:37:34,793 - mmpose - INFO - Epoch [23][50/83]	lr: 5.000e-04, eta: 4:12:36, time: 1.883, data_time: 0.428, memory: 9832, mse_loss: 0.0011, acc_pose: 0.6589, loss: 0.0011
2021-05-25 00:39:52,271 - mmpose - INFO - Epoch [24][50/83]	lr: 5.000e-04, eta: 4:12:31, time: 1.810, data_time: 0.329, memory: 9832, mse_loss: 0.0011, acc_pose: 0.6727, loss: 0.0011
2021-05-25 00:42:03,364 - mmpose - INFO - Epoch [25][50/83]	lr: 5.000e-04, eta: 4:11:47, time: 1.724, data_time: 0.330, memory: 9832, mse_loss: 0.0011, acc_pose: 0.6648, loss: 0.0011
2021-05-25 00:44:18,980 - mmpose - INFO - Epoch [26][50/83]	lr: 5.000e-04, eta: 4:11:24, time: 1.792, data_time: 0.324, memory: 9832, mse_loss: 0.0011, acc_pose: 0.6666, loss: 0.0011
2021-05-25 00:46:31,602 - mmpose - INFO - Epoch [27][50/83]	lr: 5.000e-04, eta: 4:10:37, time: 1.736, data_time: 0.380, memory: 9832, mse_loss: 0.0011, acc_pose: 0.6734, loss: 0.0011
2021-05-25 00:48:42,384 - mmpose - INFO - Epoch [28][50/83]	lr: 5.000e-04, eta: 4:09:41, time: 1.720, data_time: 0.328, memory: 9832, mse_loss: 0.0011, acc_pose: 0.6833, loss: 0.0011
2021-05-25 00:50:56,304 - mmpose - INFO - Epoch [29][50/83]	lr: 5.000e-04, eta: 4:09:00, time: 1.772, data_time: 0.308, memory: 9832, mse_loss: 0.0011, acc_pose: 0.6700, loss: 0.0011
2021-05-25 00:53:12,125 - mmpose - INFO - Epoch [30][50/83]	lr: 5.000e-04, eta: 4:08:19, time: 1.782, data_time: 0.509, memory: 9832, mse_loss: 0.0011, acc_pose: 0.6712, loss: 0.0011
2021-05-25 00:53:56,501 - mmpose - INFO - Saving checkpoint at 30 epochs
2021-05-25 01:06:32,119 - mmpose - INFO - Epoch(val) [30][83]	AP: 0.5932, AP .5: 0.7833, AP .75: 0.6446, AR: 0.6951, AR .5: 0.8847, AR .75: 0.7449, AP(E): 0.6875, AP(M): 0.6035, AP(H): 0.4710
2021-05-25 01:07:58,037 - mmpose - INFO - Epoch [31][50/83]	lr: 5.000e-04, eta: 4:07:16, time: 1.718, data_time: 0.351, memory: 9832, mse_loss: 0.0011, acc_pose: 0.6787, loss: 0.0011
2021-05-25 01:10:11,429 - mmpose - INFO - Epoch [32][50/83]	lr: 5.000e-04, eta: 4:06:07, time: 1.705, data_time: 0.314, memory: 9832, mse_loss: 0.0011, acc_pose: 0.6780, loss: 0.0011
2021-05-25 01:12:22,629 - mmpose - INFO - Epoch [33][50/83]	lr: 5.000e-04, eta: 4:04:59, time: 1.708, data_time: 0.316, memory: 9832, mse_loss: 0.0011, acc_pose: 0.6801, loss: 0.0011
2021-05-25 01:14:36,527 - mmpose - INFO - Epoch [34][50/83]	lr: 5.000e-04, eta: 4:04:08, time: 1.780, data_time: 0.446, memory: 9832, mse_loss: 0.0011, acc_pose: 0.6898, loss: 0.0011
2021-05-25 01:16:52,194 - mmpose - INFO - Epoch [35][50/83]	lr: 5.000e-04, eta: 4:03:06, time: 1.743, data_time: 0.317, memory: 9832, mse_loss: 0.0011, acc_pose: 0.6970, loss: 0.0011
2021-05-25 01:19:12,169 - mmpose - INFO - Epoch [36][50/83]	lr: 5.000e-04, eta: 4:02:38, time: 1.889, data_time: 0.588, memory: 9832, mse_loss: 0.0011, acc_pose: 0.6855, loss: 0.0011
2021-05-25 01:21:23,593 - mmpose - INFO - Epoch [37][50/83]	lr: 5.000e-04, eta: 4:01:24, time: 1.709, data_time: 0.352, memory: 9832, mse_loss: 0.0011, acc_pose: 0.6871, loss: 0.0011
2021-05-25 01:23:34,377 - mmpose - INFO - Epoch [38][50/83]	lr: 5.000e-04, eta: 4:00:09, time: 1.709, data_time: 0.347, memory: 9832, mse_loss: 0.0011, acc_pose: 0.6897, loss: 0.0011
2021-05-25 01:25:44,949 - mmpose - INFO - Epoch [39][50/83]	lr: 5.000e-04, eta: 3:58:50, time: 1.695, data_time: 0.349, memory: 9832, mse_loss: 0.0010, acc_pose: 0.7016, loss: 0.0010
2021-05-25 01:27:55,139 - mmpose - INFO - Epoch [40][50/83]	lr: 5.000e-04, eta: 3:57:33, time: 1.700, data_time: 0.295, memory: 9832, mse_loss: 0.0010, acc_pose: 0.7071, loss: 0.0010
2021-05-25 01:28:40,107 - mmpose - INFO - Saving checkpoint at 40 epochs
2021-05-25 01:40:46,144 - mmpose - INFO - Epoch(val) [40][83]	AP: 0.5926, AP .5: 0.7725, AP .75: 0.6441, AR: 0.7065, AR .5: 0.8870, AR .75: 0.7579, AP(E): 0.6861, AP(M): 0.6040, AP(H): 0.4655
2021-05-25 01:42:07,738 - mmpose - INFO - Epoch [41][50/83]	lr: 5.000e-04, eta: 3:56:00, time: 1.630, data_time: 0.284, memory: 9832, mse_loss: 0.0010, acc_pose: 0.7030, loss: 0.0010
2021-05-25 01:44:24,317 - mmpose - INFO - Epoch [42][50/83]	lr: 5.000e-04, eta: 3:55:06, time: 1.815, data_time: 0.397, memory: 9832, mse_loss: 0.0010, acc_pose: 0.6891, loss: 0.0010
2021-05-25 01:46:36,914 - mmpose - INFO - Epoch [43][50/83]	lr: 5.000e-04, eta: 3:53:54, time: 1.740, data_time: 0.307, memory: 9832, mse_loss: 0.0010, acc_pose: 0.6969, loss: 0.0010
2021-05-25 01:48:49,176 - mmpose - INFO - Epoch [44][50/83]	lr: 5.000e-04, eta: 3:52:41, time: 1.733, data_time: 0.327, memory: 9832, mse_loss: 0.0010, acc_pose: 0.6951, loss: 0.0010
2021-05-25 01:51:01,867 - mmpose - INFO - Epoch [45][50/83]	lr: 5.000e-04, eta: 3:51:29, time: 1.742, data_time: 0.309, memory: 9832, mse_loss: 0.0010, acc_pose: 0.6961, loss: 0.0010
2021-05-25 01:53:16,315 - mmpose - INFO - Epoch [46][50/83]	lr: 5.000e-04, eta: 3:50:25, time: 1.788, data_time: 0.352, memory: 9832, mse_loss: 0.0010, acc_pose: 0.7035, loss: 0.0010
2021-05-25 01:55:33,076 - mmpose - INFO - Epoch [47][50/83]	lr: 5.000e-04, eta: 3:49:18, time: 1.783, data_time: 0.301, memory: 9832, mse_loss: 0.0010, acc_pose: 0.7075, loss: 0.0010
2021-05-25 01:57:53,985 - mmpose - INFO - Epoch [48][50/83]	lr: 5.000e-04, eta: 3:48:23, time: 1.854, data_time: 0.540, memory: 9832, mse_loss: 0.0010, acc_pose: 0.7127, loss: 0.0010
2021-05-25 02:00:06,231 - mmpose - INFO - Epoch [49][50/83]	lr: 5.000e-04, eta: 3:47:07, time: 1.734, data_time: 0.305, memory: 9832, mse_loss: 0.0010, acc_pose: 0.6969, loss: 0.0010
2021-05-25 02:02:21,960 - mmpose - INFO - Epoch [50][50/83]	lr: 5.000e-04, eta: 3:45:52, time: 1.748, data_time: 0.360, memory: 9832, mse_loss: 0.0010, acc_pose: 0.7036, loss: 0.0010
2021-05-25 02:03:07,645 - mmpose - INFO - Saving checkpoint at 50 epochs
2021-05-25 02:15:39,328 - mmpose - INFO - Epoch(val) [50][83]	AP: 0.6125, AP .5: 0.7851, AP .75: 0.6654, AR: 0.7157, AR .5: 0.8891, AR .75: 0.7655, AP(E): 0.7007, AP(M): 0.6223, AP(H): 0.4949
2021-05-25 02:26:39,495 - mmpose - INFO - Epoch [51][50/83]	lr: 5.000e-04, eta: 4:14:32, time: 13.145, data_time: 6.327, memory: 9832, mse_loss: 0.0010, acc_pose: 0.7162, loss: 0.0010
2021-05-25 02:40:07,606 - mmpose - INFO - Epoch [52][50/83]	lr: 5.000e-04, eta: 4:35:22, time: 10.691, data_time: 4.343, memory: 9832, mse_loss: 0.0010, acc_pose: 0.7078, loss: 0.0010
2021-05-25 02:44:30,233 - mmpose - INFO - Epoch [53][50/83]	lr: 5.000e-04, eta: 4:35:21, time: 2.774, data_time: 1.168, memory: 9832, mse_loss: 0.0010, acc_pose: 0.7135, loss: 0.0010
2021-05-25 02:48:24,442 - mmpose - INFO - Epoch [54][50/83]	lr: 5.000e-04, eta: 4:36:57, time: 3.476, data_time: 1.148, memory: 9832, mse_loss: 0.0010, acc_pose: 0.7053, loss: 0.0010
2021-05-25 02:51:06,497 - mmpose - INFO - Epoch [55][50/83]	lr: 5.000e-04, eta: 4:35:07, time: 2.099, data_time: 0.418, memory: 9832, mse_loss: 0.0010, acc_pose: 0.7169, loss: 0.0010
2021-05-25 02:53:51,004 - mmpose - INFO - Epoch [56][50/83]	lr: 5.000e-04, eta: 4:33:22, time: 2.138, data_time: 0.374, memory: 9832, mse_loss: 0.0010, acc_pose: 0.7204, loss: 0.0010
2021-05-25 02:56:34,015 - mmpose - INFO - Epoch [57][50/83]	lr: 5.000e-04, eta: 4:31:33, time: 2.102, data_time: 0.359, memory: 9832, mse_loss: 0.0010, acc_pose: 0.7069, loss: 0.0010
2021-05-25 02:59:15,330 - mmpose - INFO - Epoch [58][50/83]	lr: 5.000e-04, eta: 4:29:46, time: 2.120, data_time: 0.366, memory: 9832, mse_loss: 0.0010, acc_pose: 0.7073, loss: 0.0010
2021-05-25 03:01:56,577 - mmpose - INFO - Epoch [59][50/83]	lr: 5.000e-04, eta: 4:27:57, time: 2.098, data_time: 0.356, memory: 9832, mse_loss: 0.0010, acc_pose: 0.7144, loss: 0.0010
2021-05-25 03:04:56,076 - mmpose - INFO - Epoch [60][50/83]	lr: 5.000e-04, eta: 4:26:51, time: 2.447, data_time: 0.741, memory: 9832, mse_loss: 0.0010, acc_pose: 0.7183, loss: 0.0010
2021-05-25 03:06:12,255 - mmpose - INFO - Saving checkpoint at 60 epochs
2021-05-25 03:21:00,135 - mmpose - INFO - Now best checkpoint is epoch_60.pth
2021-05-25 03:21:00,560 - mmpose - INFO - Epoch(val) [60][83]	AP: 0.6227, AP .5: 0.7970, AP .75: 0.6771, AR: 0.7218, AR .5: 0.8919, AR .75: 0.7727, AP(E): 0.7078, AP(M): 0.6326, AP(H): 0.5044
2021-05-25 03:22:39,055 - mmpose - INFO - Epoch [61][50/83]	lr: 5.000e-04, eta: 4:24:45, time: 1.969, data_time: 0.371, memory: 9832, mse_loss: 0.0010, acc_pose: 0.7236, loss: 0.0010
2021-05-25 03:25:31,370 - mmpose - INFO - Epoch [62][50/83]	lr: 5.000e-04, eta: 4:23:15, time: 2.260, data_time: 0.463, memory: 9832, mse_loss: 0.0010, acc_pose: 0.7196, loss: 0.0010
2021-05-25 03:28:16,260 - mmpose - INFO - Epoch [63][50/83]	lr: 5.000e-04, eta: 4:21:34, time: 2.175, data_time: 0.373, memory: 9832, mse_loss: 0.0010, acc_pose: 0.7175, loss: 0.0010
2021-05-25 03:30:58,133 - mmpose - INFO - Epoch [64][50/83]	lr: 5.000e-04, eta: 4:19:44, time: 2.104, data_time: 0.340, memory: 9832, mse_loss: 0.0010, acc_pose: 0.7183, loss: 0.0010
2021-05-25 03:33:42,291 - mmpose - INFO - Epoch [65][50/83]	lr: 5.000e-04, eta: 4:17:59, time: 2.137, data_time: 0.426, memory: 9832, mse_loss: 0.0010, acc_pose: 0.7236, loss: 0.0010
2021-05-25 03:36:23,208 - mmpose - INFO - Epoch [66][50/83]	lr: 5.000e-04, eta: 4:16:08, time: 2.090, data_time: 0.399, memory: 9832, mse_loss: 0.0010, acc_pose: 0.7225, loss: 0.0010
2021-05-25 03:39:01,766 - mmpose - INFO - Epoch [67][50/83]	lr: 5.000e-04, eta: 4:14:14, time: 2.054, data_time: 0.451, memory: 9832, mse_loss: 0.0010, acc_pose: 0.7266, loss: 0.0010
2021-05-25 03:41:44,747 - mmpose - INFO - Epoch [68][50/83]	lr: 5.000e-04, eta: 4:12:25, time: 2.105, data_time: 0.373, memory: 9832, mse_loss: 0.0010, acc_pose: 0.7277, loss: 0.0010
2021-05-25 03:44:24,125 - mmpose - INFO - Epoch [69][50/83]	lr: 5.000e-04, eta: 4:10:32, time: 2.062, data_time: 0.381, memory: 9832, mse_loss: 0.0010, acc_pose: 0.7151, loss: 0.0010
2021-05-25 03:47:10,835 - mmpose - INFO - Epoch [70][50/83]	lr: 5.000e-04, eta: 4:08:41, time: 2.083, data_time: 0.358, memory: 9832, mse_loss: 0.0010, acc_pose: 0.7304, loss: 0.0010
2021-05-25 03:48:07,493 - mmpose - INFO - Saving checkpoint at 70 epochs
2021-05-25 04:02:23,412 - mmpose - INFO - Epoch(val) [70][83]	AP: 0.6201, AP .5: 0.7921, AP .75: 0.6737, AR: 0.7209, AR .5: 0.8907, AR .75: 0.7707, AP(E): 0.7083, AP(M): 0.6305, AP(H): 0.4991
2021-05-25 04:04:15,404 - mmpose - INFO - Epoch [71][50/83]	lr: 5.000e-04, eta: 4:07:06, time: 2.239, data_time: 0.674, memory: 9832, mse_loss: 0.0010, acc_pose: 0.7242, loss: 0.0010
2021-05-25 04:06:58,408 - mmpose - INFO - Epoch [72][50/83]	lr: 5.000e-04, eta: 4:05:17, time: 2.096, data_time: 0.362, memory: 9832, mse_loss: 0.0010, acc_pose: 0.7203, loss: 0.0010
2021-05-25 04:09:42,287 - mmpose - INFO - Epoch [73][50/83]	lr: 5.000e-04, eta: 4:03:33, time: 2.151, data_time: 0.420, memory: 9832, mse_loss: 0.0010, acc_pose: 0.7296, loss: 0.0010
2021-05-25 04:12:24,134 - mmpose - INFO - Epoch [74][50/83]	lr: 5.000e-04, eta: 4:01:44, time: 2.102, data_time: 0.357, memory: 9832, mse_loss: 0.0010, acc_pose: 0.7313, loss: 0.0010
2021-05-25 04:15:05,553 - mmpose - INFO - Epoch [75][50/83]	lr: 5.000e-04, eta: 3:59:55, time: 2.100, data_time: 0.395, memory: 9832, mse_loss: 0.0010, acc_pose: 0.7300, loss: 0.0010
2021-05-25 04:17:46,533 - mmpose - INFO - Epoch [76][50/83]	lr: 5.000e-04, eta: 3:58:06, time: 2.090, data_time: 0.368, memory: 9832, mse_loss: 0.0010, acc_pose: 0.7305, loss: 0.0010
2021-05-25 04:20:20,048 - mmpose - INFO - Epoch [77][50/83]	lr: 5.000e-04, eta: 3:56:09, time: 2.006, data_time: 0.382, memory: 9832, mse_loss: 0.0009, acc_pose: 0.7365, loss: 0.0009
2021-05-25 04:23:11,449 - mmpose - INFO - Epoch [78][50/83]	lr: 5.000e-04, eta: 3:54:52, time: 2.472, data_time: 0.316, memory: 9832, mse_loss: 0.0009, acc_pose: 0.7314, loss: 0.0009
2021-05-25 04:26:06,762 - mmpose - INFO - Epoch [79][50/83]	lr: 5.000e-04, eta: 3:53:26, time: 2.372, data_time: 0.864, memory: 9832, mse_loss: 0.0009, acc_pose: 0.7383, loss: 0.0009
2021-05-25 04:28:48,814 - mmpose - INFO - Epoch [80][50/83]	lr: 5.000e-04, eta: 3:51:38, time: 2.119, data_time: 0.354, memory: 9832, mse_loss: 0.0009, acc_pose: 0.7229, loss: 0.0009
2021-05-25 04:29:45,350 - mmpose - INFO - Saving checkpoint at 80 epochs
2021-05-25 04:44:23,947 - mmpose - INFO - Now best checkpoint is epoch_80.pth
2021-05-25 04:44:24,404 - mmpose - INFO - Epoch(val) [80][83]	AP: 0.6309, AP .5: 0.8017, AP .75: 0.6837, AR: 0.7274, AR .5: 0.8942, AR .75: 0.7791, AP(E): 0.7171, AP(M): 0.6407, AP(H): 0.5189
2021-05-25 04:46:08,087 - mmpose - INFO - Epoch [81][50/83]	lr: 5.000e-04, eta: 3:49:47, time: 2.073, data_time: 0.409, memory: 9832, mse_loss: 0.0009, acc_pose: 0.7398, loss: 0.0009
2021-05-25 04:48:46,205 - mmpose - INFO - Epoch [82][50/83]	lr: 5.000e-04, eta: 3:47:52, time: 2.022, data_time: 0.365, memory: 9832, mse_loss: 0.0009, acc_pose: 0.7256, loss: 0.0009
2021-05-25 04:51:16,807 - mmpose - INFO - Epoch [83][50/83]	lr: 5.000e-04, eta: 3:45:41, time: 1.815, data_time: 0.332, memory: 9832, mse_loss: 0.0009, acc_pose: 0.7370, loss: 0.0009
2021-05-25 04:53:55,478 - mmpose - INFO - Epoch [84][50/83]	lr: 5.000e-04, eta: 3:44:03, time: 2.234, data_time: 0.510, memory: 9832, mse_loss: 0.0009, acc_pose: 0.7365, loss: 0.0009
2021-05-25 04:56:38,651 - mmpose - INFO - Epoch [85][50/83]	lr: 5.000e-04, eta: 3:42:17, time: 2.132, data_time: 0.380, memory: 9832, mse_loss: 0.0009, acc_pose: 0.7342, loss: 0.0009
2021-05-25 04:59:19,839 - mmpose - INFO - Epoch [86][50/83]	lr: 5.000e-04, eta: 3:40:29, time: 2.095, data_time: 0.417, memory: 9832, mse_loss: 0.0009, acc_pose: 0.7380, loss: 0.0009
2021-05-25 05:02:09,808 - mmpose - INFO - Epoch [87][50/83]	lr: 5.000e-04, eta: 3:38:52, time: 2.267, data_time: 0.581, memory: 9832, mse_loss: 0.0009, acc_pose: 0.7240, loss: 0.0009
2021-05-25 05:04:51,458 - mmpose - INFO - Epoch [88][50/83]	lr: 5.000e-04, eta: 3:37:03, time: 2.094, data_time: 0.432, memory: 9832, mse_loss: 0.0009, acc_pose: 0.7351, loss: 0.0009
2021-05-25 05:07:31,889 - mmpose - INFO - Epoch [89][50/83]	lr: 5.000e-04, eta: 3:35:15, time: 2.095, data_time: 0.384, memory: 9832, mse_loss: 0.0009, acc_pose: 0.7381, loss: 0.0009
2021-05-25 05:10:10,973 - mmpose - INFO - Epoch [90][50/83]	lr: 5.000e-04, eta: 3:33:26, time: 2.090, data_time: 0.345, memory: 9832, mse_loss: 0.0009, acc_pose: 0.7396, loss: 0.0009
2021-05-25 05:11:06,977 - mmpose - INFO - Saving checkpoint at 90 epochs
2021-05-25 05:25:39,199 - mmpose - INFO - Now best checkpoint is epoch_90.pth
2021-05-25 05:25:39,650 - mmpose - INFO - Epoch(val) [90][83]	AP: 0.6359, AP .5: 0.8016, AP .75: 0.6916, AR: 0.7308, AR .5: 0.8930, AR .75: 0.7820, AP(E): 0.7198, AP(M): 0.6455, AP(H): 0.5206
2021-05-25 05:27:18,071 - mmpose - INFO - Epoch [91][50/83]	lr: 5.000e-04, eta: 3:31:29, time: 1.963, data_time: 0.323, memory: 9832, mse_loss: 0.0009, acc_pose: 0.7413, loss: 0.0009
2021-05-25 05:29:59,336 - mmpose - INFO - Epoch [92][50/83]	lr: 5.000e-04, eta: 3:29:40, time: 2.096, data_time: 0.411, memory: 9832, mse_loss: 0.0009, acc_pose: 0.7308, loss: 0.0009
2021-05-25 05:32:40,730 - mmpose - INFO - Epoch [93][50/83]	lr: 5.000e-04, eta: 3:27:53, time: 2.115, data_time: 0.425, memory: 9832, mse_loss: 0.0009, acc_pose: 0.7409, loss: 0.0009
2021-05-25 05:35:22,726 - mmpose - INFO - Epoch [94][50/83]	lr: 5.000e-04, eta: 3:26:05, time: 2.091, data_time: 0.374, memory: 9832, mse_loss: 0.0009, acc_pose: 0.7356, loss: 0.0009
2021-05-25 05:38:04,137 - mmpose - INFO - Epoch [95][50/83]	lr: 5.000e-04, eta: 3:24:16, time: 2.078, data_time: 0.345, memory: 9832, mse_loss: 0.0009, acc_pose: 0.7346, loss: 0.0009
2021-05-25 05:40:57,853 - mmpose - INFO - Epoch [96][50/83]	lr: 5.000e-04, eta: 3:22:43, time: 2.349, data_time: 0.418, memory: 9832, mse_loss: 0.0009, acc_pose: 0.7354, loss: 0.0009
2021-05-25 05:43:38,025 - mmpose - INFO - Epoch [97][50/83]	lr: 5.000e-04, eta: 3:20:55, time: 2.102, data_time: 0.369, memory: 9832, mse_loss: 0.0009, acc_pose: 0.7442, loss: 0.0009
2021-05-25 05:46:20,787 - mmpose - INFO - Epoch [98][50/83]	lr: 5.000e-04, eta: 3:19:09, time: 2.121, data_time: 0.337, memory: 9832, mse_loss: 0.0009, acc_pose: 0.7414, loss: 0.0009
2021-05-25 05:49:08,241 - mmpose - INFO - Epoch [99][50/83]	lr: 5.000e-04, eta: 3:17:24, time: 2.160, data_time: 0.603, memory: 9832, mse_loss: 0.0009, acc_pose: 0.7368, loss: 0.0009
2021-05-25 05:51:47,286 - mmpose - INFO - Epoch [100][50/83]	lr: 5.000e-04, eta: 3:15:35, time: 2.064, data_time: 0.422, memory: 9832, mse_loss: 0.0009, acc_pose: 0.7457, loss: 0.0009
2021-05-25 05:52:41,747 - mmpose - INFO - Saving checkpoint at 100 epochs
2021-05-25 06:06:56,062 - mmpose - INFO - Epoch(val) [100][83]	AP: 0.6335, AP .5: 0.7988, AP .75: 0.6882, AR: 0.7303, AR .5: 0.8926, AR .75: 0.7804, AP(E): 0.7156, AP(M): 0.6437, AP(H): 0.5184
2021-05-25 06:08:30,302 - mmpose - INFO - Epoch [101][50/83]	lr: 5.000e-04, eta: 3:13:35, time: 1.883, data_time: 0.321, memory: 9832, mse_loss: 0.0009, acc_pose: 0.7401, loss: 0.0009
2021-05-25 06:11:24,318 - mmpose - INFO - Epoch [102][50/83]	lr: 5.000e-04, eta: 3:11:57, time: 2.273, data_time: 0.407, memory: 9832, mse_loss: 0.0009, acc_pose: 0.7466, loss: 0.0009
2021-05-25 06:14:08,679 - mmpose - INFO - Epoch [103][50/83]	lr: 5.000e-04, eta: 3:10:11, time: 2.134, data_time: 0.369, memory: 9832, mse_loss: 0.0009, acc_pose: 0.7405, loss: 0.0009
2021-05-25 06:16:54,138 - mmpose - INFO - Epoch [104][50/83]	lr: 5.000e-04, eta: 3:08:25, time: 2.131, data_time: 0.370, memory: 9832, mse_loss: 0.0009, acc_pose: 0.7453, loss: 0.0009
2021-05-25 06:19:43,093 - mmpose - INFO - Epoch [105][50/83]	lr: 5.000e-04, eta: 3:06:44, time: 2.231, data_time: 0.704, memory: 9832, mse_loss: 0.0009, acc_pose: 0.7433, loss: 0.0009
2021-05-25 06:22:25,555 - mmpose - INFO - Epoch [106][50/83]	lr: 5.000e-04, eta: 3:04:56, time: 2.098, data_time: 0.352, memory: 9832, mse_loss: 0.0009, acc_pose: 0.7528, loss: 0.0009
2021-05-25 06:24:58,567 - mmpose - INFO - Epoch [107][50/83]	lr: 5.000e-04, eta: 3:03:04, time: 2.001, data_time: 0.315, memory: 9832, mse_loss: 0.0009, acc_pose: 0.7552, loss: 0.0009
2021-05-25 06:27:32,322 - mmpose - INFO - Epoch [108][50/83]	lr: 5.000e-04, eta: 3:01:20, time: 2.162, data_time: 0.295, memory: 9832, mse_loss: 0.0009, acc_pose: 0.7470, loss: 0.0009
2021-05-25 06:30:13,403 - mmpose - INFO - Epoch [109][50/83]	lr: 5.000e-04, eta: 2:59:33, time: 2.122, data_time: 0.375, memory: 9832, mse_loss: 0.0009, acc_pose: 0.7456, loss: 0.0009
2021-05-25 06:32:58,320 - mmpose - INFO - Epoch [110][50/83]	lr: 5.000e-04, eta: 2:57:47, time: 2.122, data_time: 0.396, memory: 9832, mse_loss: 0.0009, acc_pose: 0.7534, loss: 0.0009
2021-05-25 06:33:55,418 - mmpose - INFO - Saving checkpoint at 110 epochs
2021-05-25 06:48:26,715 - mmpose - INFO - Now best checkpoint is epoch_110.pth
2021-05-25 06:48:27,180 - mmpose - INFO - Epoch(val) [110][83]	AP: 0.6393, AP .5: 0.8012, AP .75: 0.6919, AR: 0.7350, AR .5: 0.8943, AR .75: 0.7857, AP(E): 0.7200, AP(M): 0.6498, AP(H): 0.5230
2021-05-25 06:50:05,967 - mmpose - INFO - Epoch [111][50/83]	lr: 5.000e-04, eta: 2:55:53, time: 1.954, data_time: 0.328, memory: 9832, mse_loss: 0.0009, acc_pose: 0.7477, loss: 0.0009
2021-05-25 06:52:44,707 - mmpose - INFO - Epoch [112][50/83]	lr: 5.000e-04, eta: 2:54:05, time: 2.081, data_time: 0.356, memory: 9832, mse_loss: 0.0009, acc_pose: 0.7505, loss: 0.0009
2021-05-25 06:55:19,952 - mmpose - INFO - Epoch [113][50/83]	lr: 5.000e-04, eta: 2:52:15, time: 2.038, data_time: 0.405, memory: 9832, mse_loss: 0.0009, acc_pose: 0.7455, loss: 0.0009
2021-05-25 06:58:03,782 - mmpose - INFO - Epoch [114][50/83]	lr: 5.000e-04, eta: 2:50:38, time: 2.341, data_time: 0.424, memory: 9832, mse_loss: 0.0009, acc_pose: 0.7470, loss: 0.0009
2021-05-25 07:00:45,324 - mmpose - INFO - Epoch [115][50/83]	lr: 5.000e-04, eta: 2:48:51, time: 2.090, data_time: 0.399, memory: 9832, mse_loss: 0.0009, acc_pose: 0.7486, loss: 0.0009
2021-05-25 07:03:26,900 - mmpose - INFO - Epoch [116][50/83]	lr: 5.000e-04, eta: 2:47:03, time: 2.098, data_time: 0.374, memory: 9832, mse_loss: 0.0009, acc_pose: 0.7510, loss: 0.0009
2021-05-25 07:06:09,015 - mmpose - INFO - Epoch [117][50/83]	lr: 5.000e-04, eta: 2:45:16, time: 2.094, data_time: 0.340, memory: 9832, mse_loss: 0.0009, acc_pose: 0.7508, loss: 0.0009
2021-05-25 07:08:49,186 - mmpose - INFO - Epoch [118][50/83]	lr: 5.000e-04, eta: 2:43:27, time: 2.060, data_time: 0.346, memory: 9832, mse_loss: 0.0009, acc_pose: 0.7494, loss: 0.0009
2021-05-25 07:11:33,660 - mmpose - INFO - Epoch [119][50/83]	lr: 5.000e-04, eta: 2:41:41, time: 2.110, data_time: 0.397, memory: 9832, mse_loss: 0.0009, acc_pose: 0.7484, loss: 0.0009
2021-05-25 07:14:14,964 - mmpose - INFO - Epoch [120][50/83]	lr: 5.000e-04, eta: 2:39:53, time: 2.087, data_time: 0.360, memory: 9832, mse_loss: 0.0009, acc_pose: 0.7564, loss: 0.0009
2021-05-25 07:15:13,628 - mmpose - INFO - Saving checkpoint at 120 epochs
2021-05-25 07:29:22,559 - mmpose - INFO - Now best checkpoint is epoch_120.pth
2021-05-25 07:29:23,048 - mmpose - INFO - Epoch(val) [120][83]	AP: 0.6401, AP .5: 0.7998, AP .75: 0.6917, AR: 0.7380, AR .5: 0.8964, AR .75: 0.7882, AP(E): 0.7225, AP(M): 0.6511, AP(H): 0.5237
2021-05-25 07:30:59,718 - mmpose - INFO - Epoch [121][50/83]	lr: 5.000e-04, eta: 2:38:00, time: 1.933, data_time: 0.307, memory: 9832, mse_loss: 0.0009, acc_pose: 0.7637, loss: 0.0009
2021-05-25 07:33:49,704 - mmpose - INFO - Epoch [122][50/83]	lr: 5.000e-04, eta: 2:36:19, time: 2.246, data_time: 0.383, memory: 9832, mse_loss: 0.0009, acc_pose: 0.7566, loss: 0.0009
2021-05-25 07:36:32,086 - mmpose - INFO - Epoch [123][50/83]	lr: 5.000e-04, eta: 2:34:33, time: 2.135, data_time: 0.359, memory: 9832, mse_loss: 0.0009, acc_pose: 0.7519, loss: 0.0009
2021-05-25 07:39:14,898 - mmpose - INFO - Epoch [124][50/83]	lr: 5.000e-04, eta: 2:32:47, time: 2.120, data_time: 0.380, memory: 9832, mse_loss: 0.0009, acc_pose: 0.7533, loss: 0.0009
2021-05-25 07:41:59,515 - mmpose - INFO - Epoch [125][50/83]	lr: 5.000e-04, eta: 2:30:59, time: 2.074, data_time: 0.441, memory: 9832, mse_loss: 0.0009, acc_pose: 0.7561, loss: 0.0009
2021-05-25 07:44:43,009 - mmpose - INFO - Epoch [126][50/83]	lr: 5.000e-04, eta: 2:29:13, time: 2.140, data_time: 0.506, memory: 9832, mse_loss: 0.0009, acc_pose: 0.7525, loss: 0.0009
2021-05-25 07:47:23,646 - mmpose - INFO - Epoch [127][50/83]	lr: 5.000e-04, eta: 2:27:26, time: 2.077, data_time: 0.367, memory: 9832, mse_loss: 0.0009, acc_pose: 0.7613, loss: 0.0009
2021-05-25 07:50:05,857 - mmpose - INFO - Epoch [128][50/83]	lr: 5.000e-04, eta: 2:25:40, time: 2.125, data_time: 0.423, memory: 9832, mse_loss: 0.0009, acc_pose: 0.7583, loss: 0.0009
2021-05-25 07:52:57,922 - mmpose - INFO - Epoch [129][50/83]	lr: 5.000e-04, eta: 2:24:00, time: 2.313, data_time: 0.525, memory: 9832, mse_loss: 0.0009, acc_pose: 0.7509, loss: 0.0009
2021-05-25 07:55:36,839 - mmpose - INFO - Epoch [130][50/83]	lr: 5.000e-04, eta: 2:22:11, time: 2.031, data_time: 0.343, memory: 9832, mse_loss: 0.0009, acc_pose: 0.7577, loss: 0.0009
2021-05-25 07:56:31,423 - mmpose - INFO - Saving checkpoint at 130 epochs
2021-05-25 08:10:34,103 - mmpose - INFO - Epoch(val) [130][83]	AP: 0.6392, AP .5: 0.8008, AP .75: 0.6924, AR: 0.7358, AR .5: 0.8957, AR .75: 0.7867, AP(E): 0.7215, AP(M): 0.6493, AP(H): 0.5291
2021-05-25 08:12:14,743 - mmpose - INFO - Epoch [131][50/83]	lr: 5.000e-04, eta: 2:20:21, time: 2.012, data_time: 0.324, memory: 9832, mse_loss: 0.0009, acc_pose: 0.7509, loss: 0.0009
2021-05-25 08:14:55,884 - mmpose - INFO - Epoch [132][50/83]	lr: 5.000e-04, eta: 2:18:35, time: 2.095, data_time: 0.368, memory: 9832, mse_loss: 0.0009, acc_pose: 0.7577, loss: 0.0009
2021-05-25 08:17:35,183 - mmpose - INFO - Epoch [133][50/83]	lr: 5.000e-04, eta: 2:16:47, time: 2.070, data_time: 0.345, memory: 9832, mse_loss: 0.0009, acc_pose: 0.7637, loss: 0.0009
2021-05-25 08:20:19,982 - mmpose - INFO - Epoch [134][50/83]	lr: 5.000e-04, eta: 2:15:02, time: 2.144, data_time: 0.338, memory: 9832, mse_loss: 0.0009, acc_pose: 0.7578, loss: 0.0009
2021-05-25 08:23:01,554 - mmpose - INFO - Epoch [135][50/83]	lr: 5.000e-04, eta: 2:13:16, time: 2.131, data_time: 0.378, memory: 9832, mse_loss: 0.0009, acc_pose: 0.7554, loss: 0.0009
2021-05-25 08:25:44,906 - mmpose - INFO - Epoch [136][50/83]	lr: 5.000e-04, eta: 2:11:30, time: 2.129, data_time: 0.436, memory: 9832, mse_loss: 0.0009, acc_pose: 0.7551, loss: 0.0009
2021-05-25 08:28:28,589 - mmpose - INFO - Epoch [137][50/83]	lr: 5.000e-04, eta: 2:09:44, time: 2.125, data_time: 0.353, memory: 9832, mse_loss: 0.0009, acc_pose: 0.7569, loss: 0.0009
2021-05-25 08:31:02,261 - mmpose - INFO - Epoch [138][50/83]	lr: 5.000e-04, eta: 2:07:55, time: 2.005, data_time: 0.485, memory: 9832, mse_loss: 0.0009, acc_pose: 0.7521, loss: 0.0009
2021-05-25 08:33:44,482 - mmpose - INFO - Epoch [139][50/83]	lr: 5.000e-04, eta: 2:06:14, time: 2.304, data_time: 0.565, memory: 9832, mse_loss: 0.0009, acc_pose: 0.7530, loss: 0.0009
2021-05-25 08:36:24,707 - mmpose - INFO - Epoch [140][50/83]	lr: 5.000e-04, eta: 2:04:26, time: 2.075, data_time: 0.407, memory: 9832, mse_loss: 0.0009, acc_pose: 0.7629, loss: 0.0009
2021-05-25 08:37:19,162 - mmpose - INFO - Saving checkpoint at 140 epochs
2021-05-25 08:51:57,507 - mmpose - INFO - Now best checkpoint is epoch_140.pth
2021-05-25 08:51:58,000 - mmpose - INFO - Epoch(val) [140][83]	AP: 0.6409, AP .5: 0.7990, AP .75: 0.6982, AR: 0.7409, AR .5: 0.8980, AR .75: 0.7922, AP(E): 0.7213, AP(M): 0.6532, AP(H): 0.5268
2021-05-25 08:53:34,863 - mmpose - INFO - Epoch [141][50/83]	lr: 5.000e-04, eta: 2:02:36, time: 1.936, data_time: 0.320, memory: 9832, mse_loss: 0.0009, acc_pose: 0.7594, loss: 0.0009
2021-05-25 08:56:13,171 - mmpose - INFO - Epoch [142][50/83]	lr: 5.000e-04, eta: 2:00:48, time: 2.053, data_time: 0.382, memory: 9832, mse_loss: 0.0009, acc_pose: 0.7671, loss: 0.0009
2021-05-25 08:58:52,911 - mmpose - INFO - Epoch [143][50/83]	lr: 5.000e-04, eta: 1:59:01, time: 2.073, data_time: 0.404, memory: 9832, mse_loss: 0.0009, acc_pose: 0.7583, loss: 0.0009
2021-05-25 09:01:24,021 - mmpose - INFO - Epoch [144][50/83]	lr: 5.000e-04, eta: 1:57:11, time: 1.955, data_time: 0.419, memory: 9832, mse_loss: 0.0009, acc_pose: 0.7582, loss: 0.0009
2021-05-25 09:04:04,169 - mmpose - INFO - Epoch [145][50/83]	lr: 5.000e-04, eta: 1:55:29, time: 2.265, data_time: 0.308, memory: 9832, mse_loss: 0.0009, acc_pose: 0.7511, loss: 0.0009
2021-05-25 09:06:43,678 - mmpose - INFO - Epoch [146][50/83]	lr: 5.000e-04, eta: 1:53:42, time: 2.079, data_time: 0.355, memory: 9832, mse_loss: 0.0009, acc_pose: 0.7648, loss: 0.0009
2021-05-25 09:09:33,185 - mmpose - INFO - Epoch [147][50/83]	lr: 5.000e-04, eta: 1:51:58, time: 2.228, data_time: 0.350, memory: 9832, mse_loss: 0.0009, acc_pose: 0.7552, loss: 0.0009
2021-05-25 09:12:12,243 - mmpose - INFO - Epoch [148][50/83]	lr: 5.000e-04, eta: 1:50:11, time: 2.073, data_time: 0.374, memory: 9832, mse_loss: 0.0009, acc_pose: 0.7607, loss: 0.0009
2021-05-25 09:14:51,878 - mmpose - INFO - Epoch [149][50/83]	lr: 5.000e-04, eta: 1:48:24, time: 2.076, data_time: 0.359, memory: 9832, mse_loss: 0.0009, acc_pose: 0.7540, loss: 0.0009
2021-05-25 09:17:34,750 - mmpose - INFO - Epoch [150][50/83]	lr: 5.000e-04, eta: 1:46:39, time: 2.127, data_time: 0.443, memory: 9832, mse_loss: 0.0009, acc_pose: 0.7697, loss: 0.0009
2021-05-25 09:18:30,818 - mmpose - INFO - Saving checkpoint at 150 epochs
2021-05-25 09:32:50,463 - mmpose - INFO - Now best checkpoint is epoch_150.pth
2021-05-25 09:32:50,883 - mmpose - INFO - Epoch(val) [150][83]	AP: 0.6441, AP .5: 0.8028, AP .75: 0.6953, AR: 0.7391, AR .5: 0.8950, AR .75: 0.7898, AP(E): 0.7265, AP(M): 0.6549, AP(H): 0.5310
2021-05-25 09:34:33,533 - mmpose - INFO - Epoch [151][50/83]	lr: 5.000e-04, eta: 1:44:51, time: 2.036, data_time: 0.281, memory: 9832, mse_loss: 0.0009, acc_pose: 0.7650, loss: 0.0009
2021-05-25 09:37:11,430 - mmpose - INFO - Epoch [152][50/83]	lr: 5.000e-04, eta: 1:43:04, time: 2.068, data_time: 0.409, memory: 9832, mse_loss: 0.0009, acc_pose: 0.7650, loss: 0.0009
2021-05-25 09:39:52,648 - mmpose - INFO - Epoch [153][50/83]	lr: 5.000e-04, eta: 1:41:18, time: 2.092, data_time: 0.396, memory: 9832, mse_loss: 0.0009, acc_pose: 0.7648, loss: 0.0009
2021-05-25 09:42:34,330 - mmpose - INFO - Epoch [154][50/83]	lr: 5.000e-04, eta: 1:39:32, time: 2.100, data_time: 0.380, memory: 9832, mse_loss: 0.0009, acc_pose: 0.7693, loss: 0.0009
2021-05-25 09:45:14,202 - mmpose - INFO - Epoch [155][50/83]	lr: 5.000e-04, eta: 1:37:45, time: 2.085, data_time: 0.347, memory: 9832, mse_loss: 0.0009, acc_pose: 0.7731, loss: 0.0009
2021-05-25 09:48:01,394 - mmpose - INFO - Epoch [156][50/83]	lr: 5.000e-04, eta: 1:36:01, time: 2.210, data_time: 0.506, memory: 9832, mse_loss: 0.0009, acc_pose: 0.7666, loss: 0.0009
2021-05-25 09:50:46,095 - mmpose - INFO - Epoch [157][50/83]	lr: 5.000e-04, eta: 1:34:15, time: 2.147, data_time: 0.405, memory: 9832, mse_loss: 0.0008, acc_pose: 0.7685, loss: 0.0008
2021-05-25 09:53:28,403 - mmpose - INFO - Epoch [158][50/83]	lr: 5.000e-04, eta: 1:32:29, time: 2.087, data_time: 0.370, memory: 9832, mse_loss: 0.0009, acc_pose: 0.7652, loss: 0.0009
2021-05-25 09:56:07,629 - mmpose - INFO - Epoch [159][50/83]	lr: 5.000e-04, eta: 1:30:42, time: 2.071, data_time: 0.379, memory: 9832, mse_loss: 0.0009, acc_pose: 0.7691, loss: 0.0009
2021-05-25 09:58:46,917 - mmpose - INFO - Epoch [160][50/83]	lr: 5.000e-04, eta: 1:28:56, time: 2.072, data_time: 0.360, memory: 9832, mse_loss: 0.0009, acc_pose: 0.7684, loss: 0.0009
2021-05-25 09:59:43,277 - mmpose - INFO - Saving checkpoint at 160 epochs
2021-05-25 10:13:55,894 - mmpose - INFO - Now best checkpoint is epoch_160.pth
2021-05-25 10:13:56,351 - mmpose - INFO - Epoch(val) [160][83]	AP: 0.6458, AP .5: 0.8010, AP .75: 0.7006, AR: 0.7416, AR .5: 0.8950, AR .75: 0.7919, AP(E): 0.7263, AP(M): 0.6566, AP(H): 0.5339
2021-05-25 10:15:34,543 - mmpose - INFO - Epoch [161][50/83]	lr: 5.000e-04, eta: 1:27:07, time: 1.954, data_time: 0.317, memory: 9832, mse_loss: 0.0009, acc_pose: 0.7724, loss: 0.0009
2021-05-25 10:18:13,360 - mmpose - INFO - Epoch [162][50/83]	lr: 5.000e-04, eta: 1:25:20, time: 2.046, data_time: 0.426, memory: 9832, mse_loss: 0.0009, acc_pose: 0.7622, loss: 0.0009
2021-05-25 10:20:51,604 - mmpose - INFO - Epoch [163][50/83]	lr: 5.000e-04, eta: 1:23:34, time: 2.074, data_time: 0.360, memory: 9832, mse_loss: 0.0008, acc_pose: 0.7793, loss: 0.0008
2021-05-25 10:23:29,815 - mmpose - INFO - Epoch [164][50/83]	lr: 5.000e-04, eta: 1:21:48, time: 2.072, data_time: 0.380, memory: 9832, mse_loss: 0.0009, acc_pose: 0.7722, loss: 0.0009
2021-05-25 10:26:09,278 - mmpose - INFO - Epoch [165][50/83]	lr: 5.000e-04, eta: 1:20:01, time: 2.076, data_time: 0.381, memory: 9832, mse_loss: 0.0008, acc_pose: 0.7734, loss: 0.0008
2021-05-25 10:28:50,697 - mmpose - INFO - Epoch [166][50/83]	lr: 5.000e-04, eta: 1:18:15, time: 2.108, data_time: 0.373, memory: 9832, mse_loss: 0.0008, acc_pose: 0.7788, loss: 0.0008
2021-05-25 10:31:34,269 - mmpose - INFO - Epoch [167][50/83]	lr: 5.000e-04, eta: 1:16:30, time: 2.120, data_time: 0.403, memory: 9832, mse_loss: 0.0008, acc_pose: 0.7681, loss: 0.0008
2021-05-25 10:34:12,078 - mmpose - INFO - Epoch [168][50/83]	lr: 5.000e-04, eta: 1:14:43, time: 2.060, data_time: 0.404, memory: 9832, mse_loss: 0.0008, acc_pose: 0.7738, loss: 0.0008
2021-05-25 10:36:45,094 - mmpose - INFO - Epoch [169][50/83]	lr: 5.000e-04, eta: 1:12:56, time: 2.009, data_time: 0.362, memory: 9832, mse_loss: 0.0008, acc_pose: 0.7819, loss: 0.0008
2021-05-25 10:39:22,125 - mmpose - INFO - Epoch [170][50/83]	lr: 5.000e-04, eta: 1:11:12, time: 2.214, data_time: 0.312, memory: 9832, mse_loss: 0.0008, acc_pose: 0.7701, loss: 0.0008
2021-05-25 10:40:18,763 - mmpose - INFO - Saving checkpoint at 170 epochs
2021-05-25 10:54:53,635 - mmpose - INFO - Epoch(val) [170][83]	AP: 0.6420, AP .5: 0.7987, AP .75: 0.6975, AR: 0.7404, AR .5: 0.8951, AR .75: 0.7906, AP(E): 0.7251, AP(M): 0.6540, AP(H): 0.5237
2021-05-25 10:56:30,045 - mmpose - INFO - Epoch [171][50/83]	lr: 5.000e-05, eta: 1:09:24, time: 1.925, data_time: 0.320, memory: 9832, mse_loss: 0.0008, acc_pose: 0.7807, loss: 0.0008
2021-05-25 10:59:17,004 - mmpose - INFO - Epoch [172][50/83]	lr: 5.000e-05, eta: 1:07:38, time: 2.092, data_time: 0.348, memory: 9832, mse_loss: 0.0008, acc_pose: 0.7787, loss: 0.0008
2021-05-25 11:02:00,155 - mmpose - INFO - Epoch [173][50/83]	lr: 5.000e-05, eta: 1:05:53, time: 2.167, data_time: 0.383, memory: 9832, mse_loss: 0.0008, acc_pose: 0.7830, loss: 0.0008
2021-05-25 11:04:52,050 - mmpose - INFO - Epoch [174][50/83]	lr: 5.000e-05, eta: 1:04:08, time: 2.260, data_time: 0.540, memory: 9832, mse_loss: 0.0008, acc_pose: 0.7869, loss: 0.0008
2021-05-25 11:07:16,671 - mmpose - INFO - Epoch [175][50/83]	lr: 5.000e-05, eta: 1:02:20, time: 1.832, data_time: 0.380, memory: 9832, mse_loss: 0.0008, acc_pose: 0.7885, loss: 0.0008
2021-05-25 11:09:56,255 - mmpose - INFO - Epoch [176][50/83]	lr: 5.000e-05, eta: 1:00:35, time: 2.246, data_time: 0.330, memory: 9832, mse_loss: 0.0008, acc_pose: 0.7854, loss: 0.0008
2021-05-25 11:12:32,988 - mmpose - INFO - Epoch [177][50/83]	lr: 5.000e-05, eta: 0:58:49, time: 2.031, data_time: 0.353, memory: 9832, mse_loss: 0.0008, acc_pose: 0.7869, loss: 0.0008
2021-05-25 11:15:11,824 - mmpose - INFO - Epoch [178][50/83]	lr: 5.000e-05, eta: 0:57:03, time: 2.069, data_time: 0.361, memory: 9832, mse_loss: 0.0008, acc_pose: 0.7920, loss: 0.0008
2021-05-25 11:17:53,618 - mmpose - INFO - Epoch [179][50/83]	lr: 5.000e-05, eta: 0:55:17, time: 2.086, data_time: 0.356, memory: 9832, mse_loss: 0.0008, acc_pose: 0.7880, loss: 0.0008
2021-05-25 11:20:33,988 - mmpose - INFO - Epoch [180][50/83]	lr: 5.000e-05, eta: 0:53:31, time: 2.088, data_time: 0.349, memory: 9832, mse_loss: 0.0008, acc_pose: 0.7964, loss: 0.0008
2021-05-25 11:21:30,102 - mmpose - INFO - Saving checkpoint at 180 epochs
2021-05-25 11:36:10,193 - mmpose - INFO - Now best checkpoint is epoch_180.pth
2021-05-25 11:36:10,586 - mmpose - INFO - Epoch(val) [180][83]	AP: 0.6591, AP .5: 0.8094, AP .75: 0.7128, AR: 0.7543, AR .5: 0.9003, AR .75: 0.8048, AP(E): 0.7385, AP(M): 0.6685, AP(H): 0.5483
2021-05-25 11:37:39,113 - mmpose - INFO - Epoch [181][50/83]	lr: 5.000e-05, eta: 0:51:43, time: 1.769, data_time: 0.399, memory: 9832, mse_loss: 0.0008, acc_pose: 0.7866, loss: 0.0008
2021-05-25 11:40:23,306 - mmpose - INFO - Epoch [182][50/83]	lr: 5.000e-05, eta: 0:49:59, time: 2.322, data_time: 0.321, memory: 9832, mse_loss: 0.0008, acc_pose: 0.7937, loss: 0.0008
2021-05-25 11:43:06,565 - mmpose - INFO - Epoch [183][50/83]	lr: 5.000e-05, eta: 0:48:13, time: 2.159, data_time: 0.350, memory: 9832, mse_loss: 0.0008, acc_pose: 0.7944, loss: 0.0008
2021-05-25 11:45:49,164 - mmpose - INFO - Epoch [184][50/83]	lr: 5.000e-05, eta: 0:46:28, time: 2.135, data_time: 0.368, memory: 9832, mse_loss: 0.0008, acc_pose: 0.7999, loss: 0.0008
2021-05-25 11:48:32,633 - mmpose - INFO - Epoch [185][50/83]	lr: 5.000e-05, eta: 0:44:42, time: 2.124, data_time: 0.416, memory: 9832, mse_loss: 0.0008, acc_pose: 0.8013, loss: 0.0008
2021-05-25 11:51:11,796 - mmpose - INFO - Epoch [186][50/83]	lr: 5.000e-05, eta: 0:42:56, time: 2.060, data_time: 0.357, memory: 9832, mse_loss: 0.0008, acc_pose: 0.7936, loss: 0.0008
2021-05-25 11:53:55,807 - mmpose - INFO - Epoch [187][50/83]	lr: 5.000e-05, eta: 0:41:11, time: 2.158, data_time: 0.353, memory: 9832, mse_loss: 0.0008, acc_pose: 0.7899, loss: 0.0008
2021-05-25 11:56:38,644 - mmpose - INFO - Epoch [188][50/83]	lr: 5.000e-05, eta: 0:39:26, time: 2.118, data_time: 0.363, memory: 9832, mse_loss: 0.0008, acc_pose: 0.7901, loss: 0.0008
2021-05-25 11:59:26,362 - mmpose - INFO - Epoch [189][50/83]	lr: 5.000e-05, eta: 0:37:40, time: 2.208, data_time: 0.435, memory: 9832, mse_loss: 0.0008, acc_pose: 0.7914, loss: 0.0008
2021-05-25 12:02:11,555 - mmpose - INFO - Epoch [190][50/83]	lr: 5.000e-05, eta: 0:35:55, time: 2.200, data_time: 0.374, memory: 9832, mse_loss: 0.0008, acc_pose: 0.7945, loss: 0.0008
2021-05-25 12:03:09,302 - mmpose - INFO - Saving checkpoint at 190 epochs
2021-05-25 12:17:19,157 - mmpose - INFO - Now best checkpoint is epoch_190.pth
2021-05-25 12:17:19,613 - mmpose - INFO - Epoch(val) [190][83]	AP: 0.6609, AP .5: 0.8093, AP .75: 0.7132, AR: 0.7558, AR .5: 0.9000, AR .75: 0.8063, AP(E): 0.7391, AP(M): 0.6700, AP(H): 0.5503
2021-05-25 12:19:00,205 - mmpose - INFO - Epoch [191][50/83]	lr: 5.000e-05, eta: 0:34:09, time: 2.011, data_time: 0.484, memory: 9832, mse_loss: 0.0008, acc_pose: 0.7976, loss: 0.0008
2021-05-25 12:22:01,952 - mmpose - INFO - Epoch [192][50/83]	lr: 5.000e-05, eta: 0:32:25, time: 2.502, data_time: 0.816, memory: 9832, mse_loss: 0.0008, acc_pose: 0.7971, loss: 0.0008
2021-05-25 12:24:44,242 - mmpose - INFO - Epoch [193][50/83]	lr: 5.000e-05, eta: 0:30:39, time: 2.080, data_time: 0.456, memory: 9832, mse_loss: 0.0008, acc_pose: 0.7979, loss: 0.0008
2021-05-25 12:27:28,002 - mmpose - INFO - Epoch [194][50/83]	lr: 5.000e-05, eta: 0:28:54, time: 2.153, data_time: 0.597, memory: 9832, mse_loss: 0.0008, acc_pose: 0.8037, loss: 0.0008
2021-05-25 12:30:12,325 - mmpose - INFO - Epoch [195][50/83]	lr: 5.000e-05, eta: 0:27:08, time: 2.134, data_time: 0.559, memory: 9832, mse_loss: 0.0008, acc_pose: 0.8026, loss: 0.0008
2021-05-25 12:32:58,858 - mmpose - INFO - Epoch [196][50/83]	lr: 5.000e-05, eta: 0:25:22, time: 2.158, data_time: 0.429, memory: 9832, mse_loss: 0.0008, acc_pose: 0.7998, loss: 0.0008
2021-05-25 12:35:44,213 - mmpose - INFO - Epoch [197][50/83]	lr: 5.000e-05, eta: 0:23:37, time: 2.179, data_time: 0.389, memory: 9832, mse_loss: 0.0008, acc_pose: 0.7941, loss: 0.0008
2021-05-25 12:38:30,742 - mmpose - INFO - Epoch [198][50/83]	lr: 5.000e-05, eta: 0:21:51, time: 2.147, data_time: 0.368, memory: 9832, mse_loss: 0.0008, acc_pose: 0.8056, loss: 0.0008
2021-05-25 12:41:27,634 - mmpose - INFO - Epoch [199][50/83]	lr: 5.000e-05, eta: 0:20:06, time: 2.198, data_time: 0.448, memory: 9832, mse_loss: 0.0008, acc_pose: 0.8006, loss: 0.0008
2021-05-25 12:44:04,470 - mmpose - INFO - Epoch [200][50/83]	lr: 5.000e-05, eta: 0:18:20, time: 2.205, data_time: 0.313, memory: 9832, mse_loss: 0.0008, acc_pose: 0.8054, loss: 0.0008
2021-05-25 12:45:00,959 - mmpose - INFO - Saving checkpoint at 200 epochs
2021-05-25 12:59:34,531 - mmpose - INFO - Now best checkpoint is epoch_200.pth
2021-05-25 12:59:34,983 - mmpose - INFO - Epoch(val) [200][83]	AP: 0.6617, AP .5: 0.8096, AP .75: 0.7141, AR: 0.7563, AR .5: 0.9004, AR .75: 0.8076, AP(E): 0.7408, AP(M): 0.6707, AP(H): 0.5469
2021-05-25 13:01:14,251 - mmpose - INFO - Epoch [201][50/83]	lr: 5.000e-06, eta: 0:16:34, time: 1.983, data_time: 0.306, memory: 9832, mse_loss: 0.0008, acc_pose: 0.8002, loss: 0.0008
2021-05-25 13:03:55,555 - mmpose - INFO - Epoch [202][50/83]	lr: 5.000e-06, eta: 0:14:48, time: 2.082, data_time: 0.412, memory: 9832, mse_loss: 0.0008, acc_pose: 0.8060, loss: 0.0008
2021-05-25 13:06:33,872 - mmpose - INFO - Epoch [203][50/83]	lr: 5.000e-06, eta: 0:13:02, time: 2.065, data_time: 0.360, memory: 9832, mse_loss: 0.0008, acc_pose: 0.8013, loss: 0.0008
2021-05-25 13:09:15,236 - mmpose - INFO - Epoch [204][50/83]	lr: 5.000e-06, eta: 0:11:16, time: 2.133, data_time: 0.373, memory: 9832, mse_loss: 0.0008, acc_pose: 0.7944, loss: 0.0008
2021-05-25 13:11:52,766 - mmpose - INFO - Epoch [205][50/83]	lr: 5.000e-06, eta: 0:09:30, time: 2.041, data_time: 0.332, memory: 9832, mse_loss: 0.0008, acc_pose: 0.8060, loss: 0.0008
2021-05-25 13:14:22,859 - mmpose - INFO - Epoch [206][50/83]	lr: 5.000e-06, eta: 0:07:45, time: 2.076, data_time: 0.305, memory: 9832, mse_loss: 0.0008, acc_pose: 0.8011, loss: 0.0008
2021-05-25 13:17:16,310 - mmpose - INFO - Epoch [207][50/83]	lr: 5.000e-06, eta: 0:05:59, time: 2.365, data_time: 0.360, memory: 9832, mse_loss: 0.0008, acc_pose: 0.8019, loss: 0.0008
2021-05-25 13:20:04,637 - mmpose - INFO - Epoch [208][50/83]	lr: 5.000e-06, eta: 0:04:13, time: 2.206, data_time: 0.485, memory: 9832, mse_loss: 0.0008, acc_pose: 0.8012, loss: 0.0008
2021-05-25 13:22:44,737 - mmpose - INFO - Epoch [209][50/83]	lr: 5.000e-06, eta: 0:02:27, time: 2.103, data_time: 0.385, memory: 9832, mse_loss: 0.0008, acc_pose: 0.8020, loss: 0.0008
2021-05-25 13:25:26,434 - mmpose - INFO - Epoch [210][50/83]	lr: 5.000e-06, eta: 0:00:42, time: 2.119, data_time: 0.355, memory: 9832, mse_loss: 0.0008, acc_pose: 0.8076, loss: 0.0008
2021-05-25 13:26:20,773 - mmpose - INFO - Saving checkpoint at 210 epochs
2021-05-25 13:40:45,922 - mmpose - INFO - Now best checkpoint is epoch_210.pth
2021-05-25 13:40:46,336 - mmpose - INFO - Epoch(val) [210][83]	AP: 0.6622, AP .5: 0.8100, AP .75: 0.7146, AR: 0.7571, AR .5: 0.9011, AR .75: 0.8084, AP(E): 0.7418, AP(M): 0.6715, AP(H): 0.5497

[leeyegy]

any suggestion would be appreciated:)

[innerlee]

You may train multiple times, and compare with https://download.openmmlab.com/mmpose/top_down/hrnet/hrnet_w32_crowdpose_256x192_20201227.log.json

[jin-s13]

We use 8 gpus.

[leeyegy]

Thanks so much for your reply.
I notice that the learning rate in https://download.openmmlab.com/mmpose/top_down/hrnet/hrnet_w32_crowdpose_256x192_20201227.log.json ending with 1e-05 while what I got ending with 5.000e-06.
Have you ever changed the config in configs/top_down/hrnet/crowdpose/hrnet_w32_crowdpose_256x192.py since you report the log
https://download.openmmlab.com/mmpose/top_down/hrnet/hrnet_w32_crowdpose_256x192_20201227.log.json
, which may result in the difference?

[jin-s13]

That's interesting. I do not remember I changed the config files. Not sure why the learning rates are different :)