dataset preprocess

[WangXiaokang610]

Hello, I encountered an issue while trying to reproduce the code. The RGBT-CC dataset's gt files have keypoints with a shape of [num_points, 2]. However, in the crowd.py file under the Fine-tuning folder, code : nearest_dis = np.clip(keypoints[:, 2], 4.0, 128.0) throws an error because my keypoints only have two columns (shape [num_points, 2]) and cannot access the third column.
Could you please provide the dataset preprocessing steps or clarify the expected format for the keypoints in the dataset? This would greatly help in resolving the issue and successfully reproducing the code. Thank you!

[HenryCilence]

Hello, thanks for your attention.
Our .npy file format is consistent with the format of the Bayesian Loss source code. The third column indicates the distances between several nearest annotation points and the current annotation. (Please refer to https://github.com/ZhihengCV/Bayesian-Crowd-Counting ) You can also contact me by email if you'd like to and I will share you our processed training set .npy file.

[WangXiaokang610]

Hello, I refer the preprocess_dataset.py in Bayesian-Crowd-Counting and generate the .npy file. But I have some problems in fine tune stage. In fine tune stage, the training loss is not decreasing.


The first image is the missing keys before I load the pratrained weight, and the second image is the missing keys after I load the pretrained weight. And this is my training set data preprocess code:

def find_dis(point):
square = np.sum(pointpoints, axis=1)
dis = np.sqrt(np.maximum(square[:, None] - 2np.matmul(point, point.T) + square[None, :], 0.0))
dis = np.mean(np.partition(dis, 3, axis=1)[:, 1:4], axis=1, keepdims=True)
return dis

def generate_data(label_path):
rgb_path = label_path.replace('GT', 'RGB').replace('json', 'jpg')
t_path = label_path.replace('GT', 'T').replace('json', 'jpg')
rgb = cv2.imread(rgb_path)[..., ::-1].copy()
t = cv2.imread(t_path)[..., ::-1].copy()
im_h, im_w, _ = rgb.shape
print('rgb and t shape', rgb.shape, t.shape)
with open(label_path, 'r') as f:
label_file = json.load(f)
points = np.asarray(label_file['points'])
# print('points', points.shape)
idx_mask = (points[:, 0] >= 0) * (points[:, 0] <= im_w) * (points[:, 1] >= 0) * (points[:, 1] <= im_h)
points = points[idx_mask]
return rgb, t, points

if name == 'main':

root_path = '/mnt/amaxdir/RGBTCrowdCounting-main/BL+IADM for RGBT Crowd Counting/RGBT-CC-CVPR2021'  # dataset root path
save_dir = '/mnt/amaxdir/RGBTCrowdCounting-main/BL+IADM for RGBT Crowd Counting/RGBT-CC-CVPR2021'
# save_dir = '/data2/cjq/RGBT-test'

#for phase in ['train', 'val', 'test']:
for phase in ['train']:
    sub_dir = os.path.join(root_path, phase)
    sub_save_dir = os.path.join(save_dir, phase)
    if not os.path.exists(sub_save_dir):
        os.makedirs(sub_save_dir)
    gt_list = glob(os.path.join(sub_dir, '*json'))
    for gt_path in gt_list:
        name = os.path.basename(gt_path)
        # print('name', name)
        rgb, t, points = generate_data(gt_path)
        dis = find_dis(points)
        points = np.concatenate((points, dis), axis=1)
        im_save_path = os.path.join(sub_save_dir, name)
        rgb_save_path = im_save_path.replace('GT', 'RGB').replace('json', 'jpg')
        t_save_path = im_save_path.replace('GT', 'T').replace('json', 'jpg')
        cv2.imwrite(rgb_save_path, rgb)
        cv2.imwrite(t_save_path, t)
        gd_save_path = im_save_path.replace('json', 'npy')
        np.save(gd_save_path, points)

Could you please help me take a look and see where the issue might be?

[WangXiaokang610]

Hello, I would like to ask if you could kindly share the processed training set .npy file or the datapreprocess.py script with me? I refer to the preprocess.py from Bayesian Crowd Counting and IADM to generate the .npy file in RGBT dataset. However, during the fine-tuning stage, I encounterd an issue where the loss does not decrease. My email is wangxiaokang610@gmail.com.

[HenryCilence]

Hello.
Sure, I will contact you via email later.

[save-me-ctrl-c]

Hello, could you please give me a copy of the data processing code? I'm also trying to reproduce it.

[RrOoOoTt]

Hello, I have added the data preprocessing code referring to https://github.com/zhiheng-ma/Bayesian-Crowd-Counting/. However, I am encountering the same issue where my training loss is not decreasing during the fine-tuning stage and remains around 60. Could you please share the preprocessed .npy data or the data preprocessing code with me?

[HenryCilence]

Could you leave an e-mail address so I can contact you?

[RrOoOoTt]

Could you leave an e-mail address so I can contact you?

Thank you. My email address is rroooott@foxmail.com.

[HenryCilence]

Thank you! I'll contact you tomorrow.

[Cht2924]

你好，请问数据预处理的脚本，可以分享一下吗，谢谢！邮箱是 cht2924@gmail.com

[HenryCilence]

你好，请问数据预处理的脚本，可以分享一下吗，谢谢！邮箱是 cht2924@gmail.com

您好，已经发送

[SeijiZains]

Hello, I would like to reproduce your code on the DroneRGBT dataset. However, I noticed in the comments that the .npy files in the dataset appear to require specific processing, and generating another modality during preprocessing seems time-consuming. Would it be possible for you to share the preprocessed dataset (including the third modality) with me? My email is SeijiZains915@gmail.com. Thank you in advance for your assistance!

[HenryCilence]

Hello, I would like to reproduce your code on the DroneRGBT dataset. However, I noticed in the comments that the .npy files in the dataset appear to require specific processing, and generating another modality during preprocessing seems time-consuming. Would it be possible for you to share the preprocessed dataset (including the third modality) with me? My email is SeijiZains915@gmail.com. Thank you in advance for your assistance!

Hello, I've sent the files you mentioned.