Regarding the reproduction of the paper

[blofn]

I noticed that the structure in your code is inconsistent with the structure in the paper. In your code, VoCoHead uses a student-teacher model, which is not mentioned in your paper. So, how is the method in your paper implemented?

[Luffy03]

Hi, many thanks for your attention to our work!
Yes, as you say, the current version has been updated with the teacher-student models, which are advanced from our previous conference paper. The details will be introduced in our extension paper.

[jsadu826]

Hi, could you please upload the code of the original model used in the CVPR 2024 paper?

Hi, many thanks for your attention to our work! Yes, as you say, the current version has been updated with the teacher-student models, which are advanced from our previous conference paper. The details will be introduced in our extension paper.

[Luffy03]

Ok, no problem! I will check it again and upload it as soon as possible.

[Luffy03]

I am so sorry for my late reply since I was on a travel. Here is the code link of the old version https://www.dropbox.com/scl/fi/ounrqw35msacn2tynm87s/voco_headv1_old.py?rlkey=ykgu5xcdt6k07plonarp8bzw7&st=zxlqifai&dl=0

[jsadu826]

I am so sorry for my late reply since I was on a travel. Here is the code link of the old version https://www.dropbox.com/scl/fi/ounrqw35msacn2tynm87s/voco_headv1_old.py?rlkey=ykgu5xcdt6k07plonarp8bzw7&st=zxlqifai&dl=0

Hello, when I pretrained VoCo on BTCV, TCIA Covid19, and LUNA16 using your old versioin of voco_head for about 60000 steps, I found the training loss not decreasing.

part_of_training_log.txt

The implementation details followed your CVPR paper, using 1 V100 GPU:

• warmup_steps=1000
• sw_batch_size=4
• lr=0.001
• reg_weight=0.01
• batch_size=2 (maximum on 1 V100 GPU)

[Luffy03]

Weired. It seems the training loss is not consistent with our provided training log https://www.dropbox.com/scl/fi/rmqy9n2gio5tptbhlt239/20240115_232208.txt?rlkey=0jmnpz3n77bb1b9r9wt9aqkrv&dl=0. This training log is produced by the old version.
Then, would you please try to change lr as 1e-4?

VoCo/voco_train.py

Line 148 in 4c3fecc

parser.add_argument("--lr", default=1e-4, type=float, help="learning rate")

[Luffy03]

Maybe it is caused by different GPU versions? I have tried H800 and A800, but I have not yet tried V100.

[jsadu826]

Thank you! I'll try lr=1e-4.

Btw, I did some code modification to adapt to the current github repo version. I'd be glad if you could help me double check.

• In voco_head_old.py/online_assign

• In data_utils.py/VoCoAugmentation

[Luffy03]

Seems no problem currently.

[Devil-Ideal]

I am so sorry for my late reply since I was on a travel. Here is the code link of the old version https://www.dropbox.com/scl/fi/ounrqw35msacn2tynm87s/voco_headv1_old.py?rlkey=ykgu5xcdt6k07plonarp8bzw7&st=zxlqifai&dl=0

Hi ! How many GPUs are needed for the old model (in CVPR2024) during training, what is the GPU memory of a single card and what is batch size?

[Luffy03]

I am so sorry for my late reply since I was on a travel. Here is the code link of the old version https://www.dropbox.com/scl/fi/ounrqw35msacn2tynm87s/voco_headv1_old.py?rlkey=ykgu5xcdt6k07plonarp8bzw7&st=zxlqifai&dl=0

Hi ! How many GPUs are needed for the old model (in CVPR2024) during training, what is the GPU memory of a single card and what is batch size?

For the CVPR version we also use H800 (80G memory) and the batch size is 4.

[jsadu826]

Here is an update on reproduction.

• device: 1 H800 (80G memory) GPU
• framework: voco_headv1_old
• pretraining datasets: BTCV, LUNA16, TCIA Covid19
• SwinUNETR: use_v2 = True
• same as in the original paper:
  • warmup_steps = 100
  • lr = 1e-3
  • decay = 1e-2
  • batch_size = 4
  • sw_batch_size = 4

My training log and modification to the official code (based on commit f70606b): modified_files_and_log.zip

Loss curves:

#24 @FengheTan9

[Luffy03]

Thank your very much