About training time

[ashevimostar]

I use a 3090 card to train for 400 epochs, which takes about a week. May I ask what your device is and how long the training time is? Is there a problem with my training time, is there a way to accelerate, or is there a way to change my parameters.

CUDA_VISIBLE_DEVICES={DEVICES} python3 main.py
--mode train
--model_type DDIM
--img_size {IMAGE_SIZE}
--num_img_channels {NUM_IMAGE_CHANNELS}
--dataset {DATASET_NAME}
--img_dir {DATA_FOLDER}
--train_batch_size 16
--eval_batch_size 8
--num_epochs 400

[nickk124]

Hi, thanks for your interest in our project! IIRC, when training a 256x256 model on a dataset of roughly 10K images for the same number of epochs and batch size as you, it took a few days, and this was on one 48 GB VRAM A6000. I have a few follow up questions for you:

1. what is your training set's size?
2. what is your image size? how many channels?
   At first glance, a week of training seems like it may be on par, diffusion models take a lot of time to train. This is just kind of a general limitation of these models, and you'll likely see faster training time (although maybe some reduced image quality) for GAN based models like StyleGAN 2.