Batch size for 80GB GPU memory

[mehadi92]

In here you are suggesting batch 64 for 80 GB GPU memory. Which model type (small, medium, large, xlarge) does this batch size applicable?

[titu1994]

This is for the Large model (the default in the config) with 120 M params

[mehadi92]

@titu1994 ,

you are suggesting batch size 32 for 32G GPU memory. But for 80G GPU memory, you are suggesting batch size 64. Is there any specific reason for it? should not it be 80 batch size for GPU memory 80G?

Thanks

[titu1994]

Training memory doesn't work like that. It's not 1 gb per sample.