Can it support multi-node training? #11

Open

zidong-onepiece1

opened

on Sep 11, 2024

I want to distill 7B from 14B or bigger.

GPU: 8*a800
ERROR_MSG : CUDA OOM

How to do multi-node training?

Metadata

Assignees

No one assigned

Labels

No labels

No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests