Replies: 2 comments 1 reply
-
|
Please see #2461 |
Beta Was this translation helpful? Give feedback.
1 reply
-
|
Then to train GPT-2 model, I have to use deep speed v0.3.0 and GPT from NVIDIA? |
Beta Was this translation helpful? Give feedback.
-
|
The general ZeRO-Offload support, excluding DPU, is available in DeepSpeed master branch. Please see this tutorial. |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
I wonder that is there any code which is used in experiment method of ZeRO-Offload paper.
thank you
Beta Was this translation helpful? Give feedback.
All reactions