-
|
in Document https://github.com/deepspeedai/DeepSpeed/tree/master/blogs/huggingface-tp there is no Supported Models list， |
Beta Was this translation helpful? Give feedback.
Replies: 2 comments 1 reply
-
Beta Was this translation helpful? Give feedback.
-
|
AutoTP inference just support models in this list. AutoTP training is a subset of this list. AutoTP uses pattern matching to look for modules in the model for tensor parallel, models out of this list may need to match with new pattern. Some models out of this list also have special architecture that are not compatible with AutoTP. If you verified AutoTP training works for certain model, you may also submit a PR to update the list in the tutorial to mark it as training supported. This will help DeepSpeed community, thanks! |
Beta Was this translation helpful? Give feedback.
-
|
tks for help, thank you |
Beta Was this translation helpful? Give feedback.
AutoTP inference just support models in this list. AutoTP training is a subset of this list.
AutoTP uses pattern matching to look for modules in the model for tensor parallel, models out of this list may need to match with new pattern. Some models out of this list also have special architecture that are not compatible with AutoTP.
If you verified AutoTP training works for certain model, you may also submit a PR to update the list in the tutorial to mark it as training supported. This will help DeepSpeed community, thanks!