#

model-parallization

Here is 1 public repository matching this topic...

CoinCheung / gdGPT

Train llm (bloom, llama, baichuan2-7b, chatglm3-6b) with deepspeed pipeline mode. Faster than zero/zero++/fsdp.

nlp bloom pipeline pytorch deepspeed llm full-finetune model-parallization flash-attention llama2 baichuan2-7b chatglm3-6b mixtral-8x7b

Updated Feb 5, 2024
Python

Improve this page

Add a description, image, and links to the model-parallization topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with the model-parallization topic, visit your repo's landing page and select "manage topics."