@@ -330,6 +330,7 @@ Specified using `--task generate`.
330
330
| ` Ernie4_5_ForCausalLM ` | Ernie4.5 | ` baidu/ERNIE-4.5-0.3B-PT ` , etc. | | ✅︎ | ✅︎ |
331
331
| ` Ernie4_5_MoeForCausalLM ` | Ernie4.5MoE | ` baidu/ERNIE-4.5-21B-A3B-PT ` , ` baidu/ERNIE-4.5-300B-A47B-PT ` , etc. | | ✅︎ | ✅︎ |
332
332
| ` ExaoneForCausalLM ` | EXAONE-3 | ` LGAI-EXAONE/EXAONE-3.0-7.8B-Instruct ` , etc. | ✅︎ | ✅︎ | ✅︎ |
333
+ | ` Fairseq2LlamaForCausalLM ` | Fairseq2Llama | ` mgleize/fairseq2-dummy-Llama-3.2-1B ` , etc. | ✅︎ | ✅︎ | ✅︎ |
333
334
| ` FalconForCausalLM ` | Falcon | ` tiiuae/falcon-7b ` , ` tiiuae/falcon-40b ` , ` tiiuae/falcon-rw-7b ` , etc. | | ✅︎ | ✅︎ |
334
335
| ` FalconMambaForCausalLM ` | FalconMamba | ` tiiuae/falcon-mamba-7b ` , ` tiiuae/falcon-mamba-7b-instruct ` , etc. | | ✅︎ | ✅︎ |
335
336
| ` FalconH1ForCausalLM ` | Falcon-H1 | ` tiiuae/Falcon-H1-34B-Base ` , ` tiiuae/Falcon-H1-34B-Instruct ` , etc. | ✅︎ | ✅︎ | ✅︎ |
@@ -352,16 +353,19 @@ Specified using `--task generate`.
352
353
| ` HunYuanMoEV1ForCausalLM ` | Hunyuan-80B-A13B | ` tencent/Hunyuan-A13B-Instruct ` , ` tencent/Hunyuan-A13B-Pretrain ` , ` tencent/Hunyuan-A13B-Instruct-FP8 ` , etc. | | | ✅︎ |
353
354
| ` InternLMForCausalLM ` | InternLM | ` internlm/internlm-7b ` , ` internlm/internlm-chat-7b ` , etc. | ✅︎ | ✅︎ | ✅︎ |
354
355
| ` InternLM2ForCausalLM ` | InternLM2 | ` internlm/internlm2-7b ` , ` internlm/internlm2-chat-7b ` , etc. | ✅︎ | ✅︎ | ✅︎ |
356
+ | ` InternLM2VEForCausalLM ` | InternLM2VE | ` OpenGVLab/Mono-InternVL-2B ` , etc. | ✅︎ | ✅︎ | ✅︎ |
355
357
| ` InternLM3ForCausalLM ` | InternLM3 | ` internlm/internlm3-8b-instruct ` , etc. | ✅︎ | ✅︎ | ✅︎ |
356
358
| ` JAISLMHeadModel ` | Jais | ` inceptionai/jais-13b ` , ` inceptionai/jais-13b-chat ` , ` inceptionai/jais-30b-v3 ` , ` inceptionai/jais-30b-chat-v3 ` , etc. | | ✅︎ | ✅︎ |
357
359
| ` JambaForCausalLM ` | Jamba | ` ai21labs/AI21-Jamba-1.5-Large ` , ` ai21labs/AI21-Jamba-1.5-Mini ` , ` ai21labs/Jamba-v0.1 ` , etc. | ✅︎ | ✅︎ | |
358
360
| ` LlamaForCausalLM ` | Llama 3.1, Llama 3, Llama 2, LLaMA, Yi | ` meta-llama/Meta-Llama-3.1-405B-Instruct ` , ` meta-llama/Meta-Llama-3.1-70B ` , ` meta-llama/Meta-Llama-3-70B-Instruct ` , ` meta-llama/Llama-2-70b-hf ` , ` 01-ai/Yi-34B ` , etc. | ✅︎ | ✅︎ | ✅︎ |
359
361
| ` MambaForCausalLM ` | Mamba | ` state-spaces/mamba-130m-hf ` , ` state-spaces/mamba-790m-hf ` , ` state-spaces/mamba-2.8b-hf ` , etc. | | ✅︎ | |
360
362
| ` Mamba2ForCausalLM ` | Mamba2 | ` mistralai/Mamba-Codestral-7B-v0.1 ` , etc. | | ✅︎ | ✅︎ |
363
+ | ` MiMoForCausalLM ` | MiMo | ` XiaomiMiMo/MiMo-7B-RL ` , etc. | ✅︎ | ✅︎ | ✅︎ |
361
364
| ` MiniCPMForCausalLM ` | MiniCPM | ` openbmb/MiniCPM-2B-sft-bf16 ` , ` openbmb/MiniCPM-2B-dpo-bf16 ` , ` openbmb/MiniCPM-S-1B-sft ` , etc. | ✅︎ | ✅︎ | ✅︎ |
362
365
| ` MiniCPM3ForCausalLM ` | MiniCPM3 | ` openbmb/MiniCPM3-4B ` , etc. | ✅︎ | ✅︎ | ✅︎ |
363
366
| ` MistralForCausalLM ` | Mistral, Mistral-Instruct | ` mistralai/Mistral-7B-v0.1 ` , ` mistralai/Mistral-7B-Instruct-v0.1 ` , etc. | ✅︎ | ✅︎ | ✅︎ |
364
367
| ` MixtralForCausalLM ` | Mixtral-8x7B, Mixtral-8x7B-Instruct | ` mistralai/Mixtral-8x7B-v0.1 ` , ` mistralai/Mixtral-8x7B-Instruct-v0.1 ` , ` mistral-community/Mixtral-8x22B-v0.1 ` , etc. | ✅︎ | ✅︎ | ✅︎ |
368
+ | ` QuantMixtralForCausalLM ` | Quantized Mixtral | ` mistral-community/Mixtral-8x22B-v0.1-AWQ ` , etc. | | ✅︎ | ✅︎ |
365
369
| ` MPTForCausalLM ` | MPT, MPT-Instruct, MPT-Chat, MPT-StoryWriter | ` mosaicml/mpt-7b ` , ` mosaicml/mpt-7b-storywriter ` , ` mosaicml/mpt-30b ` , etc. | | ✅︎ | ✅︎ |
366
370
| ` NemotronForCausalLM ` | Nemotron-3, Nemotron-4, Minitron | ` nvidia/Minitron-8B-Base ` , ` mgoin/Nemotron-4-340B-Base-hf-FP8 ` , etc. | ✅︎ | ✅︎ | ✅︎ |
367
371
| ` NemotronHForCausalLM ` | Nemotron-H | ` nvidia/Nemotron-H-8B-Base-8K ` , ` nvidia/Nemotron-H-47B-Base-8K ` , ` nvidia/Nemotron-H-56B-Base-8K ` , etc. | ✅︎ | ✅︎ | ✅︎ |
@@ -605,9 +609,9 @@ Specified using `--task generate`.
605
609
| ` TarsierForConditionalGeneration ` | Tarsier | T + I<sup >E+</sup > | ` omni-search/Tarsier-7b ` , ` omni-search/Tarsier-34b ` | | ✅︎ | ✅︎ |
606
610
| ` Tarsier2ForConditionalGeneration ` <sup >^</sup > | Tarsier2 | T + I<sup >E+</sup > + V<sup >E+</sup > | ` omni-research/Tarsier2-Recap-7b ` , ` omni-research/Tarsier2-7b-0115 ` | | ✅︎ | ✅︎ |
607
611
608
- <sup >^</sup > You need to set the architecture name via ` --hf-overrides ` to match the one in vLLM.
609
-   ;  ;  ;  ; • For example, to use DeepSeek-VL2 series models:
610
-   ;  ;  ;  ;  ;  ; ` --hf-overrides '{"architectures": ["DeepseekVLV2ForCausalLM"]}' `
612
+ <sup >^</sup > You need to set the architecture name via ` --hf-overrides ` to match the one in vLLM.
613
+   ;  ;  ;  ; • For example, to use DeepSeek-VL2 series models:
614
+   ;  ;  ;  ;  ;  ; ` --hf-overrides '{"architectures": ["DeepseekVLV2ForCausalLM"]}' `
611
615
<sup >E</sup > Pre-computed embeddings can be inputted for this modality.
612
616
<sup >+</sup > Multiple items can be inputted per text prompt for this modality.
613
617
0 commit comments