You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
|`Llama4ForConditionalGeneration`| Llama 4 | T + I<sup>+</sup> |`meta-llama/Llama-4-Scout-17B-16E-Instruct`, `meta-llama/Llama-4-Maverick-17B-128E-Instruct-FP8`, `meta-llama/Llama-4-Maverick-17B-128E-Instruct`, etc. || ✅︎ | ✅︎ |
584
-
|`LlavaForConditionalGeneration`| LLaVA-1.5| T + I<sup>E+</sup> |`llava-hf/llava-1.5-7b-hf`, `TIGER-Lab/Mantis-8B-siglip-llama3` (see note), etc. || ✅︎ | ✅︎ |
584
+
|`LlavaForConditionalGeneration`| LLaVA-1.5, Pixtral (HF Transformers) | T + I<sup>E+</sup> |`llava-hf/llava-1.5-7b-hf`, `TIGER-Lab/Mantis-8B-siglip-llama3` (see note), `mistral-community/pixtral-12b`, etc. || ✅︎ | ✅︎ |
585
585
|`LlavaNextForConditionalGeneration`| LLaVA-NeXT | T + I<sup>E+</sup> |`llava-hf/llava-v1.6-mistral-7b-hf`, `llava-hf/llava-v1.6-vicuna-7b-hf`, etc. || ✅︎ | ✅︎ |
586
586
|`LlavaNextVideoForConditionalGeneration`| LLaVA-NeXT-Video | T + V |`llava-hf/LLaVA-NeXT-Video-7B-hf`, etc. || ✅︎ | ✅︎ |
587
587
|`LlavaOnevisionForConditionalGeneration`| LLaVA-Onevision | T + I<sup>+</sup> + V<sup>+</sup> |`llava-hf/llava-onevision-qwen2-7b-ov-hf`, `llava-hf/llava-onevision-qwen2-0.5b-ov-hf`, etc. || ✅︎ | ✅︎ |
588
588
|`MiniCPMO`| MiniCPM-O | T + I<sup>E+</sup> + V<sup>E+</sup> + A<sup>E+</sup> |`openbmb/MiniCPM-o-2_6`, etc. | ✅︎ | ✅︎ | ✅︎ |
589
589
|`MiniCPMV`| MiniCPM-V | T + I<sup>E+</sup> + V<sup>E+</sup> |`openbmb/MiniCPM-V-2` (see note), `openbmb/MiniCPM-Llama3-V-2_5`, `openbmb/MiniCPM-V-2_6`, etc. | ✅︎ || ✅︎ |
590
590
|`MiniMaxVL01ForConditionalGeneration`| MiniMax-VL | T + I<sup>E+</sup> |`MiniMaxAI/MiniMax-VL-01`, etc. || ✅︎ | ✅︎ |
591
-
|`Mistral3ForConditionalGeneration`| Mistral3 | T + I<sup>+</sup> |`mistralai/Mistral-Small-3.1-24B-Instruct-2503`, etc. | ✅︎ | ✅︎ | ✅︎ |
591
+
|`Mistral3ForConditionalGeneration`| Mistral3 (HF Transformers) | T + I<sup>+</sup> |`mistralai/Mistral-Small-3.1-24B-Instruct-2503`, etc. | ✅︎ | ✅︎ | ✅︎ |
592
592
|`MllamaForConditionalGeneration`| Llama 3.2 | T + I<sup>+</sup> |`meta-llama/Llama-3.2-90B-Vision-Instruct`, `meta-llama/Llama-3.2-11B-Vision`, etc. ||||
593
593
|`MolmoForCausalLM`| Molmo | T + I<sup>+</sup> |`allenai/Molmo-7B-D-0924`, `allenai/Molmo-7B-O-0924`, etc. | ✅︎ | ✅︎ | ✅︎ |
594
594
|`NVLM_D_Model`| NVLM-D 1.0 | T + I<sup>+</sup> |`nvidia/NVLM-D-72B`, etc. || ✅︎ | ✅︎ |
595
595
|`Ovis`| Ovis2, Ovis1.6 | T + I<sup>+</sup> |`AIDC-AI/Ovis2-1B`, `AIDC-AI/Ovis1.6-Llama3.2-3B`, etc. || ✅︎ | ✅︎ |
596
596
|`PaliGemmaForConditionalGeneration`| PaliGemma, PaliGemma 2 | T + I<sup>E</sup> |`google/paligemma-3b-pt-224`, `google/paligemma-3b-mix-224`, `google/paligemma2-3b-ft-docci-448`, etc. || ✅︎ | ⚠️ |
597
597
|`Phi3VForCausalLM`| Phi-3-Vision, Phi-3.5-Vision | T + I<sup>E+</sup> |`microsoft/Phi-3-vision-128k-instruct`, `microsoft/Phi-3.5-vision-instruct`, etc. || ✅︎ | ✅︎ |
598
598
|`Phi4MMForCausalLM`| Phi-4-multimodal | T + I<sup>+</sup> / T + A<sup>+</sup> / I<sup>+</sup> + A<sup>+</sup> |`microsoft/Phi-4-multimodal-instruct`, etc. | ✅︎ | ✅︎ | ✅︎ |
599
-
|`PixtralForConditionalGeneration`| Pixtral | T + I<sup>+</sup> |`mistralai/Mistral-Small-3.1-24B-Instruct-2503`, `mistral-community/pixtral-12b`, etc. || ✅︎ | ✅︎ |
599
+
|`PixtralForConditionalGeneration`|Mistral 3 (Mistral format), Pixtral (Mistral format) | T + I<sup>+</sup> |`mistralai/Mistral-Small-3.1-24B-Instruct-2503`, `mistralai/Pixtral-12B-2409`, etc. || ✅︎ | ✅︎ |
600
600
|`QwenVLForConditionalGeneration`<sup>^</sup> | Qwen-VL | T + I<sup>E+</sup> |`Qwen/Qwen-VL`, `Qwen/Qwen-VL-Chat`, etc. | ✅︎ | ✅︎ | ✅︎ |
0 commit comments