[RFC][Feature][Model] Add templated fallback HF `transformers` model backend in SRT

[XuehaiPan]

Checklist

• 1. If the issue you raised is not a feature but a question, please raise a discussion at https://github.com/sgl-project/sglang/discussions/new/choose Otherwise, it will be closed.
• 2. Please use English, otherwise it will be closed.

Feature Request

This issue requests a feature that SRT can out-of-the-box support new models without extra maintenance effort when a new open-source model is released. It would help to improve the model coverage on the Hugging Face repo.

This will be similar to what we (Hugging Face Transformers maintainers) do for vLLM.

• [Model]: Add transformers backend support vllm-project/vllm#11330
• Add pipeline parallel support to TransformersModel vllm-project/vllm#12832
• [Model] Add LoRA support for TransformersModel vllm-project/vllm#13770
• [Model] Support VLMs with transformers backend vllm-project/vllm#13754

Road Map

• Initial TransformersModel backend for LM based on Transformers - Attention Interface.
• Initial support for popular VLMs.
• SRT DP support for TransformersModel.
• SRT PP support for TransformersModel.

Feel free to comment and update the RoadMap.

cc Transformers maintainers @LysandreJik @ArthurZucker
cc SGLang Team for co- code review @merrymercy @Ying1123 @zhyncs

Related resources

No response

[merrymercy]

This sounds like a very good feature. Will you or anyone work on this?

[XuehaiPan]

Will you or anyone work on this?

Please assign the issue to me if you are fine with this feature. I will send the PRs soon.

[zhaochenyang20]

great work