[Model] Re-add the implicit conversion feature for as_seq_cls_model

[noooop]

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan, such as providing test command.
• The test results, such as pasting the results comparison before and after, or e2e results
• (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.

Purpose

• Fix the issue reported in Support for LlamaForSequenceClassification #20807 where [Model][2/N] Automatic conversion of CrossEncoding model #19978 makes ForSequenceClassification using TRANSFORMERS Impl unusable.

ForSequenceClassification using TRANSFORMERS Impl It is implemented using TransformersForCausalLM + as_classification_model, instead of directly using TransformersForSequenceClassification

• We should implicit conversion ForSequenceClassification models, instead of adding them one by one to registry.py

• FIX the issue reported in [Feature]: Make SequenceClassification(Qwen3ForSequenceClassification) models support auto prefix cache #20894，

vllm/vllm/model_executor/models/adapters.py

Lines 175 to 180 in 3fc9644

	ModelForPooling = _create_pooling_model_cls(
	cls,
	default_pooling_type=PoolingType.LAST,
	default_normalize=False,
	default_softmax=True,
	)

This piece of code may not affect the main process…

refer to #20012

cc @DarkLight1337 @maxdebayser

Test Plan

pytest -s -vvv tests/models/test_transformers.py::test_classify
pytest -s -vvv tests/test_config.py::test_default_pooling_type

Test Result

(Optional) Documentation Update

passed

Fix #20894

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

🚀

[gemini-code-assist]

Summary of Changes

Hello @noooop, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly improves how vLLM handles Hugging Face ForSequenceClassification models. It introduces an automatic conversion mechanism that allows any ForCausalLM model to be used for sequence classification without requiring explicit registration or custom wrapper classes. This change streamlines model integration, centralizes classification task detection, and enhances the flexibility of vLLM's model loading and execution pipeline, while also addressing a reported issue with the TRANSFORMERS implementation for these models.

Highlights

• Automatic ForSequenceClassification Support: I've implemented a new mechanism to automatically convert ForCausalLM models into ForSequenceClassification models by dynamically applying an adapter. This removes the need for explicit registration and custom wrapper classes for each specific model type, streamlining model integration.
• Centralized Classification Task Logic: I've introduced a new _is_classify_task method and updated the task resolution in vllm/config.py. This provides a more robust way to identify and handle classification tasks, ensuring they are correctly routed to the 'pooling' runner.
• Refactored Model Registry: The model registry has been cleaned up by removing explicit entries for automatically convertible ForSequenceClassification models. The registry's ability to inspect and normalize these architectures on the fly has been enhanced, making it more flexible.
• Improved ScoreModel Bias Handling: The _ScoreModel adapter now correctly respects the score_bias configuration from the Hugging Face model config. This allows classification heads to have a bias if specified by the original model.
• Enhanced Testing Coverage: The test suite for model initialization (test_initialization.py) now includes automatically converted ForSequenceClassification models, ensuring the new automatic support mechanism functions as expected.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in issue comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist is currently in preview and may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments to provide feedback.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request refactors the model loading and registration logic to automatically support ForSequenceClassification models, which is a great improvement. The changes involve removing hardcoded model registrations and introducing dynamic conversion logic.

My review has identified a critical issue in the new conversion logic in vllm/model_executor/model_loader/utils.py that could lead to incorrect model loading. I've also pointed out a few medium-severity issues related to code clarity, maintainability, and a typo. Addressing these points will improve the robustness and readability of the new implementation.

[vrdn-23]

This looks awesome @noooop!
Question: Would this extend to only currently supported models or would it negate the need to have separate PRs for non-LLM models (like this for example #20215)?

[noooop]

This looks awesome @noooop! Question: Would this extend to only currently supported models or would it negate the need to have separate PRs for non-LLM models (like this for example #20215)?

DebertaV2ForSequenceClassification uses classifier, while this pr uses score, so it is not supported. Is the title not well chosen? In fact, this pr only implements a small amount of functionality.

[maxdebayser]

Nice, this is going in the right direction

[maxdebayser]

There is something strange here, vllm_supported has the exact same definition as vllm_not_supported.

Suggested change

	vllm_supported = not any(arch in vllm_supported_archs
	for arch in architectures)
	if vllm_supported:
	if vllm_not_supported:

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @noooop.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork