Skip to content

[CI/Build] Add Transformers nightly tests in CI #20924

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Jul 14, 2025
Merged

[CI/Build] Add Transformers nightly tests in CI #20924

Changes from 2 commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
332c94b
add transformers nightly test
Isotr0py Jul 13, 2025
01076f8
Merge branch 'vllm-project:main' into transformers-nightly-ci
Isotr0py Jul 14, 2025
11e12ef
include test_initialization.py
Isotr0py Jul 14, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions .buildkite/test-pipeline.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -630,6 +630,17 @@ steps:
# e.g. pytest -v -s models/encoder_decoder/vision_language/test_mllama.py
# *To avoid merge conflicts, remember to REMOVE (not just comment out) them before merging the PR*

- label: Transformers Nightly Models Test
working_dir: "/vllm-workspace/"
optional: true
commands:
- pip install --upgrade git+https://github.com/huggingface/transformers
- pytest -s -v tests/models/multimodal/processing/
- pytest -s -v tests/models/multimodal/test_mapping.py
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The -s flag in pytest disables output capturing, which can lead to excessively verbose CI logs. Consider removing it to keep logs clean and focused on failures.

- pytest -v tests/models/multimodal/processing/
- pytest -v tests/models/multimodal/test_mapping.py

- python3 examples/offline_inference/basic/chat.py
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Specify the model explicitly in the chat.py command to avoid unexpected behavior due to default value changes in the script.

- python3 examples/offline_inference/basic/chat.py --model meta-llama/Llama-3.2-1B-Instruct

- python3 examples/offline_inference/audio_language.py --model-type whisper
- python3 examples/offline_inference/vision_language.py --model-type qwen2_5_vl

##### 1 GPU test #####
##### multi gpus test #####

Expand Down