[CB] Add scheduling tests

[sducouedic]

This PR adds a scheduling steps test where new prompts are joining during the decode of other sequences, when there is still room left in the batch for new sequences.

Execution was tested on AIU as well (passing)

[github-actions]

👋 Hi! Thank you for contributing to vLLM support on Spyre.
Just a reminder: Make sure that your code passes all the linting checks, otherwise your PR won't be able to be merged. To do so, first install the linting requirements, then run format.sh and commit the changes. This can be done with uv directly:

uv sync --frozen --group lint --active --inexact

Or this can be done with pip:

uv pip compile --group lint > requirements-lint.txt
pip install -r requirements-lint.txt
bash format.sh

Now you are good to go 🚀

[sducouedic]

bot:test
MARKERS="spyre and not quantized and not multi and not embedding"

[prashantgupta24]

Excited for this!

[sducouedic]

Let me break this PR in to two PRs

1. This PR: add a simple additional test for steps
2. Another PR: check the end output of scheduling steps tests

[maxdebayser]

Thanks, this is very easy to follow with pencil and paper. It's almost like a documentation.

[maxdebayser]

Suggestion: maybe start counting at 0 here to use the sequence IDs

[prashantgupta24]

I wonder if we can lower the values here - the time for CB testing on CPU is on the rise, maybe (if possible) having shorter max_tokens can speed tests up if the test logic remains the same

[prashantgupta24]

I think the eventual goal could be to reduce the total number of steps - the lesser the steps, the faster the test. I don't think we really need 197 steps for this test case?

[prashantgupta24]

Something like

    seqs_max_tokens = [3, 10, 5]
    prompts_lengths = [10, 10, 10]
    steps_add_reqs = [0, 0, 5]

where request 0 would finish first, request 1 would be still decoding when request 2 shows up? Or am I missing something obvious?

[prashantgupta24]

If this can be made to work with lesser max_tokens, then perhaps we can open an issue to change all tests within this file to use lesser values to speed things up?