[CI] Tests for graph comparison between vllm and AFTU

[wallashss]

Description

This PR adds tests that compare graphs generated by vLLM vs AFTU (aiu-fms-testing-utils). The tests do an inference with vLLM and execute inference.py from AFTU. For both execution, the tests dump the compiled graphs to files by setting the env variable DEE_DUMP_GRAPHS=name. Then, they compare the corresponding graphs generated for both method.

The main limitation of this implementation is AFTU is not a proper package. So, I had to make some workarounds:

• I added an environment variable: VLLM_SPYRE_TEST_AFTU_SCRIPTS_DIR. If set, we can use to inform where is the scripts dirs (where it has the inference.py) to copy from there and execute the script within the tests.
• If the above env var is not set, then we try to get from the install by importing the module and using the aiu_fms_testing_utils.__file__ variable to find the directory.

I added a marker aftu to ease select the tests of this PR, and also to filter out from other test suite.

[github-actions]

👋 Hi! Thank you for contributing to vLLM support on Spyre.
Just a reminder: Make sure that your code passes all the linting checks, otherwise your PR won't be able to be merged. To do so, first install the linting requirements, then run format.sh and commit the changes. This can be done with uv directly:

uv sync --frozen --group lint --active --inexact

Or this can be done with pip:

uv pip compile --group lint > requirements-lint.txt
pip install -r requirements-lint.txt
bash format.sh

Now you are good to go 🚀

[prashantgupta24]

wondering if we need to update the uv.lock file as well 🤔

[wallashss]

updated

[rafvasq]

nit: something like

Suggested change

	# Get G1 graphs, it assumes the input_dir has the folder export_dtcompiler
	# where are the files
	# Get G1 graphs.
	# Assumes the 'input_dir' contains 'export_dtcompiler' with the files.

[rafvasq]

just a nit: comment could be a bit more succinct, something like:

Suggested change

	# Silly regex to find all s#.
	# We are only considered those that surrounds by space (whole word)
	# or started with space and terminated with comma
	# examples:
	# ' s1 '
	# ' s1,'
	# ' s1 s2 '
	# Regex to find all 's#' patterns surrounded by spaces,
	# or starting with a space and ending with a comma.
	# Examples: ' s1 ', ' s1,', ' s1 s2 '

[joerunde]

If this test can only run on the sendnn backend then we don't need to parameterize it, we can just set the backend as sendnn in the test.

Also, we should mark all tests running with the sendnn backend with pytest.mark.spyre for consistency with the rest of the test suite

[joerunde]

What does AFTU is not a lib yet mean here? Is the problem that inference.py is not shipped with the AFTU wheel?

[joerunde]

I'd like to avoid creating more custom markers unless it's completely necessary. (Unrelated but it looks like utils and worker are unused and we should delete them as well)

These tests seem to be important to catch problems early so I do want them running with our default set of markers if possible