Hierarchical test parametrization?

[HiPhish]

Hello,

I have a Python script (HiPhish /
anno-1503-resolution-patcher) which patches a DLL file from a game (Anno 1503) to run at higher resolutions. There is some guesswork involved, so the patcher produces multiple files per resolution. I am using Pytest to run an end-to-end test which calls the patcher and then compares the generated files to manually created files for equality.

Here is what I am trying to achive: I want to run the patcher once per resolution, but run the test once per generated file. My current test looks like this:

@fixture(autouse=True)
def invocation(width: int, height: int, game_path: Path) -> None:
    args = [...]  # Omitted for brevity
    run(args)

EXPECTED_FNAMES = [f'AnnoFrame_patched_{i}.dll' for i in range(1, 4)]

@mark.parametrize('fname', EXPECTED_FNAMES)
def test_execution(game_path: Path, patch_path: Path, fname: str):
    assert filecmp.cmp(game_path / fname, patch_path / fname, shallow=False)

The fixture invocation is parameterized through width and height by the resolution and is run before each test. The test itself is parameterized on the resolution (throughpatch_path) and the name of the file to check (through fname). So in total the test is run for each resolution and each file. This is fine. However, I do not want to run invocation for each combination for resolution and file name, I only want to run it once per resolution.

The only solution I have found is to set the scope of all fixtures to session. This will cause Pytest to run fixtures only when it needs to. Is this the correct way? I seems very hacky, like I'm abusing the purpose of the scope.

[RonnyPfannschmidt]

i believe sucha hierarchy can be mirrored by fixture parameters and dependences

@pytest.fixture(params=resolutions)
def invocation():
 ...

@fixture(params=FNAMES)
def invocation_filename(invocation):
   ...
   ```

[HiPhish]

That won't work. Here is a little test script:

from pytest import fixture


count = 0


@fixture(params=['r1', 'r2', 'r3'])
def invocation(request):
    global count
    count += 1


@fixture(params=['a', 'b', 'c', 'd'])
def fname(invocation, request):
    return request.param


def test_call(fname):
    assert True
    print(count)

The invocation still gets called twelve times.

[twmr]

If you want that invocation fixture is only called 3 times then you can pass a scope (@fixture(params=['r1', 'r2', 'r3'], scope="module")) to it, see https://docs.pytest.org/en/latest/how-to/fixtures.html#parametrizing-fixtures

[HiPhish]

Yes, I already figured that out, but then I need to change all fixtures as well. The real dependency graph of fixtures is as follows:

• game path (creates a temporary directory and returns the path)
• patch path (parameterized on a listing of directories, returns the path)
  • resolution (depends on the name of the patch path, returns the x/y values as a tuple)
    • width (first value of the resolution)
    • height (second value of the resolution)
      • invocation (invokes the script as a separate process, depends on width, height and game path)

Finally the test depends on the invocation and the name of a file to compare. Thus increasing the scope of the invocation fixture means I have to increase the scope of all other fixtures as well. For a simple toy project like this it doesn't matter because there is only one test, but I want to know if there is a better way for when I find myself with the same problem in a real project.