Context managers instead of fixtures?

[fredrikaverpil]

I've worked on a very large test suite which was made up of likely hundreds of fixtures. This got cumbersome to work with over time because we couldn't do "go to def" on the fixtures in their test function signatures. I'm now looking at setting up a brand new pytest test suite from scratch, and in my team there are thoughts on writing context managers instead of fixtures.

The reason is my team is using a mix of neovim and vscode and cannot do a "go to def" on fixtures in test function signatures. I believe it's really only pycharm that has this ability (built-in).

Since the intent by pytest is you write fixtures, I'm very hesitant here to break away from that pattern. There might come a day when e.g. pylance/vscode adds support for this "go to def", and then I'll likely regret the choice of going with context managers.

But the greatest concern would be if pytest is doing something clever with fixtures, which won't be possible with context managers, such as managing how they are setup/torn down (loaded/unloaded in memory)...?
So, what I'm wondering here is ...

• What kind of arcane magic might I be forfeiting by opting for context managers in favour for fixtures?
• What's the general consensus on breaking free from fixtures and instead write your own context managers?

[fredrikaverpil]

@nicoddemus - what pitfalls might we be looking at if we decide to write our own context managers instead of writing fixtures?

[nicoddemus]

@RonnyPfannschmidt and @The-Compiler already beat me to it with great replies. 👍

In the end you can list the features fixtures bring to the table (parametrization, scopes, easy and seamless reuse of other fixtures, indentation, etc) and compare against the IDE support (also accounting for the possibility of it happening in the future).

I'm a PyCharm user and can attest how awesome that feature is too. 👍

[RonnyPfannschmidt]

key items is that you forfeit parameterization integration, scope based test redordering ( to reduce setup/teardown of higher scopes)

its generally suggested to use fixtures in pytest, and its indeed unfortunate that IDE integration for the lookup is rather limited

[The-Compiler]

The most direct benefit of fixtures is probably the easier syntax. Context managers would often lead to indenting all your tests by one level, was well as it being rather awkward to combine multiple context managers. @RonnyPfannschmidt mentioned some more, I can think of some additional ones:

• Easy usage of fixtures from fixtures (rather than having to pass fixtures into the context manager for every test, if you need to access some existing functionality exposed via fixtures)
• Caching, i.e. using the same fixture value for different tests where needed
• Probably more I'm not considering right now.

As for the general consensus: I can't speak for everyone obviously, but it seems like a strange thing to do, as you're essentially ignoring what's probably pytest's most central feature/pattern. Note that it's easy to get that functionality in VS Code as well, via a pytest-fixtures plugin. Something similar seems to exist for NeoVim too. FWIW I don't think "built-in" is a relevant criteria really, as with both VS Code and NeoVim you're likely using a (if not many) extensions to use them as Python IDE already.

[fredrikaverpil]

Thanks @RonnyPfannschmidt @The-Compiler for your input 👍

Some reflections:

• I definitively agree on forfeiting what you have mentioned is not a good trade-off.
• I've tried all existing plugins that you find my searching "pytest" in vscode, and they don't work for me (and I'm not alone). Have you gotten any of these to work?
• I am wary to say that forfeiting "go to def" and type hinting is actually something we'd be okay with. This severely hampers working efficiently and confidently with writing and maintaining tests.

The overall sum of this all is that I think pytest's strengths still outweigh the weaknesses/shortcomings of IDE/editor (plugin) integrations.

[nicoddemus]

Good thought.

This caught my eye though:

and type hinting

What do you mean? You can (and should) still type-annotate the parameters in test functions, in the case of fixtures, you should annotate with the fixture's return value.

[fredrikaverpil]

@nicoddemus hi, nice to see you chime in on this 🙇

What do you mean? You can (and should) still type-annotate the parameters in test functions, in the case of fixtures, you should annotate with the fixture's return value.

Ah. Okay. So you mean I should be able to do something like this?

# conftest.py

import pytest


class Something:
    def sum(self, x: int, y: int) -> int:
        return x + y


@pytest.fixture()
def somefixture() -> Generator[Something, None, None]:
    # setup()
    yield Something()
    # teardown()

# test_something.py

from typing import TYPE_CHECKING

if TYPE_CHECKING:
    from tests.conftest import Something


def test_sum(somefixture: Something):
    result = something.sum(1, 2)
    assert result == 3

[nicoddemus]

Hi! 👋

Exactly!

From the type checker's perspective, the somefixture parameter is a parameter as any other, and it should be type checked with what's expected to be passed to it. 👍

[The-Compiler]

FWIW, one thing that helped me is simply imagining pytest calling your test with the fixture return values as keyword arguments: test_sum(somefixture=somefixture()).