Pytest crash when exception is raised in fixture teardown and --exitfirst flag is set

[oleg-kondaurov]

UPD. The issue is reproduced only when --exitfirst flag is set, and some test should be skipped after unhandled error in fixture teardown.

It is an additional question for closed issue How to reuse same fixture implementation with different scopes? #3425
#3425 (comment)

Is it possible to somehow trigger the test state to become error or fail in case if an exception is raised in setup or teardown block of the contextmanager?

For example, current code sample causes the unhandled exception and pytest process close:

@pytest.fixture(scope='module')
def my_fixture_mod():
    with fixture_impl(...) as result:
       yield result

@contextmanager
def fixture_impl():
    print('raise exception in BEFORE...')
    print(1/0)
    yield
    print('raise exception in AFTER...')
    print(1/0)

[nicoddemus]

Currently the only way to change the outcome is by overriding the pytest_runtest_makereport hook, there's an example here:

https://docs.pytest.org/en/stable/example/simple.html?highlight=pytest_runtest_makereport#post-process-test-reports-failures

[oleg-kondaurov]

Using another issue I discovered, that the trigger of this issue is --exitfirst flag.
The following test suite returns 2 passed, 2 errors if the flag is not set, but pytest crash if the flag is set

import pytest


class TestT:
    @pytest.fixture(scope='class')
    def fix1(self):
        print('setup 1')
        yield
        print('teardown 1')
        raise Exception('test 1 teardown exception')

    @pytest.fixture(scope='function')
    def fix2(self):
        print('setup 2')
        yield
        print('teardown 2')
        raise Exception('test 2 teardown exception')

    def test1(self, fix1, fix2):
        print('test 1')

    def test2(self):
        print('test 2')

No --etixfirst set:

 $ py.test tests/test2.py  -s 
======================================================================================================================================================================== test session starts ========================================================================================================================================================================
platform linux -- Python 3.8.10, pytest-6.2.5, py-1.10.0, pluggy-0.13.1
rootdir: /home/okondaurov/GitHub/kyiv-qa-tests, configfile: pytest.ini
plugins: metadata-1.11.0, cov-3.0.0, mock-3.6.1, html-3.1.1, dependency-0.5.1
collected 2 items                                                                                                                                                                                                                                                                                                                                                   

tests/test2.py setup 1
setup 2
test 1
.teardown 2
Etest 2
.teardown 1
E

============================================================================================================================================================================== ERRORS ===============================================================================================================================================================================
_________________________________________________________________________________________________________________________________________________________________ ERROR at teardown of TestT.test1 __________________________________________________________________________________________________________________________________________________________________

self = <tests.test2.TestT object at 0x7f060d412460>

    @pytest.fixture(scope='function')
    def fix2(self):
        print('setup 2')
        yield
        print('teardown 2')
>       raise Exception('test 2 teardown exception')
E       Exception: test 2 teardown exception

tests/test2.py:17: Exception
_________________________________________________________________________________________________________________________________________________________________ ERROR at teardown of TestT.test2 __________________________________________________________________________________________________________________________________________________________________

self = <tests.test2.TestT object at 0x7f060d3f9fd0>

    @pytest.fixture(scope='class')
    def fix1(self):
        print('setup 1')
        yield
        print('teardown 1')
>       raise Exception('test 1 teardown exception')
E       Exception: test 1 teardown exception

tests/test2.py:10: Exception
====================================================================================================================================================================== short test summary info ======================================================================================================================================================================
ERROR tests/test2.py::TestT::test1 - Exception: test 2 teardown exception
ERROR tests/test2.py::TestT::test2 - Exception: test 1 teardown exception
==================================================================================================================================================================== 2 passed, 2 errors in 0.06s ====================================================================================================================================================================

--etixfirst is set:

$ py.test tests/test2.py  -s --exitfirst
======================================================================================================================================================================== test session starts ========================================================================================================================================================================
platform linux -- Python 3.8.10, pytest-6.2.5, py-1.10.0, pluggy-0.13.1
rootdir: /home/okondaurov/GitHub/kyiv-qa-tests, configfile: pytest.ini
plugins: metadata-1.11.0, cov-3.0.0, mock-3.6.1, html-3.1.1, dependency-0.5.1
collected 2 items                                                                                                                                                                                                                                                                                                                                                   

tests/test2.py setup 1
setup 2
test 1
.teardown 2
Eteardown 1
Traceback (most recent call last):
  File "/home/okondaurov/GitHub/kyiv-qa-tests/venv/bin/py.test", line 8, in <module>
    sys.exit(console_main())
  File "/home/okondaurov/GitHub/kyiv-qa-tests/venv/lib/python3.8/site-packages/_pytest/config/__init__.py", line 185, in console_main
    code = main()
  File "/home/okondaurov/GitHub/kyiv-qa-tests/venv/lib/python3.8/site-packages/_pytest/config/__init__.py", line 162, in main
    ret: Union[ExitCode, int] = config.hook.pytest_cmdline_main(
  File "/home/okondaurov/GitHub/kyiv-qa-tests/venv/lib/python3.8/site-packages/pluggy/hooks.py", line 286, in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
  File "/home/okondaurov/GitHub/kyiv-qa-tests/venv/lib/python3.8/site-packages/pluggy/manager.py", line 93, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "/home/okondaurov/GitHub/kyiv-qa-tests/venv/lib/python3.8/site-packages/pluggy/manager.py", line 84, in <lambda>
    self._inner_hookexec = lambda hook, methods, kwargs: hook.multicall(
  File "/home/okondaurov/GitHub/kyiv-qa-tests/venv/lib/python3.8/site-packages/pluggy/callers.py", line 208, in _multicall
    return outcome.get_result()
  File "/home/okondaurov/GitHub/kyiv-qa-tests/venv/lib/python3.8/site-packages/pluggy/callers.py", line 80, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/home/okondaurov/GitHub/kyiv-qa-tests/venv/lib/python3.8/site-packages/pluggy/callers.py", line 187, in _multicall
    res = hook_impl.function(*args)
  File "/home/okondaurov/GitHub/kyiv-qa-tests/venv/lib/python3.8/site-packages/_pytest/main.py", line 316, in pytest_cmdline_main
    return wrap_session(config, _main)
  File "/home/okondaurov/GitHub/kyiv-qa-tests/venv/lib/python3.8/site-packages/_pytest/main.py", line 304, in wrap_session
    config.hook.pytest_sessionfinish(
  File "/home/okondaurov/GitHub/kyiv-qa-tests/venv/lib/python3.8/site-packages/pluggy/hooks.py", line 286, in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
  File "/home/okondaurov/GitHub/kyiv-qa-tests/venv/lib/python3.8/site-packages/pluggy/manager.py", line 93, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "/home/okondaurov/GitHub/kyiv-qa-tests/venv/lib/python3.8/site-packages/pluggy/manager.py", line 84, in <lambda>
    self._inner_hookexec = lambda hook, methods, kwargs: hook.multicall(
  File "/home/okondaurov/GitHub/kyiv-qa-tests/venv/lib/python3.8/site-packages/pluggy/callers.py", line 203, in _multicall
    gen.send(outcome)
  File "/home/okondaurov/GitHub/kyiv-qa-tests/venv/lib/python3.8/site-packages/_pytest/terminal.py", line 803, in pytest_sessionfinish
    outcome.get_result()
  File "/home/okondaurov/GitHub/kyiv-qa-tests/venv/lib/python3.8/site-packages/pluggy/callers.py", line 80, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/home/okondaurov/GitHub/kyiv-qa-tests/venv/lib/python3.8/site-packages/pluggy/callers.py", line 187, in _multicall
    res = hook_impl.function(*args)
  File "/home/okondaurov/GitHub/kyiv-qa-tests/venv/lib/python3.8/site-packages/_pytest/runner.py", line 103, in pytest_sessionfinish
    session._setupstate.teardown_all()
  File "/home/okondaurov/GitHub/kyiv-qa-tests/venv/lib/python3.8/site-packages/_pytest/runner.py", line 412, in teardown_all
    self._pop_and_teardown()
  File "/home/okondaurov/GitHub/kyiv-qa-tests/venv/lib/python3.8/site-packages/_pytest/runner.py", line 387, in _pop_and_teardown
    self._teardown_with_finalization(colitem)
  File "/home/okondaurov/GitHub/kyiv-qa-tests/venv/lib/python3.8/site-packages/_pytest/runner.py", line 405, in _teardown_with_finalization
    self._callfinalizers(colitem)
  File "/home/okondaurov/GitHub/kyiv-qa-tests/venv/lib/python3.8/site-packages/_pytest/runner.py", line 402, in _callfinalizers
    raise exc
  File "/home/okondaurov/GitHub/kyiv-qa-tests/venv/lib/python3.8/site-packages/_pytest/runner.py", line 395, in _callfinalizers
    fin()
  File "/home/okondaurov/GitHub/kyiv-qa-tests/venv/lib/python3.8/site-packages/_pytest/fixtures.py", line 1034, in finish
    raise exc
  File "/home/okondaurov/GitHub/kyiv-qa-tests/venv/lib/python3.8/site-packages/_pytest/fixtures.py", line 1027, in finish
    func()
  File "/home/okondaurov/GitHub/kyiv-qa-tests/venv/lib/python3.8/site-packages/_pytest/fixtures.py", line 941, in _teardown_yield_fixture
    next(it)
  File "/home/okondaurov/GitHub/kyiv-qa-tests/tests/test2.py", line 10, in fix1
    raise Exception('test 1 teardown exception')
Exception: test 1 teardown exception