Skip to content

thread-unsafe test failures are hard to spot #70

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
crusaderky opened this issue Jun 10, 2025 · 1 comment · Fixed by #71
Closed

thread-unsafe test failures are hard to spot #70

crusaderky opened this issue Jun 10, 2025 · 1 comment · Fixed by #71

Comments

@crusaderky
Copy link

crusaderky commented Jun 10, 2025

If a test tagged with pytest.mark.thread_unsafe fails, there is a FAILED tag on the line of the test. There is no traceback. At the end of the test suite, the summary reports that all tests passed. Finally, pytest exits 1.
When the test suite is very large, one has to comb through the pytest stdout to find which test failed.

@pytest.mark.thread_unsafe
def test1():
    assert False
$ pytest -v --parallel-threads=4 test1.py:
=============================================================== test session starts ===============================================================
test1.py::test1 FAILED ([thread-unsafe]: uses thread_unsafe marker)                                                               [100%]

*********************************************************** pytest-run-parallel report ************************************************************
1 tests were not run in parallel because of use of thread-unsafe functionality, to list the tests that were skipped, re-run while setting PYTEST_RUN_PARALLEL_VERBOSE=1 in your shell environment
======================================================== 1 passed, 258 deselected in 0.17s ========================================================
$ echo $?
1

Compare to plain pytest:

$ pytest -v test1.py
tests/test_lazy.py::test1 FAILED                                                                                                            [100%]

==================================================================== FAILURES =====================================================================
______________________________________________________________________ test1 ______________________________________________________________________

    @pytest.mark.thread_unsafe
    def test1():
>       assert False
E       assert False


test1.py:3: AssertionError
============================================================= short test summary info =============================================================
FAILED tests/test_lazy.py::test1 - assert False
======================================================== 1 failed, 258 deselected in 0.15s ========================================================

Or pytest-run-parallel on an unmarked test:

$ pytest -v --parallel-threads=4 test1.py:
test1.py::test1 PARALLEL FAILED                                                                                                   [100%]

===================================================================== ERRORS ======================================================================
_____________________________________________________________ ERROR at call of test1 ______________________________________________________________

    def test1():
>       assert False
E       assert False


test1.py:2: AssertionError
*********************************************************** pytest-run-parallel report ************************************************************
All tests were run in parallel! 🎉
============================================================= short test summary info =============================================================
PARALLEL FAILED test1.py::test1 - assert False
@lysnikolaou
Copy link
Member

Opened #71 to fix this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants