Requesting code review of hook for failing slow tests

[jwodder]

I've written the following pytest hook (currently just for inclusion in a project's conftest.py) for marking an otherwise-passing test as failed if it takes too long to run. It seems to work correctly, but I feel the need for input from pytest experts before I go using it in the wild. Are there any cases that this fails to handle correctly or anything I should know about doing this sort of thing? (Also, minor side question: is there a way to make a custom message for failed slow tests appear in the "short test summary info" at the end of the pytest output? Currently, the tests failed by this hook just show up as "FAILED " there, which isn't very helpful.)

import pytest

def pytest_addoption(parser) -> None:
    parser.addoption("--test-timeout", type=float)

@pytest.hookimpl(hookwrapper=True)
def pytest_runtest_makereport(item, call):
    report = (yield).get_result()
    timeout = item.config.getoption("--test-timeout")
    if (
        report is not None
        and timeout is not None
        and report.when == "call"
        and report.outcome == "passed"
        and call.duration > timeout
    ):
        report.outcome = "failed"
        report.longrepr = f"Test took too long: Duration {call.duration}s > {timeout}s"
    return report

[jwodder]

Well, since no one had any comments, I've gone ahead and released this as a plugin on PyPI: https://github.com/jwodder/pytest-fail-slow. If there is anything wrong with the code above, post an issue on the repository.

[nicoddemus]

That's great @jwodder, thanks for sharing!

[The-Compiler]

Let me note that there's pytest-timeout too, which can also take care of interrupting the test.

[jwodder]

I'm aware of that, but killing slow tests is not what we want to do.