-
Notifications
You must be signed in to change notification settings - Fork 228
Open
Description
This isn't truly an issue with pytest-bdd, but I thought this was worth mentioning given the popularity of the pytest-timeout plugin.
I noticed that if the pytest-timeout plugin is used, it can correctly fail a scenario, but all steps show as passed in the cucumberjson output. A minimal example:
test_timeout.py:
from pytest_bdd import scenario, when
import time
@scenario('timeout.feature', 'Long running operation')
def test_long_running():
pass
@when('I wait for a really long time')
def wait_long_time():
time.sleep(3)
timeout.feature:
Feature: Timeout demonstration
Demonstration of a long-running operation
Scenario: Long running operation
When I wait for a really long time
then run pytest:
pytest -s tests/test_timeout.py --cucumberjson=output.json --timeout 1
to get the following logs:
=================================================================================================================================================== test session starts ====================================================================================================================================================
platform linux -- Python 3.12.7, pytest-8.3.5, pluggy-1.5.0
rootdir: /media/jason/e/pytest-bdd
configfile: pytest.ini
plugins: bdd-8.1.0, timeout-2.3.1
timeout: 1.0s
timeout method: signal
timeout func_only: False
collected 1 item
tests/test_timeout.py OUTCOME: failed
F
========================================================================================================================================================= FAILURES =========================================================================================================================================================
____________________________________________________________________________________________________________________________________________________ test_long_running _____________________________________________________________________________________________________________________________________________________
fixturefunc = <function wait_long_time at 0x7130aa194360>, request = <FixtureRequest for <Function test_long_running>>, kwargs = {}
def call_fixture_func(
fixturefunc: _FixtureFunc[FixtureValue], request: FixtureRequest, kwargs
) -> FixtureValue:
if is_generator(fixturefunc):
fixturefunc = cast(
Callable[..., Generator[FixtureValue, None, None]], fixturefunc
)
generator = fixturefunc(**kwargs)
try:
fixture_result = next(generator)
except StopIteration:
raise ValueError(f"{request.fixturename} did not yield a value") from None
finalizer = functools.partial(_teardown_yield_fixture, fixturefunc, generator)
request.addfinalizer(finalizer)
else:
fixturefunc = cast(Callable[..., FixtureValue], fixturefunc)
> fixture_result = fixturefunc(**kwargs)
venv/lib/python3.12/site-packages/_pytest/fixtures.py:898:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
@when('I wait for a really long time')
def wait_long_time():
> time.sleep(3)
E Failed: Timeout >1.0s
tests/test_timeout.py:10: Failed
------------------------------------------------------------------------------------------------------------------------- generated json file: /media/jason/e/pytest-bdd/output.json -------------------------------------------------------------------------------------------------------------------------
================================================================================================================================================= short test summary info ==================================================================================================================================================
FAILED tests/test_timeout.py::test_long_running - Failed: Timeout >1.0s
but the following output.json shows the step passed:
[
{
"keyword": "Feature",
"uri": "tests/timeout.feature",
"name": "Timeout demonstration",
"id": "tests/timeout.feature",
"line": 1,
"description": "Demonstration of a long-running operation",
"language": "en",
"tags": [],
"elements": [
{
"keyword": "Scenario",
"id": "test_long_running",
"name": "Long running operation",
"line": 4,
"description": "",
"tags": [],
"type": "scenario",
"steps": [
{
"keyword": "When",
"name": "I wait for a really long time",
"line": 5,
"match": {
"location": ""
},
"result": {
"status": "passed",
"duration": 0
}
}
]
}
]
}
]
pytest==8.3.5
pytest-bdd==8.1.0
pytest-timeout==2.3.1
Metadata
Metadata
Assignees
Labels
No labels