Trying to determine if test has been an expected fail.

[f00dog-x300]

I'm trying to see if the test that was run was failed due to an assertion or if it was failed because of an expected fail.

test_result = "failed" if (request.node.rep_call.failed or request.node.rep_call.xfailed) else "passed"

I can't find any documentation on how to do that, but I have been using request.node.rep_call.failed to check if a test was failed and set a variable (test_result) as "failed." However, I cannot find anything on how to check if the call was an xfail. Can someone help me? The above was a feeble attempt to try to figure out it.

[EmptyRabbit]

It seems pytest_runtest_makereport hook can check a test result is xfail or not

@pytest.hookimpl(tryfirst=True, hookwrapper=True)
def pytest_runtest_makereport(item):
    outcome = yield
    rep = outcome.get_result()
    
    rep.xfailed = False
    if hasattr(rep, "wasxfail"):
        if rep.passed:
            rep.xfailed = True    # unexpected pass
    setattr(item, "rep_" + rep.when, rep)

[f00dog-x300]

@EmptyRabbit , does the pytest_runtest_makereport get run for each test run or after the test suite is run?

[EmptyRabbit]

@EmptyRabbit , does the pytest_runtest_makereport get run for each test run or after the test suite is run?

Each test run, as mentioned in the documentation：Called to create a TestReport each of the setup, call and teardown runtest phases of a test item

[eleanorjboyd]

Hi! I found this thread because I was trying to correctly store the status xfail tests. Below is the code I am running, the plugin I wrote to connect to the given hook, and the output from the plugin. Im confused because the outcome of this xfail tests is "passed" during "setup" and "teardown" but then is listed as "skipped" during "call". Why does the xfail tests list as "skipped" during "call"? With all other tests I have written, the "call" phase has been accurate in collecting pass/fails etc but I wasn't sure about xfail. What does the skip mean and is it always going to say skip for xfail tests during this phase? Thanks and I appreciate any help!

@pytest.mark.xfail
def test_example():
    assert 1 == 2, "This test is expected to fail because 1 is not equal to 2"

@pytest.hookimpl(tryfirst=True, hookwrapper=True)
def pytest_runtest_makereport(item):
    outcome = yield
    report = outcome.get_result()
    print("pytest_report_teststatus")
    print("when", report.when)
    print("outcome", report.outcome)
    print("xfail", hasattr(report, "wasxfail"))

test_xfail.py 
pytest_report_teststatus
when setup
outcome passed
xfail False
pytest_report_teststatus
when call
outcome skipped
xfail True
xpytest_report_teststatus
when teardown
outcome passed
xfail False