how to flag a test case as failed when the code under test raises an exception?

[Coder2023Coder]

I am testing several Python code snippets by pytest. One of the code snippets under test uses a built-in function from a Python library without importing the related library. When the testing process reaches the mentioned code, "NameError" is raised, and the whole testing is stopped. How should I take care of such cases in pytest so that pytest flags the tests related to that program as failed and continues testing the rest of the code snippets without termination? I am looking for a solution that deals with any kind of exception being raised like above.

I am using the following setup:

pytest_params = ["-v", "tests.py", "-c", "ini_file.ini"]
pytest.main(pytest_params)

Thanks.

[ChVeen]

[The-Compiler]

That's not what xfail is/does. pytest.fail(...) fails a test, xfail marks it as "expected to fail" (which is basically a passing test).

Also, there's no point in having a try/except like that - pytest will already fail a test when there is an exception, so all this snippet would do is hide the information about what failed.

[Coder2023Coder]

Hi ChVeen
As The-Compiler explained, neither pytest.fail nor try... except works here.

[The-Compiler]

Can you show the exact output/behavior you get instead of describing it, and maybe show some of the related code too?

[Coder2023Coder]

Suppose the code under test is as follows:

def foo(lst: List(int)) -> List(int):
             # body of the function

Here, List has to be imported but the import which is from typing import List is missing. Assume I have written 5 tests to run against foo by using pytest. As soon as pytest tries to run the first test, the program exits with the following error message:

NameError: List is not defined

It means before pytest determines the test result, Python raises that error and exits. I want pytest to handle the situation and makes each individual test fail instead of whole process suddenly stops. This is a correct fail as the import part is missing in the code and therefore code is not complete. My intention is that in all cases where an import is missing or any exception is raised because of not being correctly handled in the code under test, pytest fails the written tests and creates the report as normal without Python exits.

[The-Compiler]

Your module isn't importable - it's the same scenario as if there was e.g. a syntax error in there, it fails at import time before your test is even run.

If you really want only a single test to fail in this scenario, you'll need to move the import inside the test function. But I'm not sure if this is really a problem worth solving.

[nicoddemus]

To complement what @The-Compiler means: pytest works by importing the file and then inspecting the test in functions there, before running them; if pytest cannot the import the file at all, it cannot possibly run the functions in there (as they do not exist, Python has not even created them).

A parallel in a compiled language (like C++ or Rust), would be the same as trying to run some tests, but the test suite itself is not even compiling successfully.