@pytest.mark.* priority seems odd

[pmeier]

Suppose I have a test class for which I want to turn warnings into errors. However, some of the parametrizations I'm going to test will emit warnings and I just want to ignore them. I thought, I can write something like

import pytest
import warnings


@pytest.mark.filterwarnings("error")
class TestMarkClass:
    @pytest.mark.parametrize("_", [pytest.param(None, marks=pytest.mark.filterwarnings("ignore"))])
    def test_foo(self, _):
        warnings.warn("foo")

However, it seems that the class wide mark takes priority over the one in the parametrization. On the flip side, if I mark the individual tests like

class TestMarkMethods:
    @pytest.mark.filterwarnings("error")
    @pytest.mark.parametrize("_", [pytest.param(None, marks=pytest.mark.filterwarnings("ignore"))])
    def test_bar(self, _):
        warnings.warn("bar")

    @pytest.mark.parametrize("_", [pytest.param(None, marks=pytest.mark.filterwarnings("ignore"))])
    @pytest.mark.filterwarnings("error")
    def test_baz(self, _):
        warnings.warn("baz")

the mark in the parametrization takes priority. Running everything together with -rA set gives

======================================= FAILURES =======================================
_____________________________ TestMarkClass.test_foo[None] _____________________________
Traceback (most recent call last):
  File "/home/philip/git/pytorch/torchvision/main.py", line 9, in test_foo
    warnings.warn("foo")
UserWarning: foo
======================================== PASSES ========================================
=============================== short test summary info ================================
PASSED main.py::TestMarkMethods::test_bar[None]
PASSED main.py::TestMarkMethods::test_baz[None]
FAILED main.py::TestMarkClass::test_foo[None] - UserWarning: foo
============================= 1 failed, 2 passed in 0.68s ==============================

Is this intentional behavior? I couldn't find anything about the priority order in the documentation.

[RonnyPfannschmidt]

this is a bug in mark application due to parametrization and collection being muddled

this would resolve with the introduction of FunctionDefinition, which is currently blocked by a few other refactorings missing
so a workaround may be necessity, which is also a "breaking change"

[pmeier]

Thanks for explanation. Could you point me to a PR / issue / ... that deals with it so I can subscribe to updates?

[RonnyPfannschmidt]

There is no bug /pr yet,

[pmeier]

Ah, ok. Do you want me to open an issue then?

[RonnyPfannschmidt]

Yes please

[pmeier]

See #10406.