Naming parameters for parametrized tests

[mentaal]

It's possible to explicitly name each "row" of a parametrized testcase with an ID. You can then filter which testcases you want to run with the -k switch from the command line.

Is it possible to explicitly name each "column", i.e. the parameters themselves?

The results of a test run might look something like this:

..\..\..\temp\test_counter.py::test_param[240-255-250-False] PASSED
..\..\..\temp\test_counter.py::test_param[240-255-255-False] PASSED
..\..\..\temp\test_counter.py::test_param[240-255-240-False] PASSED
..\..\..\temp\test_counter.py::test_param[240-10-255-False] PASSED
..\..\..\temp\test_counter.py::test_param[240-10-256-False] PASSED
..\..\..\temp\test_counter.py::test_param[240-10-0-False] PASSED
..\..\..\temp\test_counter.py::test_param[240-10-11-True] PASSED
..\..\..\temp\test_counter.py::test_param[100-200-120-False] PASSED
..\..\..\temp\test_counter.py::test_param[100-200-100-False] PASSED
..\..\..\temp\test_counter.py::test_param[100-200-201-True] PASSED

It would be handy if the test result referred to the parameters in some way as well so the reader of the test report can easily understand what the arguments mean. I know that it would be difficult to do that without making the result very verbose.

It would be very cool though if you could then select which tests you want to run based on the columns.
For example, in this case, to run all cases of test_param where the first argument == 240.

[bluetech]

I think it would be interesting to explore how this could work! Do you have some syntax for -k in mind on how you would want to specify this?

[mentaal]

I guess there are number of ways to even define the feature. The simplest I guess would be a static pinning of a column to be a certain value. So assuming a particular testcase, call it test_param was parametrized with a number of variables, one of which is var_a, then one way would be to then specify a constraint like this: -k test_param[var_a==240] though I don't know if that conflicts with the existing syntax?

Of course, if such a feature would be entertained another possibility would be to have a more feature rich version where a variable can be constrained, not just pinned. And whilst we're at it, we could throw the kitchen sink at it and allow multiple constraints simultaneously.

For example: -k test_param[var_a<240,var_a>100,var_b==200] so this would require a more sophisticated expression parser. Just thinking about the inevitable corner cases, this already wouldn't work with commas or at least prevent future updates with lists/tuples etc though this may be taking things too far. Maybe a safe first cut would be to specify the constraints as embedded strings like this: -k test_param['var_a<240', 'var_a>100', 'var_b==200'] which already seems a bit yucky.

I also think that anything like this would have limited usability if the various parameters are not somehow printed in the output as well so a user can reference them without having to consult the test case source the whole time.

Even something like this: test_param[240-255-250-False] (var_a-var_b-var_c-var_d) would be very helpful here I think.