[Pytest Mechanism related] How to auto define the test cases then run them?

[lucky33newman]

This thing is like this, I am building up a pytest that collects and judges the status of the Yocto package tests.

A traditional Yocto package test has a log like this:

........
Start testing decompress......
PASS: bzip2 /usr/lib/xxx.bz2 decompress successfully.
........

I remember bzip2 package test contains 400+ cases, I have to collect all of them, so I imported 'regex' to do the work.

After regex and some file IO, It, part of the pytest runtest script would create a JSON file that contains the result, it is like this:

case_status:{'bzip2_xxx_decompress','PASS','pass'}

First 'PASS' is the Yocto package test status, second 'pass' stands for that we make rules to judge its status.

Here comes the question, in our development workflow, we first define no pytest cases, only let the pytest runtest script run, all the case name being collected from the JSON file, then we manually run the code generater script (written by me myself :) ) to generate the pytest cases like this:

define test_bzip2_xxx_decompress:
......
......

Some might ask why do we need the code generater, it seems simple, what if we collected 400+ cases, the code for defines it is gonna be over 3000 lines. I can not type it all day long right?

It, the code generater brought a problem like this, if upstream, the Yocto package tests change its case name, for example, from xxx.bz2 to yyy.bz2, we have to maintain it, but it can be done by the code generater automatically, that those defines would be generated first, then run.

Yet the question is how. I need some advices, for it is pytest mechanism related, there should be solutions.

[lucky33newman]

I mean, is there such mechanism in Pytest?

Maybe not, but I am not sure.

Anybody?

[RonnyPfannschmidt]

Pytest is a test framework, not a test result aggregator

[lucky33newman]

Yep, I am gonna try some other ways. To get my aggregator moving. XD Ronny Pfannschmidt ***@***.***> 于 2022年10月24日周一 15:22写道：

…

Pytest is a test framework, not a test result aggregator — Reply to this email directly, view it on GitHub <#10399 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AWBCZ3NSN35J64ML4HSZPQDWEY2MZANCNFSM6AAAAAARJVSEYA> . You are receiving this because you authored the thread.Message ID: ***@***.***>