testinfra with ansible : capacity to exclude groups ?

[mneveu-mx]

Hello,

I use pytest/testinfra with ansible module for unitary tests on my linux nodes.
The command I use is like :
/usr/local/bin/pytest [... options ...] **--hosts=ansible://group1,ansible://group2**
For example, group2 can be a subelement of group1.

My problem : I want to execute some tests on group 1 hosts but without the hosts of group 2..
Is there a way to exclude group2 with a specific syntax ?

Note : with Ansible, there is a syntax to exclude a group from a playbook, for example :
hosts: group1 !group2
So maybe it can be the same with testinfra.

Thank you by advance,
Maxime

[philpep]

Yes, I think we could implement this using the same ansible syntax as you suggest.

[xeor]

I'm looking for almost the same thing. Something I can mark a test with to only include that test for certain groups. Something like @pytest.mark.ansible_only_group('mygroup').

Using parametrize with testinfra is nice, but I end up with hundreds of yellow skip statuses, since I currently need to do a check inside the test like this..

    if 'mygroup' not in host.ansible.get_variables()['group_names']:
        pytest.skip()

[danschmidt5189]

This would be a really useful feature — any updates? Happy to devote some time to it if I can get direction on implementation.

For background, my use-case is running tests as part of an Ansible playbook run. Essentially, the playbook updates a host, runs the tests for that specific host, and then either finalizes or rolls back the update based on the test results. Since hosts within a group are identical, it makes sense to group their test-cases by hostgroup rather than host, i.e. something like:

@pytest.mark.limit('web')
def webservers_specific_test(host):
    # ...

@pytest.mark.limit('db')
def dbservers_specific_test(host):
    # ...

Unless I'm missing something, it doesn't seem like this is possible with testinfra_hosts — that seems to be literally host-specific and to not support groups.

Edit: Just wanted to clarify exactly what's confusing here. Suppose you have the following test structure:

# test/web_test.py
testinfra_hosts = ['web']
def web_test(host):
    # ...

# test/db_test.py
testinfra_hosts = ['db']
def db_test(host):
    # ...

Then I would expect that if I run the following, pytest will run web_test but not db_test:

py.test --hosts 'ansible://web'

However what actually happens is that both tests are run.