pytest-xdist crashing when using sys.argv

[pawan-qa]

initialized: 2/2 workers[gw0] node down: Traceback (most recent call last):
File "/Users/pawankumar/automation/flex-automated-tests/virtual-env-automation/lib/python3.10/site-packages/_pytest/config/init.py", line 641, in _importconftest
mod = import_path(conftestpath, mode=importmode, root=rootpath)
File "/Users/pawankumar/automation/flex-automated-tests/virtual-env-automation/lib/python3.10/site-packages/_pytest/pathlib.py", line 567, in import_path
importlib.import_module(module_name)
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/importlib/init.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1050, in _gcd_import
File "", line 1027, in _find_and_load
File "", line 1006, in _find_and_load_unlocked
File "", line 688, in _load_unlocked
File "/Users/pawankumar/automation/flex-automated-tests/virtual-env-automation/lib/python3.10/site-packages/_pytest/assertion/rewrite.py", line 178, in exec_module
exec(co, module.dict)
from utils.constants import Constants
File "/Users/pawankumar/automation/flex-automated-tests/droms-automation-pytest/utils/constants.py", line 11, in
env = sys.argv[-3] # '-3' is index of list returned by sys.argv
IndexError: list index out of range

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/Users/pawankumar/automation/flex-automated-tests/virtual-env-automation/lib/python3.10/site-packages/execnet/gateway_base.py", line 1157, in executetask
exec(co, loc)
File "/Users/pawankumar/automation/flex-automated-tests/virtual-env-automation/lib/python3.10/site-packages/xdist/remote.py", line 345, in
config = _prepareconfig(args, None)
File "/Users/pawankumar/automation/flex-automated-tests/virtual-env-automation/lib/python3.10/site-packages/_pytest/config/init.py", line 331, in _prepareconfig
config = pluginmanager.hook.pytest_cmdline_parse(
File "/Users/pawankumar/automation/flex-automated-tests/virtual-env-automation/lib/python3.10/site-packages/pluggy/_hooks.py", line 493, in call
return self._hookexec(self.name, self._hookimpls, kwargs, firstresult)
File "/Users/pawankumar/automation/flex-automated-tests/virtual-env-automation/lib/python3.10/site-packages/pluggy/_manager.py", line 115, in _hookexec
return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
File "/Users/pawankumar/automation/flex-automated-tests/virtual-env-automation/lib/python3.10/site-packages/pluggy/_callers.py", line 130, in _multicall
teardown[0].send(outcome)
File "/Users/pawankumar/automation/flex-automated-tests/virtual-env-automation/lib/python3.10/site-packages/_pytest/helpconfig.py", line 104, in pytest_cmdline_parse
config: Config = outcome.get_result()
File "/Users/pawankumar/automation/flex-automated-tests/virtual-env-automation/lib/python3.10/site-packages/pluggy/_result.py", line 114, in get_result
raise exc.with_traceback(exc.traceback)
File "/Users/pawankumar/automation/flex-automated-tests/virtual-env-automation/lib/python3.10/site-packages/pluggy/_callers.py", line 77, in _multicall
res = hook_impl.function(*args)
File "/Users/pawankumar/automation/flex-automated-tests/virtual-env-automation/lib/python3.10/site-packages/_pytest/config/init.py", line 1075, in pytest_cmdline_parse
self.parse(args)
File "/Users/pawankumar/automation/flex-automated-tests/virtual-env-automation/lib/python3.10/site-packages/_pytest/config/init.py", line 1425, in parse
self._preparse(args, addopts=addopts)
File "/Users/pawankumar/automation/flex-automated-tests/virtual-env-automation/lib/python3.10/site-packages/_pytest/config/init.py", line 1327, in _preparse
self.hook.pytest_load_initial_conftests(
File "/Users/pawankumar/automation/flex-automated-tests/virtual-env-automation/lib/python3.10/site-packages/pluggy/_hooks.py", line 493, in call
return self._hookexec(self.name, self._hookimpls, kwargs, firstresult)
File "/Users/pawankumar/automation/flex-automated-tests/virtual-env-automation/lib/python3.10/site-packages/pluggy/_manager.py", line 115, in _hookexec
return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
File "/Users/pawankumar/automation/flex-automated-tests/virtual-env-automation/lib/python3.10/site-packages/pluggy/_callers.py", line 152, in _multicall
return outcome.get_result()
File "/Users/pawankumar/automation/flex-automated-tests/virtual-env-automation/lib/python3.10/site-packages/pluggy/_result.py", line 114, in get_result
raise exc.with_traceback(exc.traceback)
File "/Users/pawankumar/automation/flex-automated-tests/virtual-env-automation/lib/python3.10/site-packages/pluggy/_callers.py", line 77, in _multicall
res = hook_impl.function(*args)
File "/Users/pawankumar/automation/flex-automated-tests/virtual-env-automation/lib/python3.10/site-packages/_pytest/config/init.py", line 1153, in pytest_load_initial_conftests
self.pluginmanager._set_initial_conftests(
File "/Users/pawankumar/automation/flex-automated-tests/virtual-env-automation/lib/python3.10/site-packages/_pytest/config/init.py", line 563, in _set_initial_conftests
self._try_load_conftest(anchor, importmode, rootpath)
File "/Users/pawankumar/automation/flex-automated-tests/virtual-env-automation/lib/python3.10/site-packages/_pytest/config/init.py", line 580, in _try_load_conftest
self._getconftestmodules(anchor, importmode, rootpath)
File "/Users/pawankumar/automation/flex-automated-tests/virtual-env-automation/lib/python3.10/site-packages/_pytest/config/init.py", line 609, in _getconftestmodules
mod = self._importconftest(conftestpath, importmode, rootpath)
File "/Users/pawankumar/automation/flex-automated-tests/virtual-env-automation/lib/python3.10/site-packages/_pytest/config/init.py", line 645, in _importconftest
raise ConftestImportFailure(conftestpath, exc_info) from e
_pytest.config.ConftestImportFailure: IndexError: list index out of range (from /Users/pawankumar/automation/flex-automated-tests/conftest.py)

maximum crashed workers reached: 8

[RonnyPfannschmidt]

This is a user error, please turn this into a cli arg pytest can take

Sys.argv is never the same on a worker as it is on the coordinator

[pawan-qa]

@RonnyPfannschmidt

def pytest_addoption(parser):
parser.addoption("--n", action="store",default=3)

def pytest_generate_tests(metafunc):
option_value = metafunc.config.option.tenant
if 'n' in metafunc.fixturenames and option_value is not None:
metafunc.parametrize("n", [option_value])

when I use pytest -sv , 3 workers are not assigned to my tests. Instead it runs serially.

[pawan-qa]

Can you pls share any example code to pass multiple workers as cli arg when using sys.argv. Thanks

[RonnyPfannschmidt]

There is no correct way to use sys.argv on a worker

So no

[pawan-qa]

Does this mean that we can't run parallel tests if using sys.argv ? In my case, I can't avoid using sys.argv.

[RonnyPfannschmidt]

AS far as I'm concerned the only way you can't avoid it is if someone with power wants to sabotage you with a rule