How should we use the default test suite class with a custom scheduler?

[pevogam]

Hi all, we have a small plugin that tries to integrate better with avocado and reuse avocado-run and avocado-list as much as possible with our custom scheduling mechanism. Our scheduler is refused with the error Suite creation for runner "traverser" is not supported. Note that if I do

diff --git a/avocado/core/suite.py b/avocado/core/suite.py
index 823ae05eb..c91587a43 100644
--- a/avocado/core/suite.py
+++ b/avocado/core/suite.py
@@ -340,7 +340,7 @@ class TestSuite:
             config.update(job_config)
         config.update(suite_config)
         runner = config.get("run.suite_runner")
-        if runner == "nrunner":
+        if runner in ["nrunner", "traverser"]:
             suite = cls._from_config_with_resolver(config, name)
             if suite.test_parameters or suite.variants:
                 suite.tests = suite._get_test_variants()

then everything works smoothly with no problem of any kind which made me wonder why this value is hardcoded there. Could we make this configurable perhaps? I am aware we can inherit via a separate test suite class but it seemed like an overkill for what is needed, especially considering the above would work just fine. Is there perhaps a better way to provide a custom scheduler with the default avocado classes?

At present I have to rely on the following to make the above work:

diff --git a/avocado_i2n/plugins/auto.py b/avocado_i2n/plugins/auto.py
index 580e6c92..6af1d454 100644
--- a/avocado_i2n/plugins/auto.py
+++ b/avocado_i2n/plugins/auto.py
@@ -76,5 +76,36 @@ class Auto(CLI):
             config["params"] = []
         cmd_parser.params_from_cmd(config)
 
+        # TODO: crude override of the avocado test suite due to
+        # hardcoded suite runner
+        from avocado.core.suite import TestSuite, TestSuiteError
+        def from_config(config, name=None, job_config=None):
+            suite_config = config
+            config = settings.as_dict()
+            if job_config:
+                config.update(job_config)
+            config.update(suite_config)
+            runner = config.get("run.suite_runner")
+            # TODO: hardcoded runner type, this method works perfectly fine for
+            # our own traverser scheduler (suite runner)
+            if runner in ["nrunner", "traverser"]:
+                suite = TestSuite._from_config_with_resolver(config, name)
+                if suite.test_parameters or suite.variants:
+                    suite.tests = suite._get_test_variants()
+            else:
+                raise TestSuiteError(
+                    f'Suite creation for runner "{runner}" ' f"is not supported"
+                )
+
+            if not config.get("run.ignore_missing_references"):
+                if not suite.tests:
+                    msg = (
+                        "Test Suite could not be created. No test references "
+                        "provided nor any other arguments resolved into tests"
+                    )
+                    raise TestSuiteError(msg)
+
+            return suite
+        TestSuite.from_config = from_config
         log.debug(f"Setting test runner to 'traverser'")
         config["run.suite_runner"] = "traverser"

and is all needed in order to overwrite the hard-coded value.

Disclaimer: We have our scheduler work just fine from our own CLICmd plugin but the above is an approach that should enable better integration with avocado's own run/list plugins.

[richtja]

Hi @pevogam the avocado nrunner uses the SuiteRunner plugin interface, and therefore it should be completely pluggable and there shouldn't be any obstacles to use a different scheduler instead of the nrunner. Therefore, I see this as a bug. Can you please open an issue about this so we could better track it? Thank you.

[pevogam]

Sure, done