Is the idea of adding a required-tag rule reasonable?

[eripley]

Hey mate @mskelton! You’re doing great work! What do you think about adding a new rule to detect required tags in test.describe? The allowed and disallowed tag rule covers most cases, but when you need to enforce a strict policy on your annotations and ensure that every required tag is applied, you’re completely powerless.

It could be extremely helpful if your team has its own requirements for tag annotations and wants to validate them. It’s also useful if you’re running tests through a TestOps process with Allure (or similar) and you have to provide strict tag to generate doca from your tests.

I actually ran into the same problem on my project and have already implemented it in my custom config. Happy to share it with everyone!

[mskelton]

Could you explain more your use case? What's the reason for requiring a tag for every test/describe block?

[eripley]

So if users forget to apply the required tags, which are used to generate dashboards in grafana or allure, we'll miss important metrics and break the documentation generation process in TestOps.

[mskelton]

Can you share any docs or examples?

[eripley]

Here’s the official Allure TestOps documentation on tags and layer mapping: link.

Additionally, the main policy around metadata extracted from test tags is described here.

Here’s how it works in my setup:

I want to emphasize that having an option to enforce required tags would be useful not only within TestOps itself, but also across different CI systems such as GitLab or any other pipeline that relies on tag annotations to orchestrate test execution. In our case, tags are strictly enforced at the CI level — if a required tag is missing, the corresponding test will never even be triggered.

And how it transforms in Allure

• in the settings:

• on the test-case:

Also couple strings from my CI pipeline to run the tests

.regex_contract_include_variables:
  variables:
    EXTERNAL_REGEX_INCLUDE_PATTERN: "(?=.*@contract)(?=.*@${SERVICE})"

        if [[ "$DYNAMIC_DEPLOY_BRANCH" == "stage" ]] || [[ "$DYNAMIC_DEPLOY_BRANCH" =~ ^release.*$ ]]; then
            echo "ℹ️ Regression demand from external services"
            allurectl watch -- ${PLAYWRIGHT_RUN_API} --workers ${API_WORKERS_AMOUNT} --grep-invert $EXTERNAL_REGEX_EXCLUDE_PATTERN --grep $EXTERNAL_REGEX_INCLUDE_PATTERN

[eripley]

Here’s a simple snippet from my custom plugin that I intend to contribute to eslint-plugin-playwright:

[mskelton]

This is just such a bizzare way to structure tests, and just doesn't make sense at all to me. This is what Playwright project are for, and let you specify how you want to target tests for a given project (e.g. api, ui), typically by using directory matching.

I'm going to close this discussion as not planned as it's just way too niche of a use case to warrant a rule that IMO is really confusing.