Feat(anta_runner): Add toggle to sort tests by status

[vitthalmagadum]

Change Summary

Add toggle to sort tests by status

Related Issue(s)

Fixes #5949

Component(s) name

arista.avd.anta_runner

Proposed changes

Add toggle to sort tests by status

How to test

Tested locally

anta_report.json
anta_report.md

Checklist

User Checklist

• N/A

Repository Checklist

• My code has been rebased from devel before I start
• I have read the CONTRIBUTING document.
• My change requires a change to the documentation and documentation have been updated accordingly.
• I have updated molecule CI testing accordingly. (check the box if not applicable)

[github-actions]

Review docs on Read the Docs

To test this pull request:

# Create virtual environment for this testing below the current directory
python -m venv test-avd-pr-6066
# Activate the virtual environment
source test-avd-pr-6066/bin/activate
# Install all requirements including PyAVD
pip install "pyavd[ansible] @ git+https://github.com/vitthalmagadum/avd.git@anta_runner/sort-toggle#subdirectory=python-avd" --force
# Point Ansible collections path to the Python virtual environment
export ANSIBLE_COLLECTIONS_PATH=$VIRTUAL_ENV/ansible_collections
# Install Ansible collection
ansible-galaxy collection install git+https://github.com/vitthalmagadum/avd.git#/ansible_collections/arista/avd/,anta_runner/sort-toggle --force
# Optional: Install AVD examples
cd test-avd-pr-6066
ansible-playbook arista.avd.install_examples

[sonarqubecloud]

Quality Gate passed

Issues
1 New issue
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[carl-baillargeon]

Let's take the opportunity to improve the data model for report:

"report": {
  "type": "dict",
  "options": {
    "csv_output": {"type": "str"},
    "md_output": {"type": "str"},
    "json_output": {"type": "str"},
    
    "filters": {
      "type": "dict",
      "options": {
        "exclude_statuses": {  # Renamed from hide_statuses
          "type": "list",
          "elements": "str",
          "choices": ["success", "failure", "error", "skipped", "unset"]
        }
      }
    },
    
    "sorting": {
      "type": "dict",
      "options": {
        "status_priority": {
          "type": "list",
          "elements": "str",
          "choices": ["success", "failure", "error", "skipped", "unset"],
          "default": ["error", "failure", "skipped", "success", "unset"],
        },
        "sort_by": {
          "type": "list",
          "elements": "str",
          "choices": ["device", "categories", "test", "description", "result", "custom_field"],
          "default": ["device", "categories", "test", "description", "result", "custom_field"],
        }
      }
    }
  }
}

status_priority would act as the primary sort. We would first group all tests by their status, following status_priority order.

@JulioPDX Would you be OK with the following:

For example:

sorting:
  status_priority:
    - failure
    - error
    - skipped
  sort_by:
    - device

With this:

1. All failures at the top (sorted by device).
2. Then, all errors (sorted by device).
3. Then, all skipped tests (also sorted by device).
4. Any remaining statuses (like "success") would come last.

I know we don't have all functionalities in ANTA ResultManager but we can do it in anta_workflow no problem.

[JulioPDX]

@carl-baillargeon The workflow makes sense. It solves the issue of having to search through the reports on failures (helpful on an active change control or troubleshooting event).