CI is too noisy #4
Description
NetDEF and LabN CI systems both post comments as bot users when a test suite passes or fails, for example:
Because a new comment is posted for each new test result, this can become very noisy, even when previous comments are collapsed. The comment section of a pull request is intended for human users to use for discussing the pull request. Posts from bots disrupt this use case.
GitHub hastwo complementary features, together called "Status checks", that are designed for this purpose. These are represented by compact status lines at the bottom of a pull request, in the status box:
Presently all CI systems post "Statuses". These statuses contain all necessary information for reviewers - whether the CI run passed or failed, and a link to further details.
In the case of a CI success, it is unnecessary to post a comment indicating that the CI passed. This is redundant information because the same information is present in the form of a green check mark at the bottom of a pull request.
In the case of a CI failure, all details about the failure can be placed directly in GitHub as a "check" result, for example here is a recent one posted by frrbot:
https://github.com/FRRouting/frr/pull/10006/checks?sha=b13f35ec6772ae3a2bcff78c9b4ff388278685b3
In this example, rather than posting a comment, the bot is able to post a Check result instead, which frees up the comment section for human users, improving the workflow.
Therefore I request that NetDEF CI transition to using check results in order to remove PR noise. This can be accomplished in two steps:
- Stop posting comments on success
This is easy because no additional work needs to be done. All that needs to be done is to remove the code which posts a comment on CI success.
- Move comment contents to a Status Check and stop posting comments on failure
This requires using the Checks API and requires some effort.