-
Notifications
You must be signed in to change notification settings - Fork 560
Open
Labels
check/CI-Testsgood first issueGood for newcomersGood for newcomerskind/docsImprovements or additions to documentationImprovements or additions to documentation
Description
Describe the bug
The CITest evaluation has inconsistencies between documentation and implemention.
The documentation says:
This check tries to determine if the project runs tests before pull requests are merged. It is currently limited to repositories hosted on GitHub, and does not support other source hosting repositories (i.e., Forges).
However, the implementation only considers succesful CI runs:
scorecard/checks/evaluation/ci_tests.go
Lines 104 to 110 in de022da
for _, cr := range r.CheckRuns { | |
if cr.Status != "completed" { | |
continue | |
} | |
if cr.Conclusion != success { | |
continue | |
} |
scorecard/checks/evaluation/ci_tests.go
Lines 84 to 86 in de022da
if status.State != success { | |
continue | |
} |
As such, projects that do run tests in their CI will fulfil the requirement specified in the documentation, but they will fail the check.
Metadata
Metadata
Assignees
Labels
check/CI-Testsgood first issueGood for newcomersGood for newcomerskind/docsImprovements or additions to documentationImprovements or additions to documentation
Type
Projects
Status
Todo