Skip to content

Not all checks returned for repo #3648

@mtcolman

Description

@mtcolman

Describe the bug
Hi, this might not be a bug, but I didn't spot documentation that gave me the answer - apologies if I've missed something.

Reproduction steps
Steps to reproduce the behavior:

  1. run curl command (with jq formatting output), see results
$ curl -s https://api.securityscorecards.dev/projects/github.com/websocket-client/websocket-client |   jq -r '. as $input | "Repo Nam
e: \($input.repo.name)", "Score: \($input.score)", (.checks[] | "\(.name) Score: \(.score), Reason: \(.reason)")'

Repo Name: github.com/websocket-client/websocket-client
Score: 5.4
Maintained Score: 10, Reason: 12 commit(s) out of 30 and 10 issue activity out of 30 found in the last 90 days -- score normalized to 10
Code-Review Score: 4, Reason: found 18 unreviewed changesets out of 30 -- score normalized to 4
CII-Best-Practices Score: 0, Reason: no effort to earn an OpenSSF best practices badge detected
License Score: 10, Reason: license file detected
Dangerous-Workflow Score: 10, Reason: no dangerous workflow patterns detected
Packaging Score: -1, Reason: packaging workflow not detected
Token-Permissions Score: 0, Reason: detected GitHub workflow tokens with excessive permissions
Binary-Artifacts Score: 10, Reason: no binaries found in the repo
Pinned-Dependencies Score: 0, Reason: dependency not pinned by hash detected -- score normalized to 0
Fuzzing Score: 10, Reason: project is fuzzed
Security-Policy Score: 0, Reason: security policy file not detected
Vulnerabilities Score: 10, Reason: 0 existing vulnerabilities detected
Signed-Releases Score: -1, Reason: no releases found
Branch-Protection Score: 0, Reason: branch protection not enabled on development/release branches
SAST Score: 0, Reason: SAST tool is not run on all commits -- score normalized to 0

15 check results returned

$ curl -s https://api.securityscorecards.dev/projects/github.com/ossf/scorecard |   jq -r '. as $input | "Repo Name: \($input.repo.na
me)", "Score: \($input.score)", (.checks[] | "\(.name) Score: \(.score), Reason: \(.reason)")'

Repo Name: github.com/ossf/scorecard
Score: 9.7
Binary-Artifacts Score: 10, Reason: no binaries found in the repo
Branch-Protection Score: -1, Reason: internal error: error during branchesHandler.setup: internal error: githubv4.Query: Resource not accessible by integration
CI-Tests Score: 10, Reason: 30 out of 30 merged PRs checked by a CI test -- score normalized to 10
CII-Best-Practices Score: 5, Reason: badge detected: passing
Code-Review Score: 10, Reason: all changesets reviewed
Contributors Score: 10, Reason: 50 different organizations found -- score normalized to 10
Dangerous-Workflow Score: 10, Reason: no dangerous workflow patterns detected
Dependency-Update-Tool Score: 10, Reason: update tool detected
Fuzzing Score: 10, Reason: project is fuzzed
License Score: 10, Reason: license file detected
Maintained Score: 10, Reason: 30 commit(s) out of 30 and 0 issue activity out of 30 found in the last 90 days -- score normalized to 10
Packaging Score: 10, Reason: publishing workflow detected
Pinned-Dependencies Score: 9, Reason: dependency not pinned by hash detected -- score normalized to 9
SAST Score: 10, Reason: SAST tool is run on all commits
Security-Policy Score: 10, Reason: security policy file detected
Signed-Releases Score: -1, Reason: no releases found
Token-Permissions Score: 10, Reason: GitHub workflow tokens follow principle of least privilege
Vulnerabilities Score: 9, Reason: 1 existing vulnerabilities detected

18 check results returned

Expected behavior
This page says "There are currently 18 checks made across 3 themes: holistic security practises, source code risk assessment and build process risk assessment." I guess I would expect the full set of checks (18) to be returned, even if I was told it the repo hadn't been assessed against this check?

Metadata

Metadata

Assignees

No one assigned

    Labels

    area/apigood first issueGood for newcomershelp wantedCommunity contributions welcome, maintainers supportive of idea but not a high prioritykind/docsImprovements or additions to documentation

    Type

    No type

    Projects

    Status

    Todo

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions