Security audit

[agriyakhetarpal]

Conduct a security audit of this repository's workflows and address any actionable issues. This shall involve:

• Look at our security posture (secrets are rotated frequently and reliably, only maintainers can access repository settings)
• Add docs on verifying releases using our existing attestations
• Perhaps add a feature to pyodide-build to verify attestations if --secure mode is enabled or maybe by default (requires sigstore as a dependency)
• Add Dependabot configuration to keep GitHub Actions up to date
• Add Zizmor as a pre-commit hook with the --pedantic option and address all findings (permissions, template injection, secrets, cache poisoning, etc.)

[Removed from tracker as this is not in our control]: Ensure that releases and actions are effectively immutable (wait for github/roadmap#1137 and github/roadmap#1103 – later this year, hopefully)

[agriyakhetarpal]

• I added pre-commit.ci as a GitHub App to the repository in Add Zizmor and other pre-commit hooks #20 (we can trust it, I guess), but I've disabled autofixes so that all the fixes we do are manually added. The two hooks are some local ones + Zizmor for static analysis.
• We are not using any secrets in the repository, so secrets: inherit has been dropped

[agriyakhetarpal]

Hi @ryanking13, I've spent a bit of time thinking about the provenance verification feature I proposed in the above points. sigstore's Python client should make it simple to implement within a couple of hours, and I could take it on. However, I would love to hear your thoughts on it before I begin. Here's my tentative approach (it's probably better discussed as a PR in pyodide-build for tracking purposes, so I'm adding a brief writeup here for now).

• Add a verify_sigstore_artifacts() function to pyodide_build.common that fetches the attestation from GitHub releases + bundles it + validates it using sigstore.Verifier().verify_artifact()
• Pass a --secure or --verify-provenance CLI argument through download_and_unpack_archive, and any other CLI
• Add an associated verify_provenance parameter in CrossBuildEnvManager.install()
• Associated docs and a test (could be an integration test, rather)

Questions we need to think about:

• Should we enable it by default, i.e., opt-in vs opt-out? (we'll provide a pyodide config variable to disable it, though)
  • If no, would there be a timeline to enable it by default (a few releases from now)?
• What would our target audience be?
  • would end-users use it? (I'm sure some users would appreciate it, especially in corporate environments)
    • if we were to enable it by default, we could make it happen silently
  • would downstream projects like pypa/cibuildwheel like to use it?
• Is there a cost to adding an extra dependency like sigstore?
  • my opinion: sigstore takes its supply chain security pretty seriously, so I would doubt it (see https://www.wheelodex.org/projects/sigstore/ for a list of dependencies, some of which coincide with ours)
  • if not enabled by default in question 1, could it be an optional dependency?

[ryanking13]

Thanks for improving our project's safety @agriyakhetarpal!

Add a verify_sigstore_artifacts() function to pyodide_build.common that fetches the attestation from GitHub releases + bundles it + validates it using sigstore.Verifier().verify_artifact()

I think it can be placed in xbuildenv if it is for verifying the xbuildenv archive. Or, in a clean-code manner, there should be a function that "downloads" the attestation, and another function that "verifies" it.

Pass a --secure or --verify-provenance CLI argument through download_and_unpack_archive, and any other CLI

I like --verify-provenance. I think we can add it first in pyodide xbuildenv install command, but not in pyodide build or pyodide build-recipes. We can tell people "hey if you want to verify the xbuildenv, download and verify it explicitly".

Associated docs and a test (could be an integration test, rather)

I think we can add both unittest and integration test, one that requires network access, one that doesn't.

Should we enable it by default, i.e., opt-in vs opt-out?

I would like to make it opt-in, disabling it by default. I don't want to introduce any mandatory security checks that may harm the user experience. Also I would like to make sigstore optional dependency.

would end-users use it?

Not sure. In my experience, end users are not interested in security unless there are legal regulations or unless they actually experience some incidents.

would downstream projects like pypa/cibuildwheel like to use it?

Maybe? We can discuss it with cibuildwheel folks.

Is there a cost to adding an extra dependency like sigstore?

I think they have lots of cryptography-related packages as dependencies. I would like to make it optional.

[agriyakhetarpal]

Many thanks for the detailed responses, Gyeongjae!

I like --verify-provenance. I think we can add it first in pyodide xbuildenv install command, but not in pyodide build or pyodide build-recipes. We can tell people "hey if you want to verify the xbuildenv, download and verify it explicitly".

--secure is quite vague about what is being secured, so I agree with --verify-provenance as well. Your point about keeping it for pyodide xbuildenv install also makes sense, and it could be implemented with --url for now (since we haven't published attestations for some of the stable Pyodide releases in the past). I'll start a PR for it when I can. The first step is to get attestations from Sigstore for this repository.