Introduce an option to use the GitHub API to commit changes, for GPG #391

s0 · 2024-08-25T17:41:50Z

fixes #392

Turns out that there wasn't any good TypeScript libraries for making modifications to files directly, let alone one that looks at the files changed locally to determine what needs to be pushed to GitHub, so I went ahead and created one here: https://github.com/s0/ghcommit

I then used this repo as a test-case for the updated actions to see if it all works, and so far so good.

Tags are now signed by default:
And commits are also signed, and attributed to github-actions:
It also works with more complex situations, such as custom version commands, like in this repo where it auto-bumps the major version in the README: Version Packages s0/changesets-action#2

Would you like this functionality? And if so, do we want to just switch to this behavior by default + have a major version bump? or do we want to hide it behind an action input / argument for now and have it as opt-in?

My thinking is that just switching to this behavior overall makes the most sense, as most people probably want to attribute the commits to the owner of the GITHUB_TOKEN, and more and more people are going to require that commits are signed as the industry takes supply-chain-security more and more seriously.

changeset-bot · 2024-08-25T17:41:53Z

🦋 Changeset detected

Latest commit: 313b693

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package

Name	Type
@changesets/action	Minor

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

Andarist · 2024-08-25T20:40:32Z

This is a clever solution but tbh... I'm not sure if I like it for this project. It feels less maintainable - with git itself we have certain freedom in how we interact with it. I'm also worried that this would result in hitting rate limits more often (which already is an issue that this action hits from time to time).

s0 · 2024-08-25T23:17:03Z

Hmm rate limits is a fair concern. Would you support having this as an opt-in feature then rather than replacing the existing git functionality for those of us that require signed commits? 🤔

Andarist · 2024-08-26T08:16:03Z

I don't like having options like this but then I also see the value in this approach 😅 cc @emmatown thoughts?

s0 · 2024-08-26T11:03:11Z

I was looking into the rate-limit issue again, and I realize I actually experienced it here: https://github.com/s0/ghcommit/actions/runs/10548632928/job/29222763569 (and can see from #192 that it's the same rate-limit as I faced).

My guess is that we can probably improve things a lot by avoiding the search APIs, which typically have higher rate-limit costs (at least 5x according to https://docs.github.com/en/rest/using-the-rest-api/rate-limits-for-the-rest-api?apiVersion=2022-11-28#calculating-points-for-the-secondary-rate-limit). But beyond that, primary rate limits for the search API are tracked separately to other REST endpoints, so it wouldn't surprise me if there's some conflict / interference with other tokens / repos based on e.g. which runners these are executed on? Given that, at least with the primary rate limits, the GitHub Actions GITHUB_TOKEN has a separate rate-limit per repo.

I imagine that we'll be able to avoid hitting the limits at all if we switch from the search API to the pull request list API, which provides filters for everything you use the search API for: https://docs.github.com/en/rest/pulls/pulls?apiVersion=2022-11-28#list-pull-requests

I could open a separate PR for this change if you like?

s0 · 2024-08-26T11:17:57Z

PR opened :) #393

kazuki-hanai · 2024-09-06T07:52:23Z

Hi @dalmena
I want to use this signed commits feature. When will this PR be merged?

graffhyrum · 2024-09-18T19:23:17Z

+1

s0 · 2024-10-04T05:25:51Z

Any more thoughts on this PR? I could update the PR to make this method opt-in or opt-out and allow the user to choose between the 2 methods?

To allow for signed tags to be created, rather than use the git CLI to push tags, manually push each tag using the GitHub API, which will sign the tag using the built-in GitHub GPG key.

To allow for all commits to be signed, use the GitHub API to push changes.

s0 · 2024-10-19T14:55:54Z

Hmm looks like I've just started experiencing some errors with this test branch: https://github.com/arcanejs/arcanejs/actions/runs/11418428423

Going to convert this back to draft now until I can investigate.

Andarist · 2024-10-21T09:53:25Z

I could consider making this an opt-in - once you resolve the issues you found

philtremblay · 2024-10-29T20:21:00Z

+1 for this feature. I would opt-in to it

s0 · 2024-11-02T11:08:18Z

Issue resolved in changesets/ghcommit#23, so latest update fixes things. Converting this to be opt-in now

Change this to a minor version bump, with a new feature that allows for using the GitHub API to create tags and commits.

s0 · 2024-11-02T11:31:47Z

Okay comments addressed, this is now opt-in via the input commitUsingApi

s0 · 2025-04-12T13:41:21Z

@Andarist okay, comments addressed.

Have tested using this functionality against my fork here:

Without setting commitUsingApi:
- https://github.com/arcanejs/arcanejs/actions/runs/14420145003
- Resulting Commit (unsigned): arcanejs/arcanejs@280d7fc
With commitUsingApi: true:
- https://github.com/arcanejs/arcanejs/actions/runs/14420074107
- Resulting Commit (signed): arcanejs/arcanejs@d90ea56
- Pushed Tag (signed): https://github.com/arcanejs/arcanejs/releases/tag/%40arcanejs%2Freact-toolkit%400.9.1

s0 · 2025-04-15T04:30:11Z

@Andarist any update? This has been open so long now other contributors are starting to duplicate the effort (see #459)

Andarist · 2025-04-21T08:37:10Z

package.json

    "@changesets/read": "^0.5.3",
    "@manypkg/get-packages": "^1.1.3",
    "@octokit/plugin-throttling": "^5.2.1",
+    "@s0/ghcommit": "^1.2.1",


I could swear I commented on this last time - but it seems I have not 😅 I think we need to inline this package here so we have full control over this code.

Hmm, are you sure? It would involve quite a lot of code, and also include having to add things like codegen for GraphQL queries (or a reduction of Type Safety). There are also a bunch of integration tests that would significantly complicate the CI for this repo:

See:

https://github.com/s0/ghcommit/blob/main/src/github/codegen.ts

https://github.com/s0/ghcommit/blob/main/src/github/graphql/queries.ts

https://github.com/s0/ghcommit/tree/main/src/test/integration

Seems like a lot of stuff to inline for something that has quite a nice abstraction boundary & test coverage.

I'm also pretty responsive to PRs and maintenance etc... but if you insist on ownership then perhaps we should consider transferring ownership of the repository & NPM package instead? (which I'm happy to do).

Or perhaps you could also just fork this repo, and publish to a different NPM package too and use that?

I'm usually very keen on using external packages - but this is a crucial piece of software that interacts with tokens and whatnot. So having a dependency on a package that is not well-established in the ecosystem makes me uneasy here.

I certainly don't really want to maintain it... 😅 but it feels like a necessity in this case.

perhaps we should consider transferring ownership of the repository & NPM package instead? (which I'm happy to do).

That sounds like the easiest option if you are willing to do that 👍

Yeah happy to do that. Would you be happy for me to continue to maintain it under the changesets org?

Invite sent: https://github.com/s0/ghcommit/invitations (you will need to accept before I can make you Admin I think)

Hmm looks like I can only add you as a collaborator and can't make you an Admin (probably as it's a user repo not org repo) 🙃. So I think adding me to the org is the only way, can deffo be temporary though.

Repo transferred, I'll work on re-publishing to new NPM package later on this week.

Repo has been updated and ready to publish to @changests/ghcommit. Could you set up an NPM_TOKEN secret for the repo, so that CI can start publishing: https://github.com/changesets/ghcommit/actions/runs/14663639483/job/41153344520

You may need to manually build & publish an older version (i.e. the previous one) to be able to create a token with the appropriate scopes, and so that the build & publish workflow succeeds. (Definitely avoid publishing main/ the current version manually though, as it would mean provenance won't be included and we'll need to bump the version again)

Andarist · 2025-05-02T10:26:43Z

@s0 I pushed out a slight refactor here, could you double-check if this makes sense to you?

s0 · 2025-05-02T11:15:44Z

@s0 I pushed out a slight refactor here, could you double-check if this makes sense to you?

@Andarist LGTM

dpnova · 2025-05-03T23:59:29Z

I just tried to use changesets/action@main to test this out but got this error:

Error: File not found: '/home/runner/work/_actions/changesets/action/main/dist/index.js'

I realise this was only merged yesterday and I came looking for it today, so happy to help with any testing 👍

s0 · 2025-05-04T09:37:35Z

I just tried to use changesets/action@main to test this out but got this error:

Error: File not found: '/home/runner/work/_actions/changesets/action/main/dist/index.js'

I realise this was only merged yesterday and I came looking for it today, so happy to help with any testing 👍

@dpnova you need to wait until #463 is merged and the new version released, then you will be able to use it by referring to specific tags. You can't use main directly

dpnova · 2025-05-05T08:47:38Z

Thanks @s0

Side note, you have a very enviable job 😁

* Remove unused variable (changesets#460) * unused variable * double negative var * Revert "double negative var" This reverts commit 13cd17e. * Introduce an option to use the GitHub API to commit changes, for GPG (changesets#391) * Create tags on GitHub using API To allow for signed tags to be created, rather than use the git CLI to push tags, manually push each tag using the GitHub API, which will sign the tag using the built-in GitHub GPG key. * Use ghcommit to push changes To allow for all commits to be signed, use the GitHub API to push changes. * Add changeset version * Allow tag publish to fail, assume it was manually published * Add to changeset * Update @s0/ghcommit * Update ghcommit to fix missing ref bug * Make using GitHub API Optional Change this to a minor version bump, with a new feature that allows for using the GitHub API to create tags and commits. * Use a strategy pattern for GitHub API vs CLI usage * refactor git interactions * fix mock thing * usechangesets pkg * switch to `commitMode` --------- Co-authored-by: Mateusz Burzyński <mateuszburzynski@gmail.com> * Version Packages (changesets#463) Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> --------- Co-authored-by: Changwan <47740690+WooWan@users.noreply.github.com> Co-authored-by: Sam Lanning <sam@samlanning.com> Co-authored-by: Mateusz Burzyński <mateuszburzynski@gmail.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

adbutterfield · 2025-05-07T08:38:00Z

(I hate to comment on a closed issue) but I just tried this out, and it doesn't work for us. I didn't dig into it yet, but I get an error:

Error: GraphqlResponseError: Request failed due to following response errors:

Repository rule violations found

Secret detected in content.

It's coming from some kind of restriction set on our GitHub organization at my company. I don't know exactly what it's detecting, or if it's a false positive or something. But I'm guessing this could be a problem for other people as well.

Maybe @s0 might have an idea what might trigger this? If not I can look into it later and open a new issue when I have something figured out. 🙇

s0 · 2025-05-07T09:09:26Z

(I hate to comment on a closed issue) but I just tried this out, and it doesn't work for us. I didn't dig into it yet, but I get an error:

Error: GraphqlResponseError: Request failed due to following response errors:

Repository rule violations found

Secret detected in content.

It's coming from some kind of restriction set on our GitHub organization at my company. I don't know exactly what it's detecting, or if it's a false positive or something. But I'm guessing this could be a problem for other people as well.

Maybe @s0 might have an idea what might trigger this? If not I can look into it later and open a new issue when I have something figured out. 🙇

Hmm sounds like it's including some files / content that weren't being added before. Does your CI workflow create files with secret values within your repository? And are they correctly excluded in .gitignore? I haven't tested the functionality with multiple gitignore files, so that may also be related if your repo has that?

Edit: actually was this working before for you at all with the previous method, or it was blocking it for not being signed? My guess is that this maybe wasn't ever working for you, and adding the relevant file to .gitignore will probably fix things.

adbutterfield · 2025-05-08T00:02:48Z

Does your CI workflow create files with secret values within your repository?

Nope. It just builds the packages, and then runs this action.

was this working before for you at all with the previous method

It is working. We want to completely block unsigned commits, but currently don't for the repos that use changesets/action. 🙇

adbutterfield · 2025-05-12T02:21:28Z

@s0 I just tried v1.5.2 and it's working great now! Thanks!!!

s0 changed the title ~~Switch to using the GitHub API to commit changes, so all commits and tags are signed~~ Switch to using the GitHub API to commit changes, for GPG Aug 25, 2024

s0 mentioned this pull request Aug 26, 2024

Switch to pull request list API instead of search #393

Merged

dalmena approved these changes Aug 27, 2024

View reviewed changes

begonaalvarezd mentioned this pull request Aug 28, 2024

Fix SDK "Version Packages" PR uses unsigned commits iotaledger/iota#2043

Open

s0 added 6 commits October 19, 2024 15:48

Create tags on GitHub using API

4a94b8e

To allow for signed tags to be created, rather than use the git CLI to push tags, manually push each tag using the GitHub API, which will sign the tag using the built-in GitHub GPG key.

Use ghcommit to push changes

9ae1eea

To allow for all commits to be signed, use the GitHub API to push changes.

Add changeset version

35f08a8

Allow tag publish to fail, assume it was manually published

72e31f0

Add to changeset

a8a756b

Update @s0/ghcommit

50b4be2

s0 force-pushed the github-api-tag-push branch from a369cc5 to 50b4be2 Compare October 19, 2024 14:49

s0 marked this pull request as draft October 19, 2024 14:55

Update ghcommit to fix missing ref bug

67a10a1

Make using GitHub API Optional

893ba16

Change this to a minor version bump, with a new feature that allows for using the GitHub API to create tags and commits.

s0 force-pushed the github-api-tag-push branch from 06a6067 to 893ba16 Compare November 2, 2024 11:30

s0 marked this pull request as ready for review November 2, 2024 11:31

Use a strategy pattern for GitHub API vs CLI usage

379e12c

This was referenced Apr 12, 2025

Use & test new version of changeset action arcanejs/arcanejs#94

Merged

Disable & Test commitUsingApi arcanejs/arcanejs#96

Merged

adbutterfield mentioned this pull request Apr 14, 2025

feat: allow using graphql API to get signed commits #459

Closed

Andarist reviewed Apr 21, 2025

View reviewed changes

Andarist added 2 commits May 2, 2025 12:23

refactor git interactions

0c32ff7

Merge remote-tracking branch 'origin/main' into github-api-tag-push

449ec02

Andarist added 2 commits May 2, 2025 12:28

fix mock thing

8bd1dac

usechangesets pkg

0105bde

switch to commitMode

313b693

Andarist merged commit 207dc3d into changesets:main May 2, 2025
1 check passed

github-actions bot mentioned this pull request May 2, 2025

Version Packages #463

Merged

HiDeoo mentioned this pull request May 6, 2025

chore(deps): update changesets/action action to v1.5.0 withastro/starlight#3179

Merged

1 task

h3rmanj mentioned this pull request May 7, 2025

Cannot read properties of null (reading 'length') #469

Closed

This was referenced May 8, 2025

Tags created by changeset tag don't trigger workflows with on.push.tags changesets/changesets#1545

Open

Update README.md #472

Open

This was referenced May 21, 2025

v1.5.0 in default commitMode is breaking on tag push #465

Open

fix: tags in git-cli commit mode #500

Open

Introduce an option to use the GitHub API to commit changes, for GPG #391

Introduce an option to use the GitHub API to commit changes, for GPG #391

Uh oh!

Conversation

s0 commented Aug 25, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

changeset-bot bot commented Aug 25, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🦋 Changeset detected

Uh oh!

Andarist commented Aug 25, 2024

Uh oh!

s0 commented Aug 25, 2024

Uh oh!

Andarist commented Aug 26, 2024

Uh oh!

s0 commented Aug 26, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

s0 commented Aug 26, 2024

Uh oh!

kazuki-hanai commented Sep 6, 2024

Uh oh!

graffhyrum commented Sep 18, 2024

Uh oh!

s0 commented Oct 4, 2024

Uh oh!

s0 commented Oct 19, 2024

Uh oh!

Andarist commented Oct 21, 2024

Uh oh!

philtremblay commented Oct 29, 2024

Uh oh!

s0 commented Nov 2, 2024

Uh oh!

s0 commented Nov 2, 2024

Uh oh!

s0 commented Apr 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

s0 commented Apr 15, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Andarist commented May 2, 2025

Uh oh!

s0 commented May 2, 2025

Uh oh!

Uh oh!

dpnova commented May 3, 2025

Uh oh!

s0 commented May 4, 2025

Uh oh!

dpnova commented May 5, 2025

Uh oh!

adbutterfield commented May 7, 2025

Uh oh!

s0 commented May 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

adbutterfield commented May 8, 2025

s0 commented Aug 25, 2024 •

edited

Loading

changeset-bot bot commented Aug 25, 2024 •

edited

Loading

s0 commented Aug 26, 2024 •

edited

Loading

s0 commented Apr 12, 2025 •

edited

Loading

s0 commented May 7, 2025 •

edited

Loading