GitHub rate limiting despite default `GITHUB_TOKEN`

[jan-ferdinand]

Hi there, thank you for your work on lychee as well as this github action. ❤

Recently, I started having CI failures due to GitHub's rate limiting, which hadn't happened before with this action.¹ As far as I understand, the lychee binary picks up on and uses the GITHUB_TOKEN environment variable by default, which should prevent rate limiting issues. It's not obvious how any of the recent commits to either this action or to lychee itself would have changed this behavior. Unfortunately, I'm at a bit of a loss: why is my action being rate limited?

Footnotes

1. This is the most recent failure at the time of writing. ↩

[GlenDC]

I see the same issue in my project: https://github.com/plabayo/rama/blob/f96ee2867e6211b7e50a28ef5f6da1d89c775188/.github/workflows/links.yml#L19

this always has worked before, do we need a different API key perhaps that has more bandwidth?

[mre]

Oh, I'm not aware of any breaking changes. We haven't changed the checking algorithm.

Looking at your summary, I noticed that you have 741 links.

# Summary
| Status        | Count |
|---------------|-------|
| 🔍 Total      | 741   |
| ✅ Successful | 288   |
| ⏳ Timeouts   | 0     |
| 🔀 Redirected | 0     |
| 👻 Excluded   | 1     |
| ❓ Unknown    | 0     |
| 🚫 Errors     | 452   |

I checked the pipeline and noticed that you run the link check on every push to master and every PR targeting `master.
So you could be hitting GitHub API rate limits if you have more than two such events per hour.
Not all of these links have to be GitHub links, but they quickly add up!

Another issue appears in your workflow logs:

[WARN] Cache is too old (age: 39d 20h 34m 25s, max age: 1d 0h 0m 0s). Discarding and recreating.

So, in summary, I guess you're facing two problems:

1. GitHub API rate limiting (1,000 requests per hour per repository)
2. Cache not updating when the link check fails

Here's what you could try:

on:
  workflow_dispatch:
  push:
    branches:
      - master
  # TODO: Remove this section:
  # pull_request:
  #  branches:
  #    - master
  schedule:
    - cron: "0 0 * * *"

- name: Restore lychee cache
  id: restore-cache
  uses: actions/cache/restore@v4
  with:
    path: .lycheecache
    key: cache-lychee-${{ github.sha }}
    restore-keys: cache-lychee-

- name: Link Checker
  uses: lycheeverse/lychee-action@v2.3.0
  id: lychee_run
  with:
    args: |
      --cache
      --verbose
      --no-progress
      '.'
  continue-on-error: true

- name: Save lychee cache
  uses: actions/cache/save@v4
  if: always()
  with:
    path: .lycheecache
    key: ${{ steps.restore-cache.outputs.cache-primary-key }}

This solution splits the cache operations into separate restore and save steps and uses if: always() to ensure the cache is saved even when the link check fails. I haven't tried it, but I'd be thankful if you gave it a shot. We can update the docs accordingly if it works, because I guess a lot of people might run into this problem now or in the future as their repos grow.

The part about the workflow triggers at the start of the YAML file isn't strictly necessary. I think the important part is to get the cache fixed.

@GlenDC, looking at a recent run, I can also see that you have a lot of links:

# Summary

| Status        | Count |
|---------------|-------|
| 🔍 Total      | 1434  |
| ✅ Successful | 1309  |
| ⏳ Timeouts   | 0     |
| 🔀 Redirected | 0     |
| 👻 Excluded   | 44    |
| ❓ Unknown    | 0     |
| 🚫 Errors     | 81    |

It looks like only around 300 of the 1434 links are GitHub links, though, and you only run lychee once per day so that's weird.

My hunch is, that you run into a secondary rate limiting problem with GitHub as described in the docs:

In addition to primary rate limits, GitHub enforces secondary rate limits in order to prevent abuse and keep the API available for all users.

You may encounter a secondary rate limit if you:

• Make too many concurrent requests. No more than 100 concurrent requests are allowed. This limit is shared across the REST API and GraphQL API.

So if more than 100 requests to GitHub run concurrently (and it's likely since lychee can run way more requests concurrently), you might be unfortunate enough to be rate-limited.

Can you try to set the max-concurrency lower for a test?
--max-concurrency 20 --max-retries 0 or so.

Also, can you add a cache similar to how @jan-ferdinand does it in their pipeline?

[GlenDC]

Can you try to set the max-concurrency lower for a test?
--max-concurrency 20 --max-retries 0 or so.

Currently I do not set any args myself.
Can you recommend me what exactly to change in https://github.com/plabayo/rama/blob/main/.github/workflows/links.yml please?

[mre]

      - name: Link Checker
        id: lychee
        uses: lycheeverse/lychee-action@v2.4.0
        with:
          args: --base . --verbose --no-progress './**/*.md' './**/*.html' './**/*.rst' --max-concurrency 20 --max-retries 0
          token: ${{ secrets.GITHUB_TOKEN }}

[GlenDC]

Thx. Updated it. Will let you know how it went tomorrow!

[jan-ferdinand]

Thanks for your help! I like the changes you suggest and have incorporated them. They might fix the issue; I'll report back if they do.

From the symptoms I have seen so far, I think it's most likely that github has changed their secondary rate limiting policy somehow.¹² I'm sure that the workflow did not hit the (documented) primary rate limit, as the workflow was triggered by only the schedule for several days in a row. As far as I can see, with the new caching behavior, having on-push and on-PR triggers should be unproblematic.

A small note regarding the suggested change to the workflow file, in particular to the “Save lychee cache” step: the docs discourage if: always(), since that makes it impossible to cancel the workflow during that step. They suggest if: ${{ !cancelled() }} instead.³

Footnotes

1. I conclude this because (1) my workflow file used to work, (2) I have not changed my workflow file recently, (3) lychee-action has no recent changes that might impact github rate limiting, and (4) lychee has no recent changes that might impact github rate limiting. ↩

2. I can't see this reflected in the docs. ↩

3. I'm aware that the docs for the cache/save action suggest always(), but the other documentation seems more authoritative on that front. ↩

[mre]

Aha! Thanks for the note about if: always(). In case this ends up fixing the problem, we should take that into account when updating the docs.

[GlenDC]

Didn't seem to work @mre : https://github.com/plabayo/rama/actions/runs/14934933577

[mre]

I don't know. It looks like you get blocked pretty rapidly, though? Do you use the GitHub token anywhere else? Maybe you'd like to create a new token and try again?