Skip to content

fix(gateway): support suffix range requests #922

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 13 commits into from
May 28, 2025
Merged

fix(gateway): support suffix range requests #922

merged 13 commits into from
May 28, 2025

Conversation

aschmahmann
Copy link
Contributor

@aschmahmann aschmahmann commented May 22, 2025

fixes ipfs/kubo#10808

Well ... the need for this PR is quite embarrassing. I thought we had this covered in gateway conformance but we don't.

@lidel IIRC we discussed this when this code path was implemented and maybe this was just supposed to not support suffix requests and return 200 with everything instead of 206 (as we do for ranges on things like dir-index-html responses), but given we found a user who could actually benefit from it it seemed reasonable to implement. Feel free to take over the PR and change as you wish, but figured I'd at least present an option to help get things moving.

  • Figure out how we want to resolve this
  • Add gateway conformance tests

@aschmahmann aschmahmann requested a review from lidel as a code owner May 22, 2025 01:23
@lidel lidel changed the title fix(gateway): support suffix requests fix(gateway): support suffix range requests May 27, 2025
@lidel
Copy link
Member

lidel commented May 27, 2025

Triage notes:

@lidel lidel requested a review from a team as a code owner May 27, 2025 22:05
@lidel lidel force-pushed the fix/range-suffix branch from 6f9fbe9 to c30b27a Compare May 27, 2025 22:23
@lidel lidel force-pushed the fix/range-suffix branch from e5f8c09 to ee5383d Compare May 27, 2025 23:34
lidel added 2 commits May 28, 2025 01:42
car backend did not support negative ranges, this attempts to fix that
for raw blocks and dag-pb files
@lidel lidel force-pushed the fix/range-suffix branch from cc1dde9 to 6274fc5 Compare May 28, 2025 00:29
@lidel
Copy link
Member

lidel commented May 28, 2025

I will take a look with fresh eyes tomorrow, remaining work is to bubble up and land everything:

Copy link
Member

@lidel lidel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Switched boxo to gateway-conformance v0.8 with explicit test for suffix range-requests
    • Kubo 0.35 fails as expected
    • Kubo with this PR passes, confirming the fix works
  • Merging this, and switching Kubo/Rainbow to boxo from main + gateway-conformance v0.8.

lidel added a commit that referenced this pull request May 28, 2025
The #922 triggered some AI checks:
https://github.com/ipfs/boxo/security/code-scanning/7

I don't think this is a problem irl, due to code in
`unixfs/mod/dagmodifier.go` being related to single block (which have
limits), but adding this extra check there just in case and to avoid
linters bringing this up in the future.
@lidel lidel mentioned this pull request May 28, 2025
//
// [HTTP Byte Range]: https://httpwg.org/specs/rfc9110.html#rfc.section.14.1.2
type ByteRange struct {
From uint64
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💭 This triggered some AI checks – https://github.com/ipfs/boxo/security/code-scanning/7 – seems to be an issue already present in legacy code, unrelated to this PR.

I don't think this is a problem blocking this PR because the gateway range-requests do not mutate UnixFS DAGs.

Still, filled potential fix in #936 to make linter warning go away.

@lidel lidel merged commit f87feb5 into main May 28, 2025
15 checks passed
@lidel lidel deleted the fix/range-suffix branch May 28, 2025 18:29
gammazero pushed a commit that referenced this pull request Jun 3, 2025
The #922 triggered some AI checks:
https://github.com/ipfs/boxo/security/code-scanning/7

I don't think this is a problem irl, due to code in
`unixfs/mod/dagmodifier.go` being related to single block (which have
limits), but adding this extra check there just in case and to avoid
linters bringing this up in the future.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Negative range requests are not implemented correctly
2 participants