-
Notifications
You must be signed in to change notification settings - Fork 84
feat: Nvidia GPU Direct Storage Support for reading RNTuple #1426
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
fstrug
wants to merge
30
commits into
scikit-hep:main
Choose a base branch
from
fstrug:main
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
30 commits
Select commit
Hold shift + click to select a range
6ab3dc5
Add nvidia GDS support for RNTuple reading.
fstrug e1e2184
Merge branch 'scikit-hep:main' into main
fstrug dc63aa2
Merge branch 'scikit-hep:main' into main
fstrug 1daaeb4
Integrate RNTuple GDS functionallity across behaviors/RNTuple.py and …
fstrug e81912c
style: pre-commit fixes
pre-commit-ci[bot] 8b030a1
Add support for LZ4 decompression. Update some RNTuple tests to verif…
fstrug 55ff9f4
Resolve merge conflicts
fstrug 3c4dd08
Parametetrized more RNTuple pytests. Added support for reading surpre…
fstrug ceac95d
Remove unnecessary imports. GDS dependencies imported through . Updat…
fstrug 84c7929
Merge branch 'main' into main
fstrug bcf6a05
style: pre-commit fixes
pre-commit-ci[bot] 6eda90a
More tests updated.
fstrug 0d206c7
style: pre-commit fixes
pre-commit-ci[bot] a1c5fa6
Stashing changes
fstrug 55b7925
style: pre-commit fixes
pre-commit-ci[bot] 546195e
Fixed bug causing repeated deserialization operations on a column of …
fstrug eb15117
Added initial implementation for interface to kvikio.CuFile.
fstrug 333397d
style: pre-commit fixes
pre-commit-ci[bot] dd02c45
Merge pull request #1 from scikit-hep/main
fstrug 8fb1921
style: pre-commit fixes
pre-commit-ci[bot] 0d9e970
Added doc strings. Code cleanup.
fstrug 41755b3
style: pre-commit fixes
pre-commit-ci[bot] 3fd629f
Fixed linter bugs
fstrug 7fc0520
Merge pull request #2 from scikit-hep/main
fstrug 28e6ae2
Skip GDS tests if no available CUDA driver.
fstrug ac31628
GDS tests should only run on supported OS with available cuda driver.
fstrug 753ac69
Linting
fstrug 4bc0c94
Update pyproject.toml
fstrug e6c0163
Update pyproject.toml
fstrug 4b3c46e
Update pyproject.toml
fstrug File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -29,6 +29,7 @@ test = [ | |
"rangehttpserver", | ||
"requests", | ||
"s3fs", | ||
'kvikio-cu12>=25.02.01; platform_system == "Linux" and python_version >= "3.10"', | ||
{include-group = "test-core"} | ||
] | ||
test-core = [ | ||
|
@@ -76,7 +77,7 @@ dependencies = [ | |
"awkward>=2.4.6", | ||
"cramjam>=2.5.0", | ||
"xxhash", | ||
"numpy", | ||
"numpy < 2.3", | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @fstrug - no need to pin it now. The CI should pick up the latest awkward release that fixes the issue. |
||
"fsspec", | ||
"packaging", | ||
"typing_extensions>=4.1.0; python_version < '3.11'" | ||
|
@@ -91,6 +92,12 @@ readme = "README.md" | |
requires-python = ">=3.9" | ||
|
||
[project.optional-dependencies] | ||
GDS_cu11 = [ | ||
"kvikio-cu11>=25.02.01" | ||
] | ||
GDS_cu12 = [ | ||
"kvikio-cu12>=25.02.01" | ||
] | ||
http = ["aiohttp"] | ||
s3 = ["s3fs"] | ||
xrootd = ["fsspec-xrootd>=0.5.0"] | ||
|
Large diffs are not rendered by default.
Oops, something went wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe it makes sense to add a
test-gpu
group?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yes, then it's clear what is failing (if something is failing :-)