-
Notifications
You must be signed in to change notification settings - Fork 1.5k
feat: Allow cancelling of grouping operations which are CPU bound #16196
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
80 commits
Select commit
Hold shift + click to select a range
b18aeaa
feat: support inability to yeild cpu for loop when it's not using Tok…
zhuqi-lucas 2a965aa
Fix fuzz test
zhuqi-lucas 67ca44b
polish code
zhuqi-lucas 82a179d
add comments
zhuqi-lucas da3c2d5
fix corner case when huge data
zhuqi-lucas 6cf3bf0
Also add grouping case
zhuqi-lucas 3251990
Address comments
zhuqi-lucas a8da370
Merge remote-tracking branch 'upstream/main' into issue_16193
zhuqi-lucas 311849d
fmt
zhuqi-lucas 2a2ead9
Move YieldStream into physical-plan crate
alamb 3b69287
Merge remote-tracking branch 'apache/main' into issue_16193
alamb 3ff9252
Use existing RecordBatchStreamAdapter
alamb 5547c3c
Add timeout testing for cancellation
zhuqi-lucas aeac0ef
fmt
zhuqi-lucas 6d56b78
add license
zhuqi-lucas 4ddd1e5
Support sort exec for cancellation
zhuqi-lucas b2ffec7
poc: unified yield exec for leaf node
zhuqi-lucas 4587c3b
polish code phase 1
zhuqi-lucas 098b1ec
Add license
zhuqi-lucas bc65c1a
Fix testing
zhuqi-lucas da58d0b
Support final path
zhuqi-lucas 021fb92
fix test
zhuqi-lucas 5027087
polish code
zhuqi-lucas 8509d0a
fix testing and address suggestions
zhuqi-lucas 97c1bb7
fix
zhuqi-lucas 5c3a14c
remove buffer
zhuqi-lucas 97923b8
address comments
zhuqi-lucas 118f801
fmt
zhuqi-lucas e7a678a
Fix test
zhuqi-lucas 54260aa
fix
zhuqi-lucas cb344af
fix
zhuqi-lucas 89a4e93
fix slt
zhuqi-lucas 896fd59
fix tpch sql
zhuqi-lucas 2887f87
Add flag for yield insert and disable default
zhuqi-lucas 2de4afb
recover testing
zhuqi-lucas 485d55a
fix
zhuqi-lucas e31a997
Update doc
zhuqi-lucas b65ab60
Address comments
zhuqi-lucas 1bf2ad3
fix fmt
zhuqi-lucas 8ab64b3
Support config for yield frequency
zhuqi-lucas c7774a7
add built-in yield support
zhuqi-lucas 7f64f5c
Merge remote-tracking branch 'upstream/main' into issue_16193
zhuqi-lucas 3eb785c
Add LazyMemoryExec built-in Yield
zhuqi-lucas 5737309
Update datafusion/datasource/src/source.rs
zhuqi-lucas 74eb5c2
Update datafusion/core/tests/physical_optimizer/enforce_distribution.rs
zhuqi-lucas 23d7a4b
Update datafusion/physical-optimizer/src/optimizer.rs
zhuqi-lucas e7c7eea
Update datafusion/physical-plan/src/memory.rs
zhuqi-lucas 7df28e5
Update datafusion/physical-plan/src/memory.rs
zhuqi-lucas d16e6e5
Update datafusion/proto/src/physical_plan/mod.rs
zhuqi-lucas f6ccecb
Address comments
zhuqi-lucas 8c74620
Add interleave reproducer
zhuqi-lucas 5e2dde8
remove unused config
zhuqi-lucas aa73a2d
Add join with aggr case
zhuqi-lucas 8d9f091
Merge remote-tracking branch 'upstream/main' into issue_16193
zhuqi-lucas b825ea6
fix clippy
zhuqi-lucas be88233
add reproducer for filter and add workaround in rule
zhuqi-lucas 4018f8f
Add reproducer and comment it
zhuqi-lucas 3b70846
adjust test
zhuqi-lucas 11035e5
Harden tests, add failing tests for TDD, minor code refactors
ozankabak b05cb85
Add remaining test
zhuqi-lucas e136a03
Add sort merge case
zhuqi-lucas 29bc2c7
change rule solution
zhuqi-lucas 187a945
fix test
zhuqi-lucas 3cc4095
add more built-in case
zhuqi-lucas 4614048
fix
zhuqi-lucas 0b8427c
Merge remote-tracking branch 'upstream/main' into issue_16193
zhuqi-lucas 8047534
fix user defined exec
zhuqi-lucas 7f4c93e
Reduce test diff
ozankabak ca7f061
Format imports
ozankabak 5806bdb
Update documentation
ozankabak b45d3dc
Update information schema per docs
ozankabak a21c6ce
Only retain the with_cooperative_yields API for now
ozankabak c820e36
Format imports
ozankabak 697bfae
Remove unnecessary clones
ozankabak 884386e
Merge branch 'apache_main' into issue_16193
ozankabak d3952d9
Fix logical conflict
ozankabak 56361a4
Exercise DRY, refactor common code to a helper function. Use period i…
ozankabak 50959ad
Update datafusion/physical-plan/src/yield_stream.rs
zhuqi-lucas edc018d
Address new comments
zhuqi-lucas ee9bb89
Address more comments.
zhuqi-lucas File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I like that this has a "escape valve" too -- if this mechanism isn't working we can disable the new yields via config