-
Notifications
You must be signed in to change notification settings - Fork 281
feat(transaction): Implement TransactionAction for FastAppendAction #1448
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @CTTY for this pr, generally looks good!
…ducer, remove add_parquet_files
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @CTTY for this pr, LGTM!
@@ -127,7 +127,7 @@ impl ManifestWriter { | |||
pub(crate) fn new( | |||
output: OutputFile, | |||
snapshot_id: Option<i64>, | |||
key_metadata: Vec<u8>, | |||
key_metadata: Option<Vec<u8>>, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is an API change, would appreciate if we could call it out at PR description and release notes.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorry about that! I've updated the PR description.
cc @liurenjie1024 probably related to the release process: Is there a "staging release notes" that we can start putting changes like this into as breaking changes?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm thinking about adding a breaking
label to pr so that it standout when we generate release notes.
|
||
/// Finished building the action and apply it to the transaction. | ||
pub async fn apply(self) -> Result<Transaction> { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is an API change, would appreciate PR description and release notes, even provide a small example for no-op migration.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Updated the PR description accordingly as well
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you!
Which issue does this PR close?
Related Issues:
What changes are included in this PR?
BREAKING CHANGES:
ManifestWriter
now takeskey_metadata: Option<Vec<u8>>
instead ofkey_metadata: Vec<u8>
FastAppendAction::apply
withcommit
as an implementation ofTransactionAction
General:
TransactionAction
forFastAppendAction
SnapshotProduceAction
toSnapshotProducer
SnapshotProduceAction
an stateless helper class that helps with fast_append::commit (as discussed here)Are these changes tested?
Added unit tests