Skip to content

feat(transaction): Implement TransactionAction for FastAppendAction #1448

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Jun 18, 2025
Merged

feat(transaction): Implement TransactionAction for FastAppendAction #1448

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
0f4b4fc
Impl TxAction for FastAppendAction, make SnapshotProducer immutable
CTTY Jun 13, 2025
575a4a4
Fix tests
CTTY Jun 16, 2025
a1e193a
Merge branch 'main' into ctty/tx-impl-3
CTTY Jun 16, 2025
b64ecda
I love clippy
CTTY Jun 16, 2025
cc885c6
fmt
CTTY Jun 16, 2025
bc40f65
Merge branch 'main' into ctty/tx-impl-3
CTTY Jun 17, 2025
fa4f6a7
Simplify tx.fast_append API, move helper methods back to snapshot pro…
CTTY Jun 18, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion crates/iceberg/src/io/object_cache.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -266,7 +266,7 @@ mod tests {
let mut writer = ManifestWriterBuilder::new(
self.next_manifest_file(),
Some(current_snapshot.snapshot_id()),
vec![],
None,
current_schema.clone(),
current_partition_spec.as_ref().clone(),
)
Expand Down
4 changes: 2 additions & 2 deletions crates/iceberg/src/scan/mod.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -752,7 +752,7 @@ pub mod tests {
let mut writer = ManifestWriterBuilder::new(
self.next_manifest_file(),
Some(current_snapshot.snapshot_id()),
vec![],
None,
current_schema.clone(),
current_partition_spec.as_ref().clone(),
)
Expand Down Expand Up @@ -964,7 +964,7 @@ pub mod tests {
let mut writer = ManifestWriterBuilder::new(
self.next_manifest_file(),
Some(current_snapshot.snapshot_id()),
vec![],
None,
current_schema.clone(),
current_partition_spec.as_ref().clone(),
)
Expand Down
12 changes: 6 additions & 6 deletions crates/iceberg/src/spec/manifest/mod.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -232,7 +232,7 @@ mod tests {
let mut writer = ManifestWriterBuilder::new(
output_file,
Some(1),
vec![],
None,
metadata.schema.clone(),
metadata.partition_spec.clone(),
)
Expand Down Expand Up @@ -417,7 +417,7 @@ mod tests {
let mut writer = ManifestWriterBuilder::new(
output_file,
Some(2),
vec![],
None,
metadata.schema.clone(),
metadata.partition_spec.clone(),
)
Expand Down Expand Up @@ -514,7 +514,7 @@ mod tests {
let mut writer = ManifestWriterBuilder::new(
output_file,
Some(3),
vec![],
None,
metadata.schema.clone(),
metadata.partition_spec.clone(),
)
Expand Down Expand Up @@ -623,7 +623,7 @@ mod tests {
let mut writer = ManifestWriterBuilder::new(
output_file,
Some(2),
vec![],
None,
metadata.schema.clone(),
metadata.partition_spec.clone(),
)
Expand Down Expand Up @@ -731,7 +731,7 @@ mod tests {
let mut writer = ManifestWriterBuilder::new(
output_file,
Some(2),
vec![],
None,
metadata.schema.clone(),
metadata.partition_spec.clone(),
)
Expand Down Expand Up @@ -1010,7 +1010,7 @@ mod tests {
let mut writer = ManifestWriterBuilder::new(
output_file,
Some(1),
vec![],
None,
metadata.schema.clone(),
metadata.partition_spec.clone(),
)
Expand Down
12 changes: 6 additions & 6 deletions crates/iceberg/src/spec/manifest/writer.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ use crate::{Error, ErrorKind};
pub struct ManifestWriterBuilder {
output: OutputFile,
snapshot_id: Option<i64>,
key_metadata: Vec<u8>,
key_metadata: Option<Vec<u8>>,
schema: SchemaRef,
partition_spec: PartitionSpec,
}
Expand All @@ -50,7 +50,7 @@ impl ManifestWriterBuilder {
pub fn new(
output: OutputFile,
snapshot_id: Option<i64>,
key_metadata: Vec<u8>,
key_metadata: Option<Vec<u8>>,
schema: SchemaRef,
partition_spec: PartitionSpec,
) -> Self {
Expand Down Expand Up @@ -115,7 +115,7 @@ pub struct ManifestWriter {

min_seq_num: Option<i64>,

key_metadata: Vec<u8>,
key_metadata: Option<Vec<u8>>,

manifest_entries: Vec<ManifestEntry>,

Expand All @@ -127,7 +127,7 @@ impl ManifestWriter {
pub(crate) fn new(
output: OutputFile,
snapshot_id: Option<i64>,
key_metadata: Vec<u8>,
key_metadata: Option<Vec<u8>>,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is an API change, would appreciate if we could call it out at PR description and release notes.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry about that! I've updated the PR description.

cc @liurenjie1024 probably related to the release process: Is there a "staging release notes" that we can start putting changes like this into as breaking changes?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm thinking about adding a breaking label to pr so that it standout when we generate release notes.

metadata: ManifestMetadata,
) -> Self {
Self {
Expand Down Expand Up @@ -416,7 +416,7 @@ impl ManifestWriter {
existing_rows_count: Some(self.existing_rows),
deleted_rows_count: Some(self.deleted_rows),
partitions: Some(partition_summary),
key_metadata: Some(self.key_metadata),
key_metadata: self.key_metadata,
})
}
}
Expand Down Expand Up @@ -622,7 +622,7 @@ mod tests {
let mut writer = ManifestWriterBuilder::new(
output_file,
Some(3),
vec![],
None,
metadata.schema.clone(),
metadata.partition_spec.clone(),
)
Expand Down
Loading
Loading