Skip to content

feat(catalog): Implement MemoryCatalog's table update/ commit path #1290

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 0 commits into from

Conversation

DerGut
Copy link
Contributor

@DerGut DerGut commented May 3, 2025

➡️ Moved to #1405

Which issue does this PR close?

Relates to #700

This is a first attempt at implementing the commit path for the MemoryCatalog. Happy about any feedback!

What changes are included in this PR?

This PR implements the MemoryCatalog::update_table method. It also adds a MetadataLocation struct to avoid some re-validations of the location format.
I'm very open to removing the MetadataLocation or moving it's addition to a separate PR.

Are these changes tested?

So far, I've included unit tests to verify

  • whether a table update that was committed via the MemoryCatalog can be observed in the updated table
  • that in the case of two competing updates, the later commit fails if it has conflicting requirements
  • that an update fails if the table isn't known to the catalog
  • parsing logic of the MetadataLocation

@DerGut DerGut marked this pull request as ready for review May 4, 2025 14:20
@DerGut DerGut changed the title Implement MemoryCatalog's table update/ commit path feat(catalog) Implement MemoryCatalog's table update/ commit path May 5, 2025
@DerGut DerGut changed the title feat(catalog) Implement MemoryCatalog's table update/ commit path feat(catalog): Implement MemoryCatalog's table update/ commit path May 5, 2025
@jonathanc-n
Copy link
Contributor

Thanks for this pr! I do not think we should move forward with this yet before our transaction API is finished. This is a nice start though and can probably be added later and tested alongside the rest of the transaction functionality.

@jonathanc-n
Copy link
Contributor

What do you think? @liurenjie1024 @Fokko

@DerGut
Copy link
Contributor Author

DerGut commented May 12, 2025

Works for me, happy to iterate on it later if needed 👍

Is there a ticket for pending transaction API improvements? I couldn't find one.

@jonathanc-n
Copy link
Contributor

#700 This holds most of the improvements for updates, deletes, etc. You can possibly look into the different pull requests that are contributing to this.

@DerGut
Copy link
Contributor Author

DerGut commented May 22, 2025

Closed for now. I can recreate the PR once we're there! and recreated in #1405 because I had to rebase onto a new branch 🙈

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants