Skip to content

Schema registry changes #3550

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

slinkydeveloper
Copy link
Contributor

Fix #3303

@slinkydeveloper
Copy link
Contributor Author

@slinkydeveloper slinkydeveloper changed the title [WIP] Schema registry changes Schema registry changes Jul 21, 2025
@slinkydeveloper slinkydeveloper marked this pull request as ready for review July 21, 2025 15:55
Copy link
Contributor

@AhmedSoliman AhmedSoliman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like the majority of the changes are mechanical. Just left a couple of nits but overall direction is reasonable.

use crate::{Version, Versioned, identifiers};
use arc_swap::ArcSwapOption;
use restate_serde_util::MapAsVecItem;
use serde_json::Value;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: I'd appreciate if imports are organized according to our convention.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As said in other PRs, I'll be happy to do that with a tool that automates it.

This commit contains the changes described in restatedev#3303

* metadata contains the "implementation" of the schema registry, and the serializable data structure `Schema`.
* `metadata/mod.rs` contains the definition of `Schema` and the implementation of the various schema registry APIs. All the fields are private, and access should be available only through schema registry APIs or `SchemaUpdater`.
* `Schema` now uses the new data types from the old `v2.rs` file. On deserialization, we infer the "active service revisions" index (the logic for that is within `serde_hacks`).
* `v2.rs` conversions was moved to `metadata/serde_hacks.rs`, the new data model is directly in `metadata/mod.rs`. Inside it, I moved the old data types *Schemas, which should not be used anymore.
* `SchemaUpdater` was moved inside `metadata`, and deals with the new `Schema` data structure. The behavior remains untouched.

Everything else is more or less copying stuff around and renaming.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Re-design the Schema serialization and code organization
2 participants