Shared Tree: In-memory format and API changes for persisted schema metadata #24759

TommyBrosman · 2025-06-03T19:27:33Z

Adds in-memory format and API changes for the persisted schema metadata feature.

Changes

Add metadata fields to Tree*StoredSchema types
- Node
- Field
Add fields to in-memory format for persisted node/field schema metadata
- Expose JsonCompatibleReadonlyObject as an alpha API so it can be used for the view schema format
SchemaFactory changes
- Introduce or update alpha types in places where persisted metadata will be passed: NodeSchemaOptionsAlpha, FieldPropsAlpha, FieldSchemaAlpha, FieldSchemaMetadataAlpha
- Update internal code to use the alpha types
- Updated node APIs in SchemaFactoryAlpha to pass persistedMetadata associated with nodes
- Updated field APIs in SchemaFactoryAlpha to take new types for specifying persistedMetadata associated with fields
- Test coverage in schemaFactory.spec.ts for the new node schema and field schema APIs
simple-tree changes

Notes for reviewers

Todo

Merge with main
Remove redundant field factory methods
Revert files that belong in the FormatV2 PR
Update SchemaFactoryAlpha to use the changes in Add non static overrides for SchemaFactoryAlpha #24801

- Added in-memory schema format v2. - Implemented the schema format v2 codec. - Updated test utils that previously assumed there was only one format. - Added snapshots for many of the v2 tests.

…ion as a parameter. - Fixed naming of schema index formats. - Changed schemaChangeFormat to be schema format-agnostic.

…formats. - Moved SchemaCodecVersion.

Co-authored-by: Craig Macomber (Microsoft) <42876482+CraigMacomber@users.noreply.github.com>

… we want something more intuitive (e.g., minimum client version). - Split the encode method for TreeNodeStoredSchema.

…luidFramework into metadata-schema-3

…cVersion. - Changed schema-stored/FormatV2 to only define the new elements. - Updated all instantiations of TreeFieldStoredSchema so that persistedMetadata is specified. - HACK: importCompressed needs to handle both schema formats. A few of the roundtrip JSON tests are failing.

Co-authored-by: Alex Villarreal <716334+alexvy86@users.noreply.github.com>

- Made importCompressed always use a v1 schema when encoding.

- Added better test data to one of the codec tests for the v2 schema format.

…added in another PR. - Updated a comment.

- Added tests calling schemaFactoryAlpha.

packages/dds/tree/api-report/tree.alpha.api.md

packages/dds/tree/src/core/schema-stored/schema.ts

CraigMacomber · 2025-06-10T23:56:55Z

Including "Docs and PR description" in the list of changes in the PR description seems odd.

Generally, though, the changes are looking pretty good.

- Fixed an API annotation. - Lint.

packages/dds/tree/src/simple-tree/api/schemaFactory.ts

packages/dds/tree/src/simple-tree/node-kinds/array/arrayNode.ts

Josmithr

Approving for API changes. Would probably be good to get another tree signoff as well for the changes in the codec layers.

Co-authored-by: Joshua Smithrud <54606601+Josmithr@users.noreply.github.com>

…FluidFramework into metadata-in-memory

TommyBrosman · 2025-06-11T19:01:00Z

Including "Docs and PR description" in the list of changes in the PR description seems odd.

Generally, though, the changes are looking pretty good.

Yeah, I accidentally kept it when I transplanted my checklist. Removed.

packages/dds/tree/src/core/schema-stored/schema.ts

packages/dds/tree/src/simple-tree/schemaTypes.ts

- Updated a doc comment.

github-actions · 2025-06-12T17:04:04Z

🔗 No broken links found! ✅

Your attention to detail is admirable.

linkcheck output


> fluid-framework-docs-site@0.0.0 ci:check-links /home/runner/work/FluidFramework/FluidFramework/docs
> start-server-and-test "npm run serve -- --no-open" 3000 check-links

1: starting server using command "npm run serve -- --no-open"
and when url "[ 'http://127.0.0.1:3000' ]" is responding with HTTP status code 200
running tests using command "npm run check-links"


> fluid-framework-docs-site@0.0.0 serve
> docusaurus serve --no-open

[SUCCESS] Serving "build" directory at: http://localhost:3000/

> fluid-framework-docs-site@0.0.0 check-links
> linkcheck http://localhost:3000 --skip-file skipped-urls.txt

Crawling...

Stats:
  225443 links
    1710 destination URLs
    1941 URLs ignored
       0 warnings
       0 errors

TommyBrosman and others added 30 commits May 12, 2025 14:59

Initial change.

bd8406a

- Added in-memory schema format v2. - Implemented the schema format v2 codec. - Updated test utils that previously assumed there was only one format. - Added snapshots for many of the v2 tests.

persistedMetadata is now backed by an object instead of a string.

3165fe3

Updated codec.spec.ts to cover FormatV2.

9709ede

Removed extraneous whitespace change.

ba058e1

- Changed TreeNodeStoredSchema implementations to take the write vers…

b819742

…ion as a parameter. - Fixed naming of schema index formats. - Changed schemaChangeFormat to be schema format-agnostic.

- Changed the stored schema implementation to handle multiple schema …

0b5c842

…formats. - Moved SchemaCodecVersion.

Merge branch 'main' into metadata-schema-3

ecdca7a

Apply suggestions from code review

1117c2c

Co-authored-by: Craig Macomber (Microsoft) <42876482+CraigMacomber@users.noreply.github.com>

- Switched to using the SchemaCodecVersion in public APIs. Eventually…

94a5c39

… we want something more intuitive (e.g., minimum client version). - Split the encode method for TreeNodeStoredSchema.

Merge branch 'metadata-schema-3' of https://github.com/TommyBrosman/F…

302cb21

…luidFramework into metadata-schema-3

Minor: reverted changes to FluidClientVersion utils.

3275ee0

Switched back to using min client version for importCompressed.

7c48b9d

Minor: reverted accidental change.

a2f4195

Apply suggestions from code review

4786dcc

Co-authored-by: Alex Villarreal <716334+alexvy86@users.noreply.github.com>

Made persisted metadata schema field naming consistent.

6fa8836

- Created a separate storedSchemaDecodeDispatcher for v2 schemas.

e4064c7

- Made importCompressed always use a v1 schema when encoding.

- Refactored the node kind dispatch.

ab1b882

- Added better test data to one of the codec tests for the v2 schema format.

Updated snapshots.

1027daa

- Removed minimum client version changes. Equivalent changes will be …

e527eba

…added in another PR. - Updated a comment.

Removed more minimumClientVersion glue.

3eae1f6

Updated API files.

98e1b76

Rename: SchemaCodecVersion -> SchemaVersion.

ecc77c9

Merge branch 'main' into metadata-schema-3

37e900f

Removed top-level persisted metadata.

737633b

Fixed comments.

4bb136c

Exposed persistedMetadata as a JsonCompatibleReadOnlyObject.

0bcb90f

Changeset.

a675338

Wired up toStoredSchema. Still needs tests.

1d1ef5f

- Updated an old snapshot that didn't include the metadata field.

b9898e8

- Added tests calling schemaFactoryAlpha.

TommyBrosman mentioned this pull request Jun 10, 2025

Shared Tree: Schema FormatV2 and codecs #24812

Merged

2 tasks

TommyBrosman added 3 commits June 10, 2025 13:01

Merge branch 'main' into metadata-in-memory

03d3bc4

Updated API Extractor output.

002a80c

Changed SchemaFactoryAlpha calls to use instance methods.

e3443bd

CraigMacomber reviewed Jun 10, 2025

View reviewed changes

packages/dds/tree/api-report/tree.alpha.api.md Show resolved Hide resolved

CraigMacomber reviewed Jun 10, 2025

View reviewed changes

packages/dds/tree/src/core/schema-stored/schema.ts Outdated Show resolved Hide resolved

TommyBrosman added 3 commits June 10, 2025 19:26

- Cleaned up the type for in-memory persistedMetadata.

72e32ab

- Fixed an API annotation. - Lint.

Merge branch 'main' into metadata-in-memory

9d86b72

Regenerated API Extractor output.

f4b258a

Josmithr reviewed Jun 11, 2025

View reviewed changes

packages/dds/tree/src/simple-tree/api/schemaFactory.ts Outdated Show resolved Hide resolved

Josmithr reviewed Jun 11, 2025

View reviewed changes

packages/dds/tree/src/simple-tree/node-kinds/array/arrayNode.ts Show resolved Hide resolved

Josmithr approved these changes Jun 11, 2025

View reviewed changes

TommyBrosman and others added 4 commits June 11, 2025 11:17

Update packages/dds/tree/src/simple-tree/api/schemaFactory.ts

a743d39

Co-authored-by: Joshua Smithrud <54606601+Josmithr@users.noreply.github.com>

Docs.

01d5eae

Merge branch 'metadata-in-memory' of https://github.com/TommyBrosman/…

58bbeb7

…FluidFramework into metadata-in-memory

Merge branch 'main' into metadata-in-memory

c4e29c8

CraigMacomber reviewed Jun 11, 2025

View reviewed changes

packages/dds/tree/src/core/schema-stored/schema.ts Outdated Show resolved Hide resolved

CraigMacomber reviewed Jun 11, 2025

View reviewed changes

packages/dds/tree/src/simple-tree/schemaTypes.ts Show resolved Hide resolved

CraigMacomber reviewed Jun 11, 2025

View reviewed changes

packages/dds/tree/src/simple-tree/schemaTypes.ts Outdated Show resolved Hide resolved

CraigMacomber reviewed Jun 11, 2025

View reviewed changes

packages/dds/tree/src/simple-tree/schemaTypes.ts Outdated Show resolved Hide resolved

TommyBrosman added 2 commits June 11, 2025 13:38

- Cleaned up comparators and added tests.

e2f350c

- Updated a doc comment.

Merge branch 'main' into metadata-in-memory

ea00cc9

TommyBrosman mentioned this pull request Jun 11, 2025

Shared Tree: Persisted Schema Format v2 with persisted metadata support #24590

Closed

6 tasks

TommyBrosman enabled auto-merge (squash) June 11, 2025 22:17

Merge branch 'main' into metadata-in-memory

a6348b4

Abe27342 approved these changes Jun 13, 2025

View reviewed changes

TommyBrosman merged commit dc14950 into microsoft:main Jun 13, 2025
35 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Shared Tree: In-memory format and API changes for persisted schema metadata #24759

Shared Tree: In-memory format and API changes for persisted schema metadata #24759

Uh oh!

TommyBrosman commented Jun 3, 2025 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

CraigMacomber commented Jun 10, 2025

Uh oh!

Uh oh!

Uh oh!

Josmithr left a comment

Uh oh!

TommyBrosman commented Jun 11, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Jun 12, 2025

Uh oh!

Uh oh!

Uh oh!

Shared Tree: In-memory format and API changes for persisted schema metadata #24759

Shared Tree: In-memory format and API changes for persisted schema metadata #24759

Uh oh!

Conversation

TommyBrosman commented Jun 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

Notes for reviewers

Todo

Uh oh!

Uh oh!

Uh oh!

CraigMacomber commented Jun 10, 2025

Uh oh!

Uh oh!

Uh oh!

Josmithr left a comment

Choose a reason for hiding this comment

Uh oh!

TommyBrosman commented Jun 11, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Jun 12, 2025

linkcheck output

Uh oh!

Uh oh!

Uh oh!

TommyBrosman commented Jun 3, 2025 •

edited

Loading