-
Notifications
You must be signed in to change notification settings - Fork 0
Open
Labels
enhancementNew feature or requestNew feature or request
Description
The original plan for blob storage was to keep things simple, and persist the blobs in a sequence of:
* Blob ID / Value Checksum (32-bytes)
* Reference Count (4-bytes)
* Value Length (8-bytes)
* Value (Length determined by Value Length)
* Metadata Checksum (4-bytes)
While simple, this format does not support lazy-loading efficiently. Instead, we will decouple the blob metadata as a fixed-length record from the unstructured dynamic value data. The new fixed-length metadata format is:
* Blob ID / Value Checksum (32-bytes)
* Reference Count (4-bytes)
* Value Length (8-bytes)
* Value Offset (8-bytes)
* Metadata Checksum (4-bytes)
With this change, value data is stored separately, after the fixed-length regions of the file. By adopting this revised format, we can sort the fixed-length metadata records by their Blob ID, and provide binary search capability for efficient lazy-loading.
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestNew feature or request