triedb/pathdb: add history index config #32875
Open
+143
−62
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
rjl493456442
commented
Oct 13, 2025
| if !last.full() { | ||
| delete(r.readers, last.id) | ||
| } | ||
| delete(r.readers, last.id) |
There was a problem hiding this comment.
In theory, if the last resolved reader is full, it's unnecessary to reload it for refreshing.
In practice, it's very rare that the last reader is full. In order to avoid passing the block capacity from the top down to the indexReader only for this purpose, this condition is removed.
The correctness shouldn't be affected.
rjl493456442
commented
Oct 13, 2025
| indexBlockRestartLen = 256 // The restart interval length of index block | ||
| historyIndexBatch = 1_000_000 // The number of state history indexes for constructing or deleting as batch | ||
| indexBlockDescSize = 14 // The size of index block descriptor | ||
| indexBlockRestartLen = 256 // The restart interval length of index block |
There was a problem hiding this comment.
While in theory, the restart size can also be configured, it's not moved into the config in this pull request yet
kmk142789
approved these changes
Oct 14, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The state history indexer configuration is introduced in this PR, primarily to allow
setting different index block capacities for different types of history data.
Notably, the index block capacity for existing state history has been reduced from
4K to 2K. This change aims to prevent the worst case where all mutated state entries
must update an almost full index block, causing significant write amplification and
potentially triggering a panic due to Pebble 4 GB batch size limitation.
This parameter adjustment remains backward compatible with the existing index data.
What's more, the database batch for writing the index data is now pre-allocated.
It's observed that database batch repeatedly grows the size of the mega-batch,
causing significant memory allocation pressure. This approach can effectively
mitigate the overhead.