[Indexer] Support 2-day retention for objects_history table #19318
Conversation
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎
3 Skipped Deployments
|
lxfind
force-pushed
the
indexer-add-table-specific-prune-config
branch
from
0220849 to
41643aa
Compare
lxfind
force-pushed
the
indexer-add-table-specific-prune-config
branch
from
41643aa to
f7d5b68
Compare
| lazy_static! { | ||
| static ref DEFAULT_EPOCHS_OVERRIDES: HashMap<&'static str, u64> = { | ||
| let mut m = HashMap::new(); | ||
| m.insert("objects_history", 2); | ||
| m | ||
| }; | ||
| } | ||
|
|
There was a problem hiding this comment.
use LazyCell instead of lazy_static
There was a problem hiding this comment.
Could we avoid the global and instead pass this config info through?
There was a problem hiding this comment.
We do that too in this PR. Here we provide a few defaults overrides.
I think if we could change our config strategy to a file instead of command line args, it would be a lot easier to move this fully to configs.
|
Is this still safe to do without a watermark table to track the available ranges for each table group? |
Good question. @wlmyng If we have a different pruning retention for objects_history table than other tables, would that break anything? |
lxfind
force-pushed
the
indexer-add-table-specific-prune-config
branch
from
f7d5b68 to
1066c19
Compare
lxfind
changed the title
lxfind
force-pushed
the
indexer-add-table-specific-prune-config
branch
from
1066c19 to
3939548
Compare
lxfind
force-pushed
the
indexer-add-table-specific-prune-config
branch
from
3939548 to
d506e7d
Compare
4 participants
Description
Add support to override epoch retention for individual tables. This can be passed from the command line.
It only supports partitioned table today. For non-partitioned tables it will take some wiring to make it work and not urgent. We need this today mainly to prune the objects_history table more aggressively.
Test plan
CI
Release notes
Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required.
For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates.