Skip to content

Commit 08a03c7

Browse files
authored
ethdb/pebble: adjust the number of memory tables (#31970)
This pull request adjusts the number of allowed memory tables in Pebble. Pebble allows configuring an arbitrary number of memory tables to hold unflushed data. When the current memtable becomes full, it is scheduled for flushing, and a new memtable is allocated to accept subsequent writes. However, if too many memtables accumulate and are waiting to be flushed, subsequent writes will stall. Originally, only two memtables were configured, each with a size of 512 MB for Ethereum mainnet. While this setup works well under normal conditions, it becomes problematic under heavy write loads. In such scenarios, flushing is only triggered when more than 512 MB of data is pending, which may not be responsive enough. Even worse, if compactions are running concurrently, flushing memtables can become slow due to the heavy IO overhead, leading to write stalls across the system. This pull request tries to mitigate the performance degradation by having more memory tables but with a smaller size. In this case, the pending writes can be flushed more smoothly and responsively.
1 parent c7e6c08 commit 08a03c7

File tree

1 file changed

+6
-3
lines changed

1 file changed

+6
-3
lines changed

ethdb/pebble/pebble.go

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -199,9 +199,12 @@ func New(file string, cache int, handles int, namespace string, readonly bool) (
199199
// Taken from https://github.com/cockroachdb/pebble/blob/master/internal/constants/constants.go
200200
maxMemTableSize := (1<<31)<<(^uint(0)>>63) - 1
201201

202-
// Two memory tables is configured which is identical to leveldb,
203-
// including a frozen memory table and another live one.
204-
memTableLimit := 2
202+
// Four memory tables are configured, each with a default size of 256 MB.
203+
// Having multiple smaller memory tables while keeping the total memory
204+
// limit unchanged allows writes to be flushed more smoothly. This helps
205+
// avoid compaction spikes and mitigates write stalls caused by heavy
206+
// compaction workloads.
207+
memTableLimit := 4
205208
memTableSize := cache * 1024 * 1024 / 2 / memTableLimit
206209

207210
// The memory table size is currently capped at maxMemTableSize-1 due to a

0 commit comments

Comments
 (0)