Skip to content

rocksdb: configurable MaxOpenFiles #272

@JayT106

Description

@JayT106

#218 made a change for setup MaxOpenFiles to 4096, which makes the RAM usage could easily go over 64GB when our application doing a snapshot or the genesis state export that requires the DB iterating. We tried to reduce it to 1024 and it turns out we can run it within 30GB.

The setup here

opts.OptimizeLevelStyleCompaction(512 * 1024 * 1024)
made each sst file target size equal to 64MB(if we look at how rocksdb decided the file size), therefore I am not sure to budget 2MB for each opened file is base on which file size?

We should consider the node's capability and decide which number is proper for running the rocksdb instead of a fixed number.

ref:
https://github.com/facebook/rocksdb/wiki/Memory-usage-in-RocksDB

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions