Pebble upgrade plan

[rjl493456442]

Pebble has released Pebble v2, which is unfortunately incompatible with
the v1 version currently used in go-ethereum.

Upgrading to v2 is certainly a desirable long-term goal. However, before
proceeding with a database version upgrade, several measurements and
evaluations must be performed.

System metrics

There is no major difference in terms of system metrics, one notable change
is pebble-v2 (bench05) has higher IO wait.

Database metrics

The notable difference is Pebble-v2 (bench05) has more aggressive compaction workload.

Overall chain progressing metrics

The node with Pebble-v2 (bench05) is slightly faster, roughly 1h ahead after running for 3 days.

Preliminary conclusion

There is no major performance gain from upgrading Pebble directly. However, Pebble team
is actively working on the Blob notion in pebble, which implements WiscKey idea to separate
value from the LSM tree to minimize read and write amplification. It will be a huge gain once
this feature is shipped and well-tested.

All the following updates regarding Pebble will be posted/tracked in this issue.