disable periodic statistics printout

[tve]

Is there a way to disable the periodic stats prinout? It just pollutes the logs. Or perhaps to change the interval to something rather long?

[Barre]

I feel that stats are pretty useful as a monitoring tool because you can see the actual data (in/out) rate that goes through ZeroFS, which helps understanding when are buffers flushed.

It's been a few reports I've been getting in this direction, so perhaps this should indeed be disabled by default, or just exposed through a subcommand).

RUST_LOG=info,zerofs::cli::server=warn ./zerofs-linux-amd64-pgo run -c zerofs.toml should give you something less chatty.

[tve]

I'm sure the stats are useful to you, but I haven't been able to get much info out. It's not that I can't but they're just too frequent to leave running all the time for me and they make it difficult to see actual errors...

More than stats about ZeroFS I'd be interested in stats about the back-end. E.g. requests and KB read/write from/to the bucket, as well as perhaps some latency stats, like min/median/max time to complete operations.

Something that I think would also be very interesting from a cost point of view is bandwidth reduction/amplification. E.g. the ratio of bytes read by clients from the filesystem to the bytes read by ZeroFS from the back-end, same for writes.

In my Hetzner torture test so far I have managed to write ~300GB to ZeroFS, although due to restarts when ZeroFS crashed I'm sure it's more. I've read all the files perhaps 1.5x. Hetzner shows 3.4TB of data transfer. I believe this means there has been a lot more traffic to the back-end than traffic to the filesystem, but I could be wrong.

[Barre]

More than stats about ZeroFS I'd be interested in stats about the back-end. E.g. requests and KB read/write from/to the bucket, as well as perhaps some latency stats, like min/median/max time to complete operations.

I think that'd be a great addition. It shouldn't be too hard to add because IIRC it's possible to provide a custom HTTP client to the underlying object_store dependency.

In my Hetzner torture test so far I have managed to write ~300GB to ZeroFS, although due to restarts when ZeroFS crashed I'm sure it's more. I've read all the files perhaps 1.5x. Hetzner shows 3.4TB of data transfer. I believe this means there has been a lot more traffic to the back-end than traffic to the filesystem, but I could be wrong.

This is intended. WAL is one layer of amplification, then you have compaction which will lead to a bunch of additional amplification. That's one drawback of LSM trees, but I think it fits the "immutable-mutable" model pretty well that we need to actually be able to offer mutation on top of S3.

[Barre]

I followed up there:

#147