-
Notifications
You must be signed in to change notification settings - Fork 86
Open
Labels
Description
Environment info
- NooBaa Version: 5.15
- Platform: RHEL
Actual behavior
- Our check for using "warmup sparse file" flow in NamespaceFS.read_object_stream() checks on the constant stat that was taken outside the loop, so after initial warmup worked as expected and prefetched the file, we keep checking with the original stat and will keep sending unneeded warmup calls. See:
noobaa-core/src/sdk/namespace_fs.js
Lines 987 to 999 in 176c6f6
// Our buffer pool keeps large buffers and we want to avoid spending | |
// all our large buffers and then have them waiting for high latency calls | |
// such as reading from archive/on-demand cache files. | |
// Instead, we detect the case where a file is "sparse", | |
// and then use just a small buffer to wait for a tiny read, | |
// which will recall the file from archive or load from remote into cache, | |
// and once it returns we can continue to the full fledged read. | |
if (config.NSFS_BUF_WARMUP_SPARSE_FILE_READS && is_sparse_file(stat)) { | |
dbg.log0('NamespaceFS: read_object_stream - warmup sparse file', { | |
file_path, pos, size: stat.size, blocks: stat.blocks, | |
}); | |
await file.read(fs_context, this.warmup_buffer, 0, 1, pos); | |
} |
Expected behavior
- We should refresh the stat - probably enough to update just once after the first warmup, assuming that the likelyhood of a file to become sparse again (evicted from cache, or migrated to tape) is low.
- Other option is to always check stat before each read, which seems wasteful too.
Steps to reproduce
- Read a large sparse file using NSFS that becomes non-sparse on read (transparent recall / fetch).
- See how every loop we keep on logging the "warmup sparse file" message and sending a useless read.