Skip to content

NSFS unneeded read calls to warmup sparse file #8166

@guymguym

Description

@guymguym

Environment info

  • NooBaa Version: 5.15
  • Platform: RHEL

Actual behavior

  1. Our check for using "warmup sparse file" flow in NamespaceFS.read_object_stream() checks on the constant stat that was taken outside the loop, so after initial warmup worked as expected and prefetched the file, we keep checking with the original stat and will keep sending unneeded warmup calls. See:

// Our buffer pool keeps large buffers and we want to avoid spending
// all our large buffers and then have them waiting for high latency calls
// such as reading from archive/on-demand cache files.
// Instead, we detect the case where a file is "sparse",
// and then use just a small buffer to wait for a tiny read,
// which will recall the file from archive or load from remote into cache,
// and once it returns we can continue to the full fledged read.
if (config.NSFS_BUF_WARMUP_SPARSE_FILE_READS && is_sparse_file(stat)) {
dbg.log0('NamespaceFS: read_object_stream - warmup sparse file', {
file_path, pos, size: stat.size, blocks: stat.blocks,
});
await file.read(fs_context, this.warmup_buffer, 0, 1, pos);
}

Expected behavior

  1. We should refresh the stat - probably enough to update just once after the first warmup, assuming that the likelyhood of a file to become sparse again (evicted from cache, or migrated to tape) is low.
  2. Other option is to always check stat before each read, which seems wasteful too.

Steps to reproduce

  1. Read a large sparse file using NSFS that becomes non-sparse on read (transparent recall / fetch).
  2. See how every loop we keep on logging the "warmup sparse file" message and sending a useless read.

More information - Screenshots / Logs / Other output

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions