RFC: Persistent Cache Architecture

[jerrykingxyz]

Feature Name: Design Persistent Cache Architecture
Start Date: 2024/11/18

Summary

Design persistent cache architecture.

Context

The ColdStart needs to build all the memory data, the build speed is slow.
The HMR can use the memory data that has been built to accelerate the build, so the speed is fast.
The HotStart that enables persistent cache will restore the memory data as much as possible in the starting phase, so that the start speed is infinitely close to HMR.

Glossary

Artifact: The output of each compile step in rspack. It can be a data structure or multiple data structures. eg. MakeArtfact is the only Artifact at Make step.

Detailed design

Architecture

The persistent cache consists of three modules:

• Snapshot: Responsible for calculating the files updated or deleted during downtime. RFC: Persistent Cache Snapshot #8463

struct Snapshot { ... }
impl Snapshot {
    fn add(&self, paths: Vec<PathBuf>);
    fn remove(&self, paths: Vec<PathBuf>);
    // return (modified_paths, deleted_paths)
    fn calc_modified_paths(&self) -> (Vec<PathBuf>, Vec<PathBuf>);
}

• Occasion: Each build step includes its own Occasion implementation, responsible for storing and transforming specific Artifact and hot-start restoration. RFC: Persistent Cache MakeOccasion #8636

struct MakeOccasion { ... }
impl MakeOccasion {
    fn save(&self, make_artifact: MakeArtifact);
    fn recovery(&self) -> MakeArtifact;
}

• Storage: Underlying persistent storage module, providing KV storage functions. RFC: Persistent Cache Storage #8646

trait Storage {
    fn get_all(&self, scope: &str) -> Vec<(Vec<u8>, Vec<u8>)>;
    fn set(&self, scope: &str, key: Vec<u8>, value: Vec<u8>);
    fn remove(&self, scope: &str, key: &[u8]);
    fn idle(&self);
}

The coupling between the persistent cache and the entire build process is low, with only the following call locations:

• before_compile: At the beginning of the compile, when the modified_file and removed_file do not exist (usually the first start), the updated or deleted files calculated by Snapshot will be set.
• before_<step_name>: Before the core step, when the corresponding Artifact is empty, the corresponding Occasion will be used to restore the Artifact for filling.
• after_<step_name>: After the core step is completed, pass Artifact to Occasion for saving.
• after_compile: After the compile is completed, all the files involved in this compile are added to the Snapshot, and the Storage is notified to be idle.

trait Cache {
    fn before_compile(&self, comp: &mut Compilation);
    fn before_<step_name>(&self, artifact: &mut StepArtifact);
    fn after_<step_name>(&self, artifact: &StepArtifact);
    fn after_compile(&self, comp: &Compilation);
}

Cache implementation

The rspack cache maintains a one-to-one relationship with the compiler, and there are the following cache types:

• DisableCache: Disable cache, which removes Artifact in compilation when before_<step_name>, and does nothing at other call locations.
• MemoryCache: Memory cache(Default), since rspack will reuse the Artifact at previous compilation , no processing is required.
• PersistentCache: Persistent cache, it will generates Artifact to memory when current compilation without Artifact.

Cache configuration

The configuration on the Rust side is relatively simple, just need to gather the configurations of various caches.

enum CacheOptions {
    Disable,
    Memory,
    Persistent(PersistentCacheOptions)
}

struct PersistentCacheOptions {
    snapshot: SnapshotOptions,
    storages: Vec<StorageOptions>
}

The cache configuration on the Napi side is to convert the above configuration into JS style.

type RawCacheOptions = Ethier<RawCommonCacheOptions, RawPersistentCacheOptions>;

struct RawCommonCacheOptions {
    #[napi(ts_type = r#""disable"|"memory""#)]
    r#type: String
}

struct RawPersistentCacheOptions {
    #[napi(ts_type = r#""persistent""#)]
    r#type: String,
    snapshot: RawSnapshotOptions,
    storage: Vec<RawStorageOption>
}

The configuration at binding.d.ts is

type Cache = {
    type: "disable" | "memory"
} | {
    type: "persistent",
    snapshot: { ... },
    storage: Array<{...}>
}

The configuration of Js side exposed to the outside is

// `undefined` will set cache to `false` in production mode，otherwise cache will be set to `true`
// `true` will be converted to { type: "memory" }, and `false` will be converted to  { type: "disable" }
type Cache = undefined | boolean | {
    type: "memory"
} | {
    type: "persistent",
    snapshot: { ... },
    storage: Array<{...}>
}

[n0th1ng-else]

This is great that we have an RFC. May I ask you to provide more specific information regarding the persistent cache - the lifecycle, where it is stored (particular paths on the file system), and how to invalidate the whole cache?

Also, it would be interesting to see the ratio is cache hits - let's say if we switch branches.

[jerrykingxyz]

In simple terms

the lifecycle of persistent cache

Rspack persistent cache corresponds to the compiler. It is initialized at the same time as the compiler. When the compiler starts building, it will try to use the persistent cache. When the build ends, the data will be saved to the persistent cache. Ref:

rspack/crates/rspack_core/src/cache/persistent/mod.rs

Line 36 in cceb0ea

pub struct PersistentCache {

where it is stored (particular paths on the file system)

Persistent cache data is saved tonode_modules/.cache/rspack by default, and can be configured via https://rspack.rs/config/experiments#cachestorage. Ref:

rspack/crates/rspack_core/src/cache/persistent/storage/mod.rs

Line 20 in cceb0ea

pub fn create_storage(

how to invalidate the whole cache

If cache.buildDependencies and cache.version are updated, Rspack will generate a new hash directory in node_modules/.cache/rspack, and rspack will clean up the directories that have not been used for 7 days when it starts. Ref:

rspack/crates/rspack_core/src/cache/persistent/storage/mod.rs

Line 33 in 1c14e9a

expire: 7 * 24 * 60 * 60 * 1000,

For more details on how rspack stores data you can see #8646. And the source code is under rspack_storage

[n0th1ng-else]

great, thank you for the details. makes sense to me 👍🏿