Loop through all handlers

[topolanekmartin]

Currently the get methods are called and the first value found is returned, even if it is null or undefined. You only go to the next handler when an exception is thrown. What if I wanted to have custom handlers in this order - local inmemory cache and then redis?

Unfortunately, the data in redis is ignored, and due to the fact that it is missing in the in-memory memory, null is returned and thus the set method is started. I would like the get methods to be called first from all handlers and only after nothing is found or an exception is thrown, the set function would be called.

[better-salmon]

Hey, @topolanekmartin! Thank you for your question. CacheHandler doesn't call the get method for all Handlers by design. I will explain why.

CacheHandler's main objective is to assist in developing multi-instance applications that can scale effectively. The team behind Next.js recommends turning off in-memory caching in such setups. This is a valid suggestion because when each application instance has its local cache, the data can eventually diverge between the different instances. Therefore, for multi-instance applications, it is always advisable to prioritize using a shared cache. Local caching must be a fallback option when a shared cache is unavailable.

CacheHandler is specifically designed to work with cache stores that have TTL (Time To Live) functionality. If the cache store doesn't support this feature, CacheHandler will internally check whether the value has expired. Users can manually set TTL periods via the revalidate option (Pages Router revalidate option), indicating their preference for seeing new data after this period. All prebuilt Handlers respect TTL, and if a value is not found in one cache store, it's highly likely that it's not available in others either. In such cases, Next.js will render the new version and pass it to the CacheHandler.

Users can revalidate the cache anytime by using the revalidate function (Pages Router revalidate function). In multi-instance setups, they can revalidate a shared cache value by calling the function once on any of the instances. However, if they only have a local cache, they need to call each instance's revalidate function.

With all these said, there is no reason to place the local cache in front of the shared one in the multi-instance setups.

Please explain why you want to use the local cache before the shared one.

[topolanekmartin]

Hi @better-salmon,

thanks for your exhausting answer.

We have a web application running in a Kubernetes cluster with multiple replicas. Therefore, we need to implement a shared cache.

The web application has many pages implemented using the ISR strategy. We are also focusing on optimizing TTFB.

Especially for pages that can have up to 17 MB of data in the cache. In this case, we are already encountering issues with implementing a shared cache through Redis.

I investigated further and found that the libraries we used to connect to Redis (node-redis, ioredis) use sockets for connection and data retrieval.

These sockets download data in 64 kB chunks and there is no way to influence this. This results in up to 272 round trips. Each of these round trips takes approximately 0.3 ms.

In other words, it takes about 80 ms just to download data from Redis. Not to mention parsing the response and other operations that add time. When compared to an in-memory cache, where it takes only 0.5 ms to retrieve 17 MB of data, a pure Redis solution is not sufficient for us.

[dross15]

@better-salmon should we then be setting cacheMaxMemorySize: 0 in next.config.js as suggested in the docs you link (turning off in-memory caching)? If so, maybe it should be added to the docs for next-shared-cache as well?

I'm using s3 instead of redis, so if I wanted local caching as a fallback option, would I want to be using using the lruCache? Isn't this also just an in-memory cache like the Next docs say not to use?

To prevent stale data from being shown since the cache is not shared between pods by default, you can configure the Next.js cache to provide a cache handler and disable in-memory caching

Thanks!

[gchallen]

This makes a bit of sense. However, I'd suggest trying to explain this more carefully, since it's different than how many other multi-level caches work. For example, the CPU memory cache will load data from memory into L{1,2,3} caches closer to the CPU to improve performance. Yes, this requires maintaining cache consistency, which is one of the harder problems in computer science. And I can understand why you don't want to take on this challenge. But, again, the documentation should be improved.

I guess it's also unclear to me why you'd ever use multiple cache implementations given this limitation.