More informations about drivers, and my custom Upstash driver

[ScreamZ]

Hello,

I'm interested in that library, I would like to say a good job and thank you as the first word.

I've been migrating from Vercel to a custom barebone solution, and as I'm using Upstash Redis https://upstash.com/ I change my code a bit to use their SDK.

I've written the driver based on the Redis-string one.

I'll share a draft here, It still needs some changes to be customizable on instantiation. However, I have some questions.

Note

Click below for my Upstash Redis driver.

➡️ My Upstash Redis Custom Driver (click)

// @ts-check
import { CacheHandler } from "@neshca/cache-handler";
import createLruHandler from "@neshca/cache-handler/local-lru";
import { Redis } from "@upstash/redis";
import { chunk } from "lodash-es";

const NEXT_CACHE_IMPLICIT_TAG_ID = "_N_T_";

/**
 * @typedef {import("@neshca/cache-handler").Handler} Handler
 * @typedef {import("@neshca/cache-handler").CacheHandlerValue} CacheHandlerValue
 */

const redis = new Redis({
  url: process.env.REDIS_URL,
  token: process.env.REDIS_TOKEN,
});

const prefix = "NextJS-Cache:";
const revalidatedTagsKey = `${prefix}__revalidated_tags__`;
const sharedTagsKey = `${prefix}__sharedTags__`;
const revalidateTagQuerySize = 100;

CacheHandler.onCreation((params) => {
  const upstashRedisHandler = createUpstashRedisHandler();
  const lruHandler = createLruHandler();

  return {
    handlers: params.dev ? [] : [upstashRedisHandler, lruHandler],
  };
});

function createUpstashRedisHandler() {
  /** @type {Handler} */
  const handler = {
    name: "Upstash",
    async get(key, meta) {
      /** @type {CacheHandlerValue|null} */
      const cacheValue = await redis.get(prefix + key);

      if (!cacheValue) {
        return null;
      }

      const combinedTags = new Set([...cacheValue.tags, ...meta.implicitTags]);

      if (combinedTags.size === 0) {
        return cacheValue;
      }

      const revalidationTimes = await redis.hmget(revalidatedTagsKey, ...Array.from(combinedTags));

      for (const timeString in revalidationTimes) {
        if (timeString && Number.parseInt(timeString, 10) > cacheValue.lastModified) {
          await redis.unlink(prefix + key);

          return null;
        }
      }

      return cacheValue;
    },
    async set(key, value) {
      const exat = value.lifespan?.expireAt;

      await Promise.all([
        redis.set(prefix + key, value, exat ? { exat } : undefined),
        value.tags.length ? redis.hset(sharedTagsKey, { [key]: value.tags }) : undefined,
      ]);
    },
    async revalidateTag(tag) {
      // The revalidation process is done by the CacheHandler class on the next get operation.
      if (tag.startsWith(NEXT_CACHE_IMPLICIT_TAG_ID)) {
        await redis.hset(revalidatedTagsKey, { [tag]: Date.now() });
      }

      /** @type{Map<string, string[]>} */
      const tagsMap = new Map();

      /** @type{string|number} */
      let cursor = 0;

      // Fetch all keys that have the tag
      do {
        const [scanCursor, fields] = await redis.hscan(sharedTagsKey, cursor, {
          count: revalidateTagQuerySize,
        });

        for (const [field, value] of chunk(fields, 2)) {
          // @ts-expect-error
          tagsMap.set(field, value);
        }

        cursor = scanCursor;
      } while (Number(cursor) !== 0);

      /** @type {string[]} */
      const keysToDelete = [];
      /** @type {string[]} */
      const tagsToDelete = [];

      for (const [key, tags] of tagsMap) {
        if (tags.includes(tag)) {
          keysToDelete.push(prefix + key);
          tagsToDelete.push(key);
        }
      }

      if (keysToDelete.length === 0) {
        return;
      }

      const deleteKeysOperation = redis.unlink(...keysToDelete);

      const updateTagsOperation = redis.hdel(sharedTagsKey, ...tagsToDelete);

      await Promise.all([deleteKeysOperation, updateTagsOperation]);
    },
    async delete(key) {
      await redis.unlink(key);
    },
  };

  return handler;
}

export default CacheHandler;

Questions

NextJS cache documentation is currently very opaque, and the documentation is being redirected to your library. Which is nice. Maybe you've some other resources to help me understand the core feature, without the addition of Next-cache ?

To stimulate community members like me to contribute, it could be nice to explain things upstream so we can make clear decisions. That's why I need some precision about some things.

➡️ What is implicit tags? What is it supposed to be ?

Are they those generated automatically by NestJS for routing cache? Do they include tags from fetch tags or unstable__cache tags ?

➡️ About NEXT_CACHE_IMPLICIT_TAG_ID: "_N_T_"

Currently, you've scoped it to the next-cache package, but it exists inside nextjs package (somewhere nester in a dist folder). My question is what is that about, is it considered stable and public API?

Why does it need a different treatment from other cache keys?

➡️ about get and revalidateTag

I'm not sure why this is required to revalidate on the next get operation? Generally speaking I would be pleased if there were more information about what are elements from the code (like implicit tags) with examples and why are things done like they were.

➡️ Side question

What's the most simple way to handle two cache? For some keys, I would like to handle it with LRU not redis. I guess my best bet is to put LRU first and throw if the key is not found in get and in the set method throw if tags are not matching my requirements, so redis would handle it ?

Thank you for answering this first batch of questions and for your work, and I would be pleased to contribute if I can manage to understand the logic of all of this.

[better-salmon]

Hello @ScreamZ!

I'm interested in that library, I would like to say a good job and thank you as the first word.

Thank you so much!

---

You've asked a lot of good question. Let me answer them one by one.

Upstash Redis client

Firstly, thank you for your effort in creating a Handler for @upstash/redis client.

You can use more robust redis-strings Handler by using redis npm package with Upstash. Here is the docs. The redis-strings is tested in every release, and guarantees that your application will work properly no matter what changes were made in the library.

Connecting to Upstash Redis with redis npm package:

import { createClient } from "redis"

const client = createClient ({
  url : "rediss://default:password@hostname:port"
});

[ScreamZ]

You can use it this is true, but upstash redis is designed to be a HTTP SDK, that is designed to be used in serverless environment with cloud function and everything related to long-running processes and connection management.

I concede that using next-shared-cache is generally because you leave vercel and go on some stateful application where this issue is no longer relevant, but the upstash redis driver remains useful (I literally copy-pasted the code from your redis-string, just renamed commands to match syntax)

[better-salmon]

I did not try, but I think that redis package can work in serverless environment because it uses a TCP socket to communicate. I never use serverless, may be there are additional restrictions.

[ScreamZ]

Yes it does, but the issue is about connection exhausting. That's why poolers for SQL like Prisma Accelerate or PgBouncer have been created.

You can read more at her in prisma doc or in our case for redis here at Upstash doc.

While using stateful application aka traditional server, this doesn't apply.

PS: Still waiting for my other questions answers :D

[better-salmon]

Still waiting for my other questions answers :D

I know :D

I need time to prepare answers.

[better-salmon]

@ScreamZ, hi there! I made sure to address all your inquiries and set up a Wiki page for those who want to delve into the internal workings of Next.js.

[ScreamZ]

Wonderful! Thank you very much.

So briefly:

• Implicit tags: Automatically generated and handled tags from NextJS to manage cache about page-router. And tied to revalidatePath feature I guess (that is a superset of revalidateTag)?
• When calling revalidatePath, this somehow call revalidateTag and pass implicit tags from the given path.

Here comes few more questions.

• What does _N_T_ stands for ? Next Tag ? ahah
• Is "implicit" a next-cache term or is this a nextJS term ? It's like an automated internal cache, but now that you explained it makes sense to me. But the naming is strange when you don't have the explanation.
• Still not exactly sure about shared tags but I'll open a sandbox project and make some tests to figure out exactly. This is not related to next-cache directly. But to driver impl.

NEXT_CACHE_IMPLICIT_TAG_ID is considered an internal constant and is not part of the public API. It is intended for use within Next.js' core and may change without notice in future releases.

Maybe we could ping or tag some next-js maintainer. As your library is the one referenced in the documentation for "custom cache" as the source of truth and the nextjs documentation on custom cache adapter is pretty light, maybe we could have some assertion on stability. I can understand that they prefer having us using vercel implementation by default, as this is the source of money 😅 But there is hosting on barebone server many time.

What's next ?

Now that I have clearer understanding, I might start contributing :)

I started to think of a driver that is hybrid in-memory and redis.
My goal is to get rid of calling redis from the redis SDK and rely only on next cache functions, like the unstable cache (or next-cache equivalent, the one you made).

The idea is to store some keys in redis, when there is a benefit of "preload" the cache for other nodes of a cluster. Like a database query. And for serializable data.
And a LRU, in memory cache for things that are not needed to be in redis, like things you would store in a JS Set or Map.

Note

In my app, I'm storing API Client in memory. As it's instances of a Client, object references, they cannot be serialized. This allows me to reuse client on the flight without connecting/disconnect on each request, depending on specific input.
Currently I'm managing this "cache" outside of next cache, as standard LRU. And I'm forced to use a TTL to expires the client sometimes and make them again by reading redis.