Add Support for custom delay time during rate limiting

[discoverlance-com]

Thanks for this package. It's very good for simple functions rate limiting which I am testing.

I think when it comes to the rate limiting, I see that the msUntilNextWindow depends on the window you set. This can be less useful in some situations such as when I want to rate limit the user for say, 10 function calls per minute. It means that they have to wait for 1 minute after failing the attempts or reaching the limit which does not make for a good experience in some situations like calling a sign in function.

If possible, let us supply a custom delay/decay time like

// Basic API call rate limiting
const { maybeExecute } = useAsyncRateLimiter(
  async (id: string) => {
    const data = await api.fetchData(id);
    return data;
  },
  { limit: 5, window: 1000 * 60, decay: 1000 * 30 } // 5 calls per minute and delay for 30 seconds
);

The delay can be optional where by default, it uses the window but can also use the delay/decay:

private rejectFunction(): void {
    this.rejectionCount++
    if (this.options.onReject) {
      const oldestExecution = Math.min(...this.executionTimes)
      const msUntilNextWindow =
        oldestExecution + ( this.options.decay ?? this.options.window) - Date.now() // change here to use the window or delay

      this.options.onReject({
        msUntilNextWindow,
        currentExecutions: this.executionTimes.length,
        limit: this.options.limit,
        rejectionCount: this.rejectionCount,
      })
    }
  }

I could submit an issue/pull request for this change but it looks like a core api change so I wanted to open this discussion first to get your ideas on the topic.

I guess my suggestion also defeats the purpose of rate limiting as it's per window.... so till the next window is available, the request should not be valid. Do you have any other suggestions though in such a case? I think the other option is to reduce the window and possibly align the attempts to that window.

[KevinVandy]

Sorry, just seeing this now. There's some good ideas here. I just made some changes to the rate limiters over the past few days, so now it's prime for even more changes lol.
I'm interested in what you think a decay callback option might look like

[discoverlance-com]

I think maybe we need to understand or state the core goal we are trying to accomplish?

It looks like the ideas you shared so far work by trying to either make the limit dynamic or the window dynamic right?

Initially, the core issue here stemmed from: if the user uses up the limit early on within the window, say 5 per minute, and they send 5 in a 10 seconds, they have to wait for 50 seconds before sending another (for the next window) which is ok for a normal server but for functions that are generally callable, I wanted to see if we could potentially reduce this time so it is more or less a focus on the remaining time after the window's limit has been used up than dynamically updating the window or limit itself (which of course is a useful thing on it's own).

One might still want to keep the same window, but maybe reduce the time the user has to wait till next execution dynamically or just specify it statically? I guess that also defeats the purpose of rate limiting the request as it's x requests per y time.

For the dynamic window or limit, will it not in time be a bit aggressive? If the user keeps on tapping the button, that means the limit could decrease to say, 1 (I guess what you wrote is an example and the user can update it anyhow they see fit)? How does that affect the next window? Will the limit or window be reset on the next window?

[KevinVandy]

ok, is this question actually asking for automatic retry logic then? That's not necessarily in the plans/scope of TanStack Pacer. Going to let that be external or even encouraged by TanStack Query. Unless we make a dedicated Retryer "primative"

[discoverlance-com]

Yes, this question was about the delay after widow is consumed so yes, I guess retries.

In general what I was looking for was, if I could say, after the limit of the window has been consumed, use this, x milliseconds or dynamic logic that returns x milliseconds to determine how long to wait thereby ignoring the window's time left but considering a fixed time to wait after each window is consumed.

re: not necessarily in plans/scope of tanstack pacer, sure that's fine. This seems to be a custom behavior than what rate limiters do.

I see, a dedicated Retryer "primitive" could be good, a function that works like a rate limiter but allows you to dictate how long to wait, static or dynamic after each window consuming?

[KevinVandy]

I'm wondering if there is already a way to do this by just taking advantage of the onReject callback? I might not be understanding what you're asking for exactly still.

The onReject param signature has changed a lot since you first started this discussion. Now it just returns the whole rateLimiter instance.

onReject: (rateLimiter) => {
  console.log(rateLimiter.getMsUntilNextWindow)
  console.log(rateLimiter.getRejectionCount()
  setTimeout(() => rateLimiter.reset(), 30000)
  // etc.
}

You should be able to just compose whatever custom delay logic you want there

[discoverlance-com]

Oh I see, so on reject, I could reset the rate limiter...

Yes, what you are talking about is what I am looking at. Basically, I tried rate limiter with a sign in function and I limited it to 10 per 60 seconds but, if say, the user presses on the sign in button 10 times in 10 seconds, it meant, they have to wait for 50 seconds before they can click the button again during the next window, which i thought was not good in that case so I just wanted to be able to just say, don't wait for 50 seconds or till the next window but allow new calls or open a new window, after say.

I wanted to maintain the window, but control how long the user waits till they can make another execution instead of waiting till the window is over.