Question on mapping authorized_calls metrics to a user

[nerdalert]

This is a general question around looking into how to use Kuardrant in a chargeback scenario. The need is being able to match an authorized call from a limitador perspective to a user ID.

The current metrics are namespace scoped like so:

# HELP limited_calls Limited calls
# TYPE limited_calls counter
limited_calls{limitador_namespace="llm-d/qwen-qwen3-0-6b"} 13
# HELP authorized_calls Authorized calls
# TYPE authorized_calls counter
authorized_calls{limitador_namespace="llm-d/qwen-qwen3-0-6b"} 2

Refining them further down to a call to ID mapping allows for an accurate scrape of authorized calls that will allow for a clear usage. Scraping the same from Authorino would give you that mapping but Authorino doesn't have an awareness of calls that are limited (from a cursory code review, maybe Im wrong which would be fantastic).

As an example, I patched limitador to include (limited_calls || authorized_calls) to user ID, allowing for accurate usage tracking and come away with:

# HELP authorized_calls Authorized calls
# TYPE authorized_calls counter
authorized_calls{limitador_namespace="llm-d/ms-sim-llm-d-modelservice",user="premiumuser2"} 10
authorized_calls{limitador_namespace="llm-d/ms-sim-llm-d-modelservice",user="premiumuser1"} 10
authorized_calls{limitador_namespace="llm-d/ms-sim-llm-d-modelservice",user="freeuser1"} 4

Code for the above metrics nerdalert/limitador

I'm sure there are plenty of issues around cardinality and separation of concerns with that approach, but I'm out of ideas on how to make this use case work otherwise.

More details/context on the use case here redhat-et/kuadrant-llm-integration.

Thanks for any ideas, they are very much appreciated. Cheers!

[jasonmadigan]

cc'ing: @alexsnaps @eguzki @didierofrivia @maleck13 @david-martin - Brent and I chatted a bit about this offline, suggested an issue to discuss the use-case would be best

[david-martin]

The cardinality of adding extra labels, in particular a user identifier, is the biggest concern with the proposed approach.
However, if the behaviour is behind some configuration (default behaviour is to do what's currently done), is communicated well and sufficiently understood what you should do to handle this (for example, you should scale your prometheus accordingly), then it sounds like a reasonable feature.

I can't think of another way to solve this right now.
I do think there's merit in exposing some level of configuration over what additional attributes to include as labels, rather than hardcoding as in your example limitador code change.

I'll leave it to someone more knowledgable of the limitador component to comment further on actual changes and feasiability.

[eguzki]

I fully agree with David's assessments.

Authorino doesn't have an awareness of calls that are limited (from a cursory code review, maybe Im wrong which would be fantastic).

Correct, Authorino does not have a native awareness of rate-limited calls.

The key distinction is that Limitador, which handles rate limits, is entirely agnostic to concepts like users and groups. It operates exclusively on "descriptors" and the limits applied to them. Whether a descriptor entry is auth.identity.userid: freeuser1 or request.path: /cars, Limitador processes them identically.

Because of this design, the configuration for appending extra labels to metrics must be generic. The logic should be simple: when a specific descriptor entry is found, its value is added to the metrics with a corresponding key name.

While Limitador's design is effective, its core principle is to be agnostic to users and groups. It operates on generic "descriptors" (like auth.identity.userid: freeuser1 or request.path: /cars) without understanding their context. This decoupling from the authentication service, however, raises a concern about whether the rate limiter is the most appropriate place to gather metrics about specific users.

A more robust approach would be to implement this functionality at the gateway, with Kuadrant's WebAssembly (Wasm) module being a strong candidate.

The gateway is uniquely positioned because it has a complete view of the entire request lifecycle. For example, if a request from "user Foo" fails authentication, the rate limiting service will never process it, and that failed attempt will not be included in its metrics. The gateway, in contrast, can verify that all policies have been successfully enforced before routing a request, allowing it to generate far more accurate and comprehensive metrics on both authorized and rejected requests.

Does it make sense to you?

[nerdalert]

Hi @eguzki @david-martin thanks for the feedback. I iterated on this a bit to get a wasm-shim PoC going. Documented here https://github.com/redhat-et/kuadrant-llm-integration/blob/main/llm-d/wasm-plugin-metrics.md

The main limitation of the approach at first look is hostcalls::define_metric doesn't support prom labels.

[eguzki]

Nice investigation work. Too bad that labels cannot be added natively from the proxy-wasm SDK. TIL metric relabeling defined at the servicemonitor. Nice trick.

Regarding the PoC, It's concerning that you need to call define_metric for each user, which leads us to the cardinality problem. What are Envoy's design principles around defined metrics? When one user never shows up again, the metric_id stored in the envoy service can be considered memory leak? Is it garbage collected?

Regarding proposed alternatives: gateways logs seems promising.

More alternatives would be metric emission from the rate limiting service as you tried initially. Or some envoy filter that emits metrics with labels. I have seen envoy metrics with labels.