Skip to content

Commit 964472b

Browse files
authored
[Doc] Update prefix cache metrics to counting tokens (#18138)
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
1 parent 59dd311 commit 964472b

File tree

1 file changed

+2
-2
lines changed

1 file changed

+2
-2
lines changed

docs/source/design/v1/metrics.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -415,8 +415,8 @@ The discussion in <gh-issue:10582> about adding prefix cache metrics yielded
415415
some interesting points which may be relevant to how we approach
416416
future metrics.
417417

418-
Every time the prefix cache is queried, we record the number of blocks
419-
queried and the number of queried blocks present in the cache
418+
Every time the prefix cache is queried, we record the number of tokens
419+
queried and the number of queried tokens present in the cache
420420
(i.e. hits).
421421

422422
However, the metric of interest is the hit rate - i.e. the number of

0 commit comments

Comments
 (0)