Skip to content

kv cache is a continuous block of memory, so how are the different tokens distributed in memory? #3372

Unanswered
yancaoweidaode asked this question in Q&A
kv cache is a continuous block of memory, so how are the different tokens distributed in memory? #3372
@yancaoweidaode yancaoweidaode
Sep 27, 2023 · 1 comment
Return to top
Discussion options

yancaoweidaode
Sep 27, 2023

You must be logged in to vote

Replies: 1 comment

Comment options

ggerganov
Oct 2, 2023
Maintainer

You must be logged in to vote
0 replies
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Category
Q&A
Labels
None yet
2 participants
@yancaoweidaode @ggerganov
Converted from issue

This discussion was converted from issue #3360 on September 28, 2023 00:37.