Skip to content

Commit 3c376f3

Browse files
committed
fix: Use @compilade's suggested fix for seq_id indexing with equal splits in unified cache
Branch: GraniteFour Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>
1 parent fa35d19 commit 3c376f3

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

src/llama-kv-cache-unified.cpp

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -568,7 +568,7 @@ int32_t llama_kv_cache_unified::find_slot(const llama_ubatch & ubatch) const {
568568
bool found = true;
569569
for (uint32_t i = 0; i < n_tokens; i++) {
570570
const llama_pos pos = ubatch.pos[i];
571-
const llama_seq_id seq_id = ubatch.seq_id[i][0];
571+
const llama_seq_id seq_id = ubatch.seq_id[i / ubatch.n_seq_tokens][0];
572572

573573
// can we use this cell? either:
574574
// - the cell is empty

0 commit comments

Comments
 (0)