Skip to content

Commit c0ef995

Browse files
committed
fix: Use @compilade's suggested fix for seq_id indexing with equal splits in unified cache
Branch: GraniteFour Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>
1 parent eba2089 commit c0ef995

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

src/llama-kv-cache-unified.cpp

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -566,7 +566,7 @@ int32_t llama_kv_cache_unified::find_slot(const llama_ubatch & ubatch) const {
566566
bool found = true;
567567
for (uint32_t i = 0; i < n_tokens; i++) {
568568
const llama_pos pos = ubatch.pos[i];
569-
const llama_seq_id seq_id = ubatch.seq_id[i][0];
569+
const llama_seq_id seq_id = ubatch.seq_id[i / ubatch.n_seq_tokens][0];
570570

571571
// can we use this cell? either:
572572
// - the cell is empty

0 commit comments

Comments
 (0)