Skip to content

Commit 3cc8350

Browse files
committed
fix: Fix wrong bool condition for split equal in hybrid cache
Branch: HybridRecurrentCache Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>
1 parent bbf6c35 commit 3cc8350

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

src/llama-kv-cache.cpp

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3123,7 +3123,7 @@ llama_pos llama_kv_cache_hybrid_recurrent::seq_pos_max(llama_seq_id seq_id) cons
31233123
llama_memory_decode_state_ptr llama_kv_cache_hybrid_recurrent::init(const llama_batch & batch, uint32_t n_ubatch, bool embd_pooled, bool logits_all) {
31243124

31253125
// since this includes a recurrent cache, we cannot use split_simple
3126-
auto sbatch = llama_sbatch(batch, hparams.n_embd, true, logits_all);
3126+
auto sbatch = llama_sbatch(batch, hparams.n_embd, false, logits_all);
31273127

31283128
// follow the recurrent pattern for creating the ubatch splits
31293129
std::vector<llama_ubatch> ubatches;

0 commit comments

Comments
 (0)