Skip to content

Commit 8cb41a9

Browse files
committed
fix: Fix wrong bool condition for split equal in hybrid cache
Branch: HybridRecurrentCache Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>
1 parent fe39803 commit 8cb41a9

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

src/llama-kv-cache-hybrid-recurrent.cpp

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -96,7 +96,7 @@ llama_pos llama_kv_cache_hybrid_recurrent::seq_pos_max(llama_seq_id seq_id) cons
9696
llama_memory_state_ptr llama_kv_cache_hybrid_recurrent::init_batch(const llama_batch & batch, uint32_t n_ubatch, bool embd_pooled, bool logits_all) {
9797

9898
// since this includes a recurrent cache, we cannot use split_simple
99-
auto sbatch = llama_sbatch(batch, hparams.n_embd, true, logits_all);
99+
auto sbatch = llama_sbatch(batch, hparams.n_embd, false, logits_all);
100100

101101
// follow the recurrent pattern for creating the ubatch splits
102102
std::vector<llama_ubatch> ubatches;

0 commit comments

Comments
 (0)