Skip to content

Commit 234dbd1

Browse files
LucasWilkinsonhj-mistral
authored andcommitted
[BugFix][CPU] Fix TorchSDPABackendImpl doesn't have use_irope (vllm-project#21200)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com> Signed-off-by: Himanshu Jaju <hj@mistral.ai>
1 parent 58ad0a6 commit 234dbd1

File tree

1 file changed

+2
-1
lines changed

1 file changed

+2
-1
lines changed

vllm/v1/worker/gpu_model_runner.py

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2668,7 +2668,8 @@ def get_kv_cache_spec(self) -> dict[str, KVCacheSpec]:
26682668
# TODO: Support other attention modules, e.g., cross-attention
26692669
if attn_module.attn_type == AttentionType.DECODER:
26702670
use_local_attention = (self.attention_chunk_size is not None
2671-
and attn_module.impl.use_irope)
2671+
and getattr(attn_module.impl,
2672+
"use_irope", False))
26722673
if attn_module.sliding_window is not None:
26732674
kv_cache_spec[layer_name] = SlidingWindowSpec(
26742675
block_size=block_size,

0 commit comments

Comments
 (0)