Skip to content

Commit 28a6d54

Browse files
authored
[Bugfix] Fix Machete zero point issue for GPTQ models on SM90 (#21066)
Signed-off-by: mgoin <mgoin64@gmail.com>
1 parent 58760e1 commit 28a6d54

File tree

1 file changed

+5
-0
lines changed
  • vllm/model_executor/layers/quantization/kernels/mixed_precision

1 file changed

+5
-0
lines changed

vllm/model_executor/layers/quantization/kernels/mixed_precision/machete.py

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -126,6 +126,11 @@ def apply_weights(self,
126126
if c.has_g_idx:
127127
x_2d = self.act_perm(x_2d)
128128

129+
if c.zero_points:
130+
assert w_zp is not None
131+
else:
132+
w_zp = None
133+
129134
output = ops.machete_mm(a=x_2d,
130135
b_q=w_q,
131136
b_type=c.weight_type,

0 commit comments

Comments
 (0)