Skip to content

Commit eba4c36

Browse files
authored
fixing bug in GPTQ (#120)
* fixing bug in GPTQ Summary: shape was always padded even when not needed. Test Plan: pythont test/quantization/test_quant_api.py -k "test_gptq_quantizer_int4wo" Reviewers: Subscribers: Tasks: Tags: * removing extra spaces Summary: Test Plan: Reviewers: Subscribers: Tasks: Tags:
1 parent ec258e0 commit eba4c36

File tree

1 file changed

+4
-1
lines changed

1 file changed

+4
-1
lines changed

torchao/quantization/GPTQ.py

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -950,7 +950,10 @@ def __init__(
950950
# TODO: this is the gpt-fast version, merge with the main version later
951951
def make_names_and_values_dict_func(q, qparams):
952952
k = q.shape[1]
953-
new_k = find_multiple(k, 1024)
953+
if not _check_linear_int4_k(k, groupsize):
954+
new_k = find_multiple(k, 1024)
955+
else:
956+
new_k = k
954957
# how much we need to pad the weight
955958
delta_k = new_k - q.shape[1]
956959
q = q.to(torch.int32)

0 commit comments

Comments
 (0)