Skip to content

Commit de91684

Browse files
authored
[Testing] Use a slightly larger model that works with group_size 128 (#1502)
SUMMARY: - The previous model was too small for group_size 128; update to use a larger one - The point of the test is to validate the preset schemes (which include 128 as the group_size) so chose to update the model instead of the recipe
1 parent db1ebc7 commit de91684

File tree

1 file changed

+2
-2
lines changed

1 file changed

+2
-2
lines changed

tests/llmcompressor/transformers/gptq/test_oneshot.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -65,7 +65,7 @@ def setUp(self):
6565
import torch
6666

6767
self.output = "./oneshot_output"
68-
self.model = "roneneldan/TinyStories-1M"
68+
self.model = "Xenova/llama2.c-stories110M"
6969
self.dataset = "open_platypus"
7070
self.device = "cuda:0" if torch.cuda.is_available() else "cpu"
7171

@@ -106,7 +106,7 @@ def test_oneshot_application(self):
106106
assert weight_args.num_bits == 4
107107

108108
# Check a specific layer is quantized
109-
targetted_linear_layer = model_loaded.transformer.h[0].attn.attention.k_proj
109+
targetted_linear_layer = model_loaded.model.layers[0].self_attn.k_proj
110110
assert hasattr(targetted_linear_layer, "quantization_scheme")
111111

112112
# Check lm-head is not quantized

0 commit comments

Comments
 (0)