Skip to content

Commit 341e27c

Browse files
authored
[Example] [Bugfix] Fix Gemma3 Generation (#1517)
## Purpose ## * Fix Gemma3 example ## Background ## * Transformers `torch.compile`s the model function before running generation. However, some models fail to compile properly. As of the latest transformers version, Gemma3 is one of those models huggingface/transformers#38333 ## Changes ## * Disable compilation when generating with Gemma3 in example Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>
1 parent 421bd61 commit 341e27c

File tree

1 file changed

+2
-1
lines changed

1 file changed

+2
-1
lines changed

examples/multimodal_vision/gemma3_example.py

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -63,8 +63,9 @@ def data_collator(batch):
6363
image_url = "http://images.cocodataset.org/train2017/000000231895.jpg"
6464
raw_image = Image.open(requests.get(image_url, stream=True).raw)
6565

66+
# Note: compile is disabled: https://github.com/huggingface/transformers/issues/38333
6667
inputs = processor(images=raw_image, text=prompt, return_tensors="pt").to("cuda")
67-
output = model.generate(**inputs, max_new_tokens=100)
68+
output = model.generate(**inputs, max_new_tokens=100, disable_compile=True)
6869
print(processor.decode(output[0], skip_special_tokens=True))
6970
print("==========================================")
7071

0 commit comments

Comments
 (0)