Skip to content

Commit 389ff0d

Browse files
committed
Enable explicit truncation for T5Tokenizer
1 parent 7120948 commit 389ff0d

File tree

1 file changed

+2
-0
lines changed

1 file changed

+2
-0
lines changed

python/src/diffusionkit/mlx/tokenizer.py

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -147,6 +147,8 @@ def encode(self, s: str) -> mx.array:
147147
s,
148148
return_tensors="np",
149149
return_attention_mask=False,
150+
max_length=self.max_length,
151+
truncation=True,
150152
)["input_ids"]
151153
)
152154

0 commit comments

Comments
 (0)