Commit 71963c3

authored

Llama test tokenizer uses the new merges style (#132)

1 parent 4f31334 commit 71963c3Copy full SHA for 71963c3

2 files changed

+2

-2

lines changed

Tests/TokenizersTests
- Resources
  - tokenizer_tests.json
- TokenizerTests.swift

2 files changed

+2

-2

lines changed

Comments

(0)