Python v0.2.0

In this release, we fixed some inconsistencies between the BPETokenizer and the original python version of this tokenizer. If you created your own vocabulary using this Tokenizer, you will need to either train a new one, or use a modified version, where you set the PreTokenizer back to Whitespace (instead of WhitespaceSplit).

Python v0.1.1

• Fix a bug where special tokens get split while encoding

Python v0.1.0

Bump python version for release

v0.0.13

Hotfix Python bindings for 32-bit systems

v0.0.12

Bump version for release

v0.0.11

Fixes the sdist build for Python

v0.0.10

Bump for release

v0.0.9

Bump version for release

v0.0.8

Bump version and update Readme

v0.0.7

Bump version for release