We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
There was an error while loading. Please reload this page.
1 parent 4226a6c commit 0f8ec70Copy full SHA for 0f8ec70
tiktoken/core.py
@@ -68,7 +68,7 @@ def encode(
68
and `disallowed_special` parameters. In particular:
69
- Setting `disallowed_special` to () will prevent this function from raising errors and
70
cause all text corresponding to special tokens to be encoded as natural text.
71
- - Setting `allowed_special` to "all" will allow cause this function to treat all text
+ - Setting `allowed_special` to "all" will cause this function to treat all text
72
corresponding to special tokens to be encoded as special tokens.
73
74
```
0 commit comments