Skip to content

Commit db5bda9

Browse files
Clarify language models in README (#203)
1 parent 3ee6c35 commit db5bda9

File tree

1 file changed

+1
-2
lines changed

1 file changed

+1
-2
lines changed

README.md

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -42,7 +42,7 @@ If you work at OpenAI, make sure to check the internal documentation or feel fre
4242

4343
## What is BPE anyway?
4444

45-
Models don't see text like you and I, instead they see a sequence of numbers (known as tokens).
45+
Language models don't see text like you and I, instead they see a sequence of numbers (known as tokens).
4646
Byte pair encoding (BPE) is a way of converting text into tokens. It has a couple desirable
4747
properties:
4848
1) It's reversible and lossless, so you can convert tokens back into the original text
@@ -128,4 +128,3 @@ setup(
128128

129129
Then simply `pip install ./my_tiktoken_extension` and you should be able to use your
130130
custom encodings! Make sure **not** to use an editable install.
131-

0 commit comments

Comments
 (0)