Skip to content

Actions: pytorch-labs/tokenizers

Actions

trunk

Actions

Loading...
Loading

Show workflow options

Create status badge

Loading
trunk.yml
161 workflow runs
161 workflow runs

Filter by Event

Filter by Status

Filter by Branch

Filter by Actor

Use weak symbol create_fallback_regex to separate the implementation using PCRE2 and std::regex trunk #86: Pull request #77 opened by larryliu0820
May 21, 2025 23:17 2m 33s export-D75173102
export-D75173102
May 21, 2025 23:17 2m 33s
Add look ahead tiktoken target trunk #85: Commit def3a96 pushed by facebook-github-bot
May 16, 2025 20:20 2m 34s main
main
May 16, 2025 20:20 2m 34s
Revert "Handle null bos and eos token (#66)" (#73) trunk #84: Commit d539a61 pushed by jackzhxng
May 13, 2025 23:51 2m 38s main
main
May 13, 2025 23:51 2m 38s
Revert "Fix tokenizer special token handling (#67)" (#72) trunk #83: Commit 5b46173 pushed by jackzhxng
May 13, 2025 23:50 2m 47s main
main
May 13, 2025 23:50 2m 47s
Accept custom pattern string and special tokens trunk #82: Commit bdda7c8 pushed by facebook-github-bot
May 8, 2025 23:36 2m 52s main
main
May 8, 2025 23:36 2m 52s
Fix tokenizer special token handling (#67) trunk #81: Commit 9ceef56 pushed by jackzhxng
May 2, 2025 17:07 2m 32s main
main
May 2, 2025 17:07 2m 32s
Handle null bos and eos token (#66) trunk #80: Commit c11169c pushed by jackzhxng
May 1, 2025 17:51 2m 33s main
main
May 1, 2025 17:51 2m 33s
Forward fix missing vtable for bpe tokenizer trunk #79: Commit 1621280 pushed by facebook-github-bot
April 30, 2025 02:46 2m 37s main
main
April 30, 2025 02:46 2m 37s
Fix duplicate bpe tokenizer base symbol (#63) trunk #78: Commit 35d185e pushed by jackzhxng
April 29, 2025 16:41 2m 25s main
main
April 29, 2025 16:41 2m 25s
Add cmakelist for llama unicode (#62) trunk #77: Commit 9c6e89d pushed by jackzhxng
April 29, 2025 16:41 2m 33s main
main
April 29, 2025 16:41 2m 33s
Log hf tokenizer load failure to cerr instead of cout (#61) trunk #76: Commit 0ed8e2e pushed by jackzhxng
April 25, 2025 23:58 2m 11s main
main
April 25, 2025 23:58 2m 11s
Fix pcre2 target (#60) trunk #75: Commit 3f9c458 pushed by jackzhxng
April 25, 2025 17:56 2m 35s main
main
April 25, 2025 17:56 2m 35s
Fix CQS signal. Id] 29511954 -- readability-redundant-string-init in … trunk #74: Commit 5695ef1 pushed by facebook-github-bot
April 24, 2025 18:12 2m 8s main
main
April 24, 2025 18:12 2m 8s
Gate regex lookahead in cmake behind compile flag trunk #73: Commit ffd4749 pushed by facebook-github-bot
April 24, 2025 12:38 2m 33s main
main
April 24, 2025 12:38 2m 33s
Gate regex lookahead in cmake behind compile flag trunk #72: Pull request #59 synchronize by jackzhxng
April 23, 2025 23:57 2m 12s jz/size-regression
jz/size-regression
April 23, 2025 23:57 2m 12s
Gate regex lookahead in cmake behind compile flag trunk #71: Pull request #59 synchronize by facebook-github-bot
April 23, 2025 23:39 1m 47s jz/size-regression
jz/size-regression
April 23, 2025 23:39 1m 47s
Gate regex lookahead in cmake behind compile flag trunk #70: Pull request #59 synchronize by jackzhxng
April 23, 2025 19:25 1m 55s jz/size-regression
jz/size-regression
April 23, 2025 19:25 1m 55s
Pcre2 buck target in third-party (#55) (#58) trunk #69: Commit 13abc73 pushed by jackzhxng
April 23, 2025 19:21 2m 28s main
main
April 23, 2025 19:21 2m 28s
Gate regex lookahead in cmake behind compile flag trunk #68: Pull request #59 opened by jackzhxng
April 23, 2025 19:18 2m 30s jz/size-regression
jz/size-regression
April 23, 2025 19:18 2m 30s
Initialize bos_tok_ = 0 in tokenizer.h (#54) trunk #67: Commit 1e2c6f7 pushed by kirklandsign
April 21, 2025 21:27 2m 4s main
main
April 21, 2025 21:27 2m 4s
Add pcre2 as re2 fallback (#50) (#50) trunk #66: Commit 9378e21 pushed by jackzhxng
April 21, 2025 20:14 2m 9s main
main
April 21, 2025 20:14 2m 9s
Add pcre2 as re2 fallback trunk #65: Pull request #50 synchronize by jackzhxng
April 21, 2025 18:52 1m 58s jz/regex-3
jz/regex-3
April 21, 2025 18:52 1m 58s
Add pcre2 as re2 fallback trunk #64: Pull request #50 synchronize by jackzhxng
April 21, 2025 18:42 2m 29s jz/regex-3
jz/regex-3
April 21, 2025 18:42 2m 29s
Add pcre2 as re2 fallback trunk #63: Pull request #50 synchronize by jackzhxng
April 19, 2025 02:01 1m 59s jz/regex-3
jz/regex-3
April 19, 2025 02:01 1m 59s
Decouple tokenizers from Re2 and use IRegex interface trunk #62: Commit f52b18b pushed by facebook-github-bot
April 19, 2025 00:51 2m 17s main
main
April 19, 2025 00:51 2m 17s