Skip to content

Commit 68f399e

Browse files
committed
add modern-bert to pre_type check
1 parent 2f5a72f commit 68f399e

File tree

1 file changed

+2
-1
lines changed

1 file changed

+2
-1
lines changed

src/llama-vocab.cpp

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1554,7 +1554,8 @@ void llama_vocab::impl::load(llama_model_loader & ml, const LLM_KV & kv) {
15541554
tokenizer_pre == "jina-v2-es" ||
15551555
tokenizer_pre == "jina-v2-de" ||
15561556
tokenizer_pre == "jina-v2-code" ||
1557-
tokenizer_pre == "roberta-bpe") {
1557+
tokenizer_pre == "roberta-bpe" ||
1558+
tokenizer_pre == "modern-bert") {
15581559
pre_type = LLAMA_VOCAB_PRE_TYPE_GPT2;
15591560
} else if (
15601561
tokenizer_pre == "refact") {

0 commit comments

Comments
 (0)