Difference in how embeddings are calculated vs Ollama

[eugeniosegala]

I'm trying to integrate nomic-embed-text with llama.cpp but it's not working as expected and I get these warnings from the Node JS library:

[node-llama-cpp] llm_load_vocab: special_eos_id is not in special_eog_ids - the tokenizer config may be incorrect
[node-llama-cpp] Using this model ("./nomic-embed-text-v1.5.f16.gguf") to tokenize text and then detokenize it resulted in a different text. There might be an issue with the model or the tokenizer implementation. Using this model may not work as intended

withcatai/node-llama-cpp#391

Something very similar is also happening with the Python library.

Basically, the embedding are not calculated properly and I get very irrelevant results when I calculate a cosine similarity.

However, it works perfectly fine with Ollama.

I thought Ollama was using llama.cpp?

I think I'm missing something regarding how Ollama works in combination with llama.cpp and embeddings generation.