Llama index seems to not be using embeddings

[brpaz]

Hello.

I am writing a python script to index and answer questions of my Obsidian vault.

But it seems the embeddings are not being properly used. When I ask a question, the answer is a general answer that seems to be coming straight from the underlying model and not from the index of my Obsidian documents.

What I am missing here?

I am not Ollama as LLM and HuggingFace BAAI/bge-small-en-v1.5 as embed model.

import logging
import sys
import os
from llama_index.core import (
    VectorStoreIndex,
    StorageContext,
    Settings,
    load_index_from_storage,
)
from llama_index.readers.obsidian import ObsidianReader
from llama_index.llms.ollama import Ollama
from llama_index.embeddings.huggingface import HuggingFaceEmbedding

# Configure logging
logging.basicConfig(stream=sys.stdout, level=logging.INFO)
logging.getLogger().addHandler(logging.StreamHandler(stream=sys.stdout))

embed_model = HuggingFaceEmbedding(model_name="BAAI/bge-small-en-v1.5")

# Load environment variable for Obsidian vault path
vault_path = os.environ.get("OBSIDIAN_VAULT_PATH")
if not vault_path:
    raise ValueError("Please set OBSIDIAN_VAULT_PATH environment variable")

# Define storage path for persistence
storage_path = "./index_storage"

# Initialize LLM and query engine
llm = Ollama(model="llama3.2:3b", request_timeout=60.0)
llm.system_prompt = (
    "You are a highly knowledgeable AI assistant designed to provide accurate and concise answers based on a given set of documents. "
    "Your responses should be solely derived from the provided context and should not include external information. "
    "If the information is not present in the context, inform the user that you cannot provide an answer. "
    "You should be polite, clear, and aim to assist with any questions related to the documents available. "
    "In cases where the query is vague or unclear, politely ask the user for more details or clarification."
)

Settings.llm = llm
Settings.embed_model = embed_model

# Check if a persisted index exists
if os.path.exists(storage_path):
    # Load existing index
    storage_context = StorageContext.from_defaults(persist_dir=storage_path)
    index = load_index_from_storage(storage_context)
else:
    # Create a new index and persist it
    documents = ObsidianReader(vault_path).load_data()  # Load documents
    index = VectorStoreIndex.from_documents(documents, embed_model=embed_model)
    index.storage_context.persist(persist_dir=storage_path)

query_engine = index.as_query_engine(embed_model=embed_model)

# Perform a sample query
res = query_engine.query(
    "What do you know about Accessing Homelab from an external network?"
)
print(res)

[dosubot[bot]]

Hey @brpaz! I'm here to help you with any bugs, questions, or troubleshooting you need assistance with. Let's work together to solve this issue.

I found a similar closed issue where users experienced problems with embeddings not being properly generated on the nodes ^[1]. However, no solution was found in the comments of that issue.

In your case, ensure that the embed_model is correctly applied when loading the index from storage. When using load_index_from_storage, you need to explicitly specify the embedding model using the embed_model parameter. If not specified, it defaults to a MockEmbedding or the default set in Settings. Make sure to pass the embed_model like this: load_index_from_storage(..., embed_model=embed_model) to ensure the correct embeddings are used ^[2].

Additionally, verify that the query_engine is correctly initialized with the embed_model. If the embeddings are not applied correctly, the query might return general answers instead of document-specific ones.

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other}

[brpaz]

@dosu setting embed_model on index retriveval doesnt seem to work.

[dosubot[bot]]

When setting the embed_model on index retrieval doesn't seem to work, ensure that the embed_model is correctly specified at every stage where it's needed. Here are a few things to check:

1. Loading the Index: When using load_index_from_storage, make sure to pass the embed_model parameter explicitly:

   index = load_index_from_storage(storage_context, embed_model=embed_model)

2. Query Engine Initialization: When creating the query engine, ensure that the embed_model is also specified:

   query_engine = index.as_query_engine(embed_model=embed_model)

3. Global Settings: If you want to set a global default, ensure that the Settings.embed_model is set correctly:

   from llama_index.core import Settings
   Settings.embed_model = embed_model

If these steps are correctly followed and the issue persists, it might be worth checking if there are any specific errors or logs that could provide more insight into why the embeddings are not being applied as expected.

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other}