How to pass llama_index vectorstore index RAG context to Perplexity?

[adaw72]

I have tried using the index.as_chat_engine but it doesn't work - I either get error OR it completely ignores the RAG context. Any code samples would be appreciated.

I can call Perplexity fine but not able to pass the index context.

[dosubot[bot]]

Hey there, @adaw72! I'm a bot here to help you out with any bugs, questions, or contributions. Let's tackle this llama_index vectorstore issue together!

To pass LlamaIndex vector store index RAG context to Perplexity, you can follow these steps:

1. Create and Store Vector Embeddings: First, ensure that your data is properly loaded, indexed, and stored using LlamaIndex's framework. This involves creating vector embeddings and storing them in a vector store, such as Chroma. Here's a brief example:

   import chromadb
   from llama_index.core import VectorStoreIndex, SimpleDirectoryReader
   from llama_index.vector_stores.chroma import ChromaVectorStore
   from llama_index.core import StorageContext
   
   # Load documents
   documents = SimpleDirectoryReader("./data").load_data()
   
   # Initialize Chroma client
   db = chromadb.PersistentClient(path="./chroma_db")
   
   # Create collection
   chroma_collection = db.get_or_create_collection("quickstart")
   
   # Assign Chroma as the vector_store
   vector_store = ChromaVectorStore(chroma_collection=chroma_collection)
   storage_context = StorageContext.from_defaults(vector_store=vector_store)
   
   # Create your index
   index = VectorStoreIndex.from_documents(documents, storage_context=storage_context)

2. Integrate with Perplexity: Use the Perplexity class from the llama_index.llms.perplexity module to integrate with Perplexity. Here's a sample code snippet:

   from llama_index.llms.perplexity import Perplexity
   from llama_index.core.llms import ChatMessage
   
   pplx_api_key = "your-perplexity-api-key"  # Replace with your actual API key
   
   llm = Perplexity(
       api_key=pplx_api_key, model="llama-3.1-sonar-small-128k-online", temperature=0.5
   )
   
   messages_dict = [
       {"role": "system", "content": "Be precise and concise."},
       {"role": "user", "content": "Tell me 5 sentences about Perplexity."},
   ]
   
   messages = [ChatMessage(**msg) for msg in messages_dict]
   
   # Get response from the model
   response = llm.chat(messages)
   print(str(response))

3. Pass Context: Ensure that the context from the vector store is properly formatted and passed to Perplexity. The expected input format is a list of dictionaries, where each dictionary represents a message with a "role" and "content". Convert these into ChatMessage objects before passing them to the chat method ^[1].

If you encounter errors with index.as_chat_engine, consider downgrading the llama-index-llms-openai package to version 0.3.10 to resolve compatibility issues ^[2]. Additionally, ensure consistency between the created engine and the one being used, and add a system prompt to provide context about the available information ^[3].

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other}

[adaw72]

The above code executes but does not work. Perplexity returns a result as if it is completely unaware of the index context.