Filters

[d3buggerdan]

@dosu

def handle_chat(query_str, contract_id):
try:
# Create a filter for contractid using MetadataFilters
filters = MetadataFilters(
filters=[ExactMatchFilter(key="contractid", value=contract_id)]
)

    # Use the retriever to get results filtered by contractid
    retriever = index.as_retriever(filters=filters, similarity_top_k=1000)
    search_results = retriever.retrieve(query_str)  # Perform the search query

    if not search_results:
        return "No relevant results found for this contract ID."

    # Convert the search results to a format suitable for the chat engine
    response = chat_engine.chat(query_str, search_results)
    return str(response
    
    
    i get this error: 'NodeWithScore' object has no attribute 'content'
   
   i want a chat_engine that searches with filters how can i do it

[dosubot[bot]]

The error you're encountering, 'NodeWithScore' object has no attribute 'content', suggests that the chat_engine.chat method is expecting a different format for the search results. The chat_engine.chat method expects a list of NodeWithScore objects as context nodes, which are used to generate the response ^[1].

To resolve this, ensure that the search_results you pass to chat_engine.chat are indeed a list of NodeWithScore objects. Here's how you can adjust your code:

1. Ensure that the retrieve method returns a list of NodeWithScore objects.
2. Pass these results directly to the chat_engine.chat method.

Here's a revised version of your function:

def handle_chat(query_str, contract_id):
    try:
        # Create a filter for contractid using MetadataFilters
        filters = MetadataFilters(
            filters=[ExactMatchFilter(key="contractid", value=contract_id)]
        )

        # Use the retriever to get results filtered by contractid
        retriever = index.as_retriever(filters=filters, similarity_top_k=1000)
        search_results = retriever.retrieve(query_str)  # Perform the search query

        if not search_results:
            return "No relevant results found for this contract ID."

        # Pass the search results directly to the chat engine
        response = chat_engine.chat(query_str, prev_chunks=search_results)
        return str(response)
    except Exception as e:
        return f"An error occurred: {e}"

Make sure that the chat_engine.chat method is correctly set up to handle the prev_chunks parameter, which should be a list of NodeWithScore objects. This should help in integrating the search results with the chat engine.

To continue talking to Dosu, mention @dosu.

[dosubot[bot]]

Yes, you can print out what the chat engine retrieves by using the following code snippet:

print("\n------------- Example Chat Engine  -------------")
chat_response = index.as_chat_engine().chat(
    "What is llama_index?",
    chat_history=[
        ChatMessage(role="system", content="You are an expert on RAG!")
    ],
)
print(chat_response)

Additionally, if you are using the ContextChatEngine, you can configure it to retrieve context from your data and print the responses. Here's an example:

from llama_index.core.memory import ChatMemoryBuffer

memory = ChatMemoryBuffer.from_defaults(token_limit=1500)

chat_engine = index.as_chat_engine(
    chat_mode="context",
    memory=memory,
    system_prompt=(
        "You are a chatbot, able to have normal interactions, as well as talk"
        " about an essay discussing Paul Grahams life."
    ),
)

response = chat_engine.chat("Hello!")
print(response)

response = chat_engine.chat("What did Paul Graham do growing up?")
print(response)

response = chat_engine.chat("Can you tell me more?")
print(response)

Moreover, you can use the instrumentation module from LlamaIndex to define custom event and span handlers to log or print events and spans during the execution. Here's a basic example of how you can set up an event handler to print events:

from llama_index.core.instrumentation.event_handlers import BaseEventHandler

class MyEventHandler(BaseEventHandler):
    @classmethod
    def class_name(cls) -> str:
        """Class name."""
        return "MyEventHandler"

    def handle(self, event) -> None:
        """Logic for handling event."""
        print(event.dict())
        print("")
        with open("log.txt", "a") as f:
            f.write(str(event))
            f.write("\n")

import llama_index.core.instrumentation as instrument

dispatcher = instrument.get_dispatcher()
dispatcher.add_event_handler(MyEventHandler())

This setup will print and log events to a file, allowing you to track what the chat engine retrieves and processes ^[1][2][3].

To continue talking to Dosu, mention @dosu.

[d3buggerdan]

@dosu when i print the retrieval it only gives me nodes corresponding to the filtered contractid but the answer comes from something else outside of the filters??

[dosubot[bot]]

If the retrieval process is correctly filtering nodes by contractid, but the chat engine is generating responses from outside these filtered results, it might be due to how the chat engine is handling the context. Here are a few steps to address this:

1. Ensure Correct Use of prev_chunks: Make sure that the prev_chunks parameter in the chat_engine.chat method is being used correctly. This parameter should be set to the filtered search_results to ensure that only these nodes are used as context. If the _get_nodes method retrieves nodes, it might override prev_chunks, so ensure that the logic is set to prioritize prev_chunks when needed ^[1].

2. Use CitationsContextChatEngine: Consider using the CitationsContextChatEngine, which is designed to ensure that only the filtered context is used by converting the retrieved nodes into a list of documents and passing them to the LLM. This approach helps in maintaining the integrity of the filtered context ^[2].

3. Debug with Instrumentation: Utilize the instrumentation module to track the flow of data through the chat engine. This can help you identify where the context might be getting overridden or ignored. Set up custom event handlers to log the events and spans during the chat engine's execution ^[3].

By following these steps, you should be able to ensure that the chat engine uses only the filtered nodes for generating responses.

To continue talking to Dosu, mention @dosu.

[d3buggerdan]

@dosu give me examples of 1 and 2 with code

[dosubot[bot]]

To address the issue where the chat engine might be generating responses from outside the filtered results, you can use the following examples for steps 1 and 2:

1. Ensure Correct Use of prev_chunks

Make sure that the prev_chunks parameter is used to pass the filtered nodes to the chat engine. Here's an example:

def handle_chat(query_str, contract_id):
    try:
        # Create a filter for contractid using MetadataFilters
        filters = MetadataFilters(
            filters=[ExactMatchFilter(key="contractid", value=contract_id)]
        )

        # Use the retriever to get results filtered by contractid
        retriever = index.as_retriever(filters=filters, similarity_top_k=1000)
        search_results = retriever.retrieve(query_str)  # Perform the search query

        if not search_results:
            return "No relevant results found for this contract ID."

        # Pass the filtered search results to the chat engine using prev_chunks
        response = chat_engine.chat(query_str, prev_chunks=search_results)
        return str(response)
    except Exception as e:
        return f"An error occurred: {e}"

This ensures that the prev_chunks parameter is used to provide the filtered nodes to the chat engine, which should prioritize these nodes when generating a response ^[1].

2. Use CitationsContextChatEngine

The CitationsContextChatEngine is designed to ensure that only the filtered context is used. Here's an example of how you can use it:

from llama_index.packs.cohere_citation_chat.citations_context_chat_engine import CitationsContextChatEngine

# Initialize the CitationsContextChatEngine
citations_chat_engine = CitationsContextChatEngine()

def handle_chat_with_citations(query_str, contract_id):
    try:
        # Create a filter for contractid using MetadataFilters
        filters = MetadataFilters(
            filters=[ExactMatchFilter(key="contractid", value=contract_id)]
        )

        # Use the retriever to get results filtered by contractid
        retriever = index.as_retriever(filters=filters, similarity_top_k=1000)
        search_results = retriever.retrieve(query_str)  # Perform the search query

        if not search_results:
            return "No relevant results found for this contract ID."

        # Use the CitationsContextChatEngine to ensure filtered context is used
        response = citations_chat_engine.chat(query_str, prev_chunks=search_results)
        return str(response)
    except Exception as e:
        return f"An error occurred: {e}"

This example uses the CitationsContextChatEngine to convert the filtered nodes into a list of documents and pass them to the LLM, ensuring that only the filtered context is used in generating the response ^[2].

To continue talking to Dosu, mention @dosu.