Unable to retrieve cypher query when it is produced with syntax error from GraphCypherQAChain

[zoitsa93]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

from langchain.chains import GraphCypherQAChain
from langchain.graphs import Neo4jGraph

llm = AzureChatOpenAI(
    deployment_name="gpt-4",
    openai_api_version = "2024-02-15-preview",
    model_version="1106-Preview",
    max_tokens= 4096,
    temperature = 0,
    openai_api_key = openai_api_key,
    azure_endpoint =azure_endpoint
)

graph = Neo4jGraph(url=url, username=username, password=password, sanitize=False)

chain = GraphCypherQAChain.from_llm(llm, graph=graph, verbose=True, return_intermediate_steps=True )

answer=chain.invoke({"query":"Question_to_database"})

Description

I am using the GraphCypherQAChain in python 3.12.4 in order to produce automatically cypher queries using LLM.

While using the return_intermediate_steps=True, when a cypher query is correctly produced from LLM I can store it in a variable.

On the other hand, when then cypher query is produced with a syntax error the chain brakes and so does the code, providing me with a value error that refers to cypher syntax error.
As a result, I am unable to store the wrong query in a variable, as the return_intermediate_steps has never run.

Is there any other way to store a query that has been produced with a syntax error?

System Info

langchain==0.2.16
langchain-community==0.2.16
langchain-core==0.2.38
langchain-openai==0.1.16
langchain-text-splitters==0.2.2

[dosubot[bot]]

Hey @zoitsa93! I'm here to help you with any bugs, questions, or contributions while waiting for a human maintainer. Let's tackle this together!

You can store a query that has been produced with a syntax error when using GraphCypherQAChain in Python by setting the return_aql_query attribute to True. This will include the AQL query in the output dictionary, even if it contains a syntax error.

Here's an example of how you can modify your code to achieve this:

from langchain.chains import GraphCypherQAChain
from langchain.graphs import Neo4jGraph

llm = AzureChatOpenAI(
    deployment_name="gpt-4",
    openai_api_version="2024-02-15-preview",
    model_version="1106-Preview",
    max_tokens=4096,
    temperature=0,
    openai_api_key=openai_api_key,
    azure_endpoint=azure_endpoint
)

graph = Neo4jGraph(url=url, username=username, password=password, sanitize=False)

chain = GraphCypherQAChain.from_llm(
    llm, graph=graph, verbose=True, return_intermediate_steps=True, return_aql_query=True
)

try:
    answer = chain.invoke({"query": "Question_to_database"})
except ValueError as e:
    print(f"Error: {e}")
    # Access the erroneous query from the result
    erroneous_query = e.args[1].get("aql_query", "No query available")
    print(f"Erroneous Query: {erroneous_query}")

In this example, if a ValueError occurs due to a syntax error in the Cypher query, the erroneous query will be stored in the erroneous_query variable and printed out ^[1].

To continue talking to Dosu, mention @dosu.