How we can use custom opensource huggingface LLM in GraphCypherQAChain in langchain and Neo4J DB

[KaifAhmad1]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

Python 
chain = GraphCypherQAChain.from_llm(
    llm(temperature=0), graph=graph, verbose=True
)

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-36-3461c6f8eea1> in <cell line: 1>()
      1 chain = GraphCypherQAChain.from_llm(
----> 2     llm(temperature=0), graph=graph, verbose=True
      3 )

/usr/local/lib/python3.10/dist-packages/langchain_core/_api/deprecation.py in warning_emitting_wrapper(*args, **kwargs)
    143                 warned = True
    144                 emit_warning()
--> 145             return wrapped(*args, **kwargs)
    146 
    147         async def awarning_emitting_wrapper(*args: Any, **kwargs: Any) -> Any:

TypeError: BaseLLM.__call__() missing 1 required positional argument: 'prompt'

### Description

How we can use custom open source llm from huggingface instead of using ChatOpenAI 
``` Python 
chain = GraphCypherQAChain.from_llm(
    ChatOpenAI(temperature=0), graph=graph, verbose=True
)

System Info

pip == 24.0
python == 3.10.10
Cuda == 12.1

[dosubot[bot]]

🤖

Hey @KaifAhmad1! Great to see you back. Diving into another adventure with LangChain, I see? Let's get cracking!

Based on the error message you're seeing, it seems like the llm function you're trying to use in the GraphCypherQAChain.from_llm method requires a 'prompt' argument that you're not providing.

However, you're interested in using a custom open-source language model from Hugging Face instead of the ChatOpenAI model. To do this, you can follow the example provided in a similar issue in the LangChain repository, where a user successfully used a self-hosted HuggingFace LLaMA2 model. Here's the code they used:

from transformers import pipeline, AutoTokenizer, AutoModelForCausalLM
import warnings

warnings.filterwarnings('ignore')

tokenizer = AutoTokenizer.from_pretrained("meta-llama/Llama-2-7b-chat-hf", use_auth_token=True)
model = AutoModelForCausalLM.from_pretrained("meta-llama/Llama-2-7b-chat-hf", device_map='auto', torch_dtype=torch.float16, use_auth_token=True)
pipe = pipeline("text-generation", model=model, tokenizer=tokenizer, torch_dtype=torch.bfloat16, device_map="auto", max_new_tokens = 720, do_sample=True, top_k=30, num_return_sequences=1, eos_token_id=tokenizer.eos_token_id)

from langchain import HuggingFacePipeline
llm = HuggingFacePipeline(pipeline = pipe, model_kwargs = {'temperature':0.25, 'max_tokens':4000, 'stop_sequence': "\n\n"})

Then, you can use llm in your GraphCypherQAChain.from_llm method:

chain = GraphCypherQAChain.from_llm(llm, graph=graph, verbose=True)

This code logs into HuggingFace, suppresses warnings, loads the model and tokenizer, creates a pipeline, and then wraps the pipeline in a HuggingFacePipeline object, which can be used in LangChain chains.

Please replace "meta-llama/Llama-2-7b-chat-hf" with the name of your custom open-source language model from Hugging Face.

You can find more details in the discussion on the LangChain repository.

I hope this helps! If you have any other questions, feel free to ask.

Sources

• Chat Model API/RetrievalQA chain - Possible to use Self-Hosted HuggingFace LLMs? - Hints? Examples?
• libs/langchain/langchain/chains/question_answering/init.py
• libs/langchain/langchain/chains/graph_qa/cypher.py

About Dosu

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[surajpatil4899]

I am using the above example. But it does not work somehow. It is running on cpu and torch_dtype is the default instead of torch.bfloat16. Here is the below stack trace.

> Entering new GraphCypherQAChain chain...
Generated Cypher:
Task:Generate Cypher statement to query a graph database.
Instructions:
Use only the provided relationship types and properties in the schema.
Do not use any other relationship types or properties that are not provided.
Schema:
Node properties are the following:

Relationship properties are the following:

The relationships are the following:

Note: Do not include any explanations or apologies in your responses.
Do not respond to any questions that might ask anything else than for you to construct a Cypher statement.
Do not include any text except the generated Cypher statement.

The question is:
Who played in Top Gun?

Please generate a Cypher statement to query the graph database and find the answer to the question.
Traceback (most recent call last):
  File "/lib/python3.9/site-packages/langchain_community/graphs/neo4j_graph.py", line 391, in query
    data = session.run(Query(text=query, timeout=self.timeout), params)
  File "/lib/python3.9/site-packages/neo4j/_sync/work/session.py", line 311, in run
    self._auto_result._run(
  File "/lib/python3.9/site-packages/neo4j/_sync/work/result.py", line 166, in _run
    self._attach()
  File "/lib/python3.9/site-packages/neo4j/_sync/work/result.py", line 274, in _attach
    self._connection.fetch_message()
  File "/lib/python3.9/site-packages/neo4j/_sync/io/_common.py", line 180, in inner
    func(*args, **kwargs)
  File "/lib/python3.9/site-packages/neo4j/_sync/io/_bolt.py", line 808, in fetch_message
    res = self._process_message(tag, fields)
  File "/lib/python3.9/site-packages/neo4j/_sync/io/_bolt5.py", line 361, in _process_message
    response.on_failure(summary_metadata or {})
  File "/lib/python3.9/site-packages/neo4j/_sync/io/_common.py", line 247, in on_failure
    raise Neo4jError.hydrate(**metadata)
neo4j.exceptions.CypherSyntaxError: {code: Neo.ClientError.Statement.SyntaxError} {message: Invalid input 'Task': expected
  "ALTER"
  "CALL"
  "CREATE"
  "DEALLOCATE"
  "DELETE"
  "DENY"
  "DETACH"
  "DROP"
  "DRYRUN"
  "ENABLE"
  "FINISH"
  "FOREACH"
  "GRANT"
  "INSERT"
  "LOAD"
  "MATCH"
  "MERGE"
  "NODETACH"
  "OPTIONAL"
  "REALLOCATE"
  "REMOVE"
  "RENAME"
  "RETURN"
  "REVOKE"
  "SET"
  "SHOW"
  "START"
  "STOP"
  "TERMINATE"
  "UNWIND"
  "USE"
  "USING"
  "WITH" (line 1, column 1 (offset: 0))
"Task:Generate Cypher statement to query a graph database."
 ^}

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/lib/python3.9/site-packages/langchain_core/_api/deprecation.py", line 168, in warning_emitting_wrapper
    return wrapped(*args, **kwargs)
  File "/lib/python3.9/site-packages/langchain/chains/base.py", line 600, in run
    return self(args[0], callbacks=callbacks, tags=tags, metadata=metadata)[
  File "/lib/python3.9/site-packages/langchain_core/_api/deprecation.py", line 168, in warning_emitting_wrapper
    return wrapped(*args, **kwargs)
  File "/lib/python3.9/site-packages/langchain/chains/base.py", line 383, in __call__
    return self.invoke(
  File "/lib/python3.9/site-packages/langchain/chains/base.py", line 166, in invoke
    raise e
  File "/lib/python3.9/site-packages/langchain/chains/base.py", line 156, in invoke
    self._call(inputs, run_manager=run_manager)
  File "/lib/python3.9/site-packages/langchain_community/chains/graph_qa/cypher.py", line 337, in _call
    context = self.graph.query(generated_cypher)[: self.top_k]
  File "/lib/python3.9/site-packages/langchain_community/graphs/neo4j_graph.py", line 397, in query
    raise ValueError(f"Generated Cypher Statement is not valid\n{e}")
ValueError: Generated Cypher Statement is not valid
{code: Neo.ClientError.Statement.SyntaxError} {message: Invalid input 'Task': expected
  "ALTER"
  "CALL"
  "CREATE"
  "DEALLOCATE"
  "DELETE"
  "DENY"
  "DETACH"
  "DROP"
  "DRYRUN"
  "ENABLE"
  "FINISH"
  "FOREACH"
  "GRANT"
  "INSERT"
  "LOAD"
  "MATCH"
  "MERGE"
  "NODETACH"
  "OPTIONAL"
  "REALLOCATE"
  "REMOVE"
  "RENAME"
  "RETURN"
  "REVOKE"
  "SET"
  "SHOW"
  "START"
  "STOP"
  "TERMINATE"
  "UNWIND"
  "USE"
  "USING"
  "WITH" (line 1, column 1 (offset: 0))
"Task:Generate Cypher statement to query a graph database."
 ^}

Python - 3.9.13
pip - 21.3.1
langchain==0.2.5

[chaitu17291390]

Don't know if this has been resolved but, the prompt should have Task: and yiu should provide schema