docs/tutorials/rag/

[giscus[bot]]

docs/tutorials/rag/

One of the most powerful applications enabled by LLMs is sophisticated question-answering (Q&A) chatbots. These are applications that can answer questions about specific source information. These applications use a technique known as Retrieval Augmented Generation, or RAG.

https://python.langchain.com/docs/tutorials/rag/

[ksairos]

If you're using Chroma during the query analysis, don't forget to change the filter for vector_store.similarity_search() function in the def retrieve(state: State) node.

Instead of
filter=lambda doc: doc.metadata.get("section") == query["section"], which causes a Expected type 'dict[str, str] | None', got '(doc: Any) -> bool' instead error, you can use {"section": query["section"]}.

[korpog]

You have to run pip install beautifulsoup4, otherwise there will be an import error

[JoshuaMichaelHanson]

On Windows 11 running Python 3.13.1 needed to change to a pydantic BaseModel to get the analyze_query to work. Keep getting ValueError: no signature found for builtin type <class 'dict'> when running the query = structured_llm.invoke(state["question"])

Updated code as follows and works. The answer is slightly different but by adjusting the query was able to come close

from typing import Literal
from pydantic import BaseModel, Field

Define the Search schema using Pydantic instead of TypedDict

class Search(BaseModel):
"""Search query schema."""
query: str = Field(description="Search query to run.")
section: Literal["beginning", "middle", "end"] = Field(
description="Section to query."
)

for step in graph.stream(
{"question": "What are Challenges of task decomposition say at the end?"},
stream_mode="updates",
):
print(f"{step}\n\n----------------\n")

If anyone knows how to make it work with the TypeDict vs BaseModel would be interested in the answer, otherwise hope this helps someone else if they get stuck. Cheers!

[0JCRG0]

The documentation is not up-to-date:

Traceback (most recent call last):
File "/Users/juanreyesgarcia/Dev/Python/LLMSandbox/main.py", line 58, in
graph_builder = StateGraph(State).add_sequence([retrieve, generate]) # type: ignore
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'StateGraph' object has no attribute 'add_sequence'

[JoshuaMichaelHanson]

Are you sure you have all the correct dependencies installed? Just did the tutorial a week ago and everything worked fine-ish, see other comment.

pip install -U langgraph
pip show langgraph
from langgraph.graph import START, StateGraph

If no error occurs, the package is installed correctly

pip check langgraph

make sure you are using a virtual environment to manage dependencies cleanly

python -m venv venv
source venv/bin/activate # On Windows: venv\Scripts\activate
pip install -U langgraph

[0JCRG0]

It is probably the version? This is what I have langgraph==0.2.39

But this worked:

graph_builder = StateGraph(State)
graph_builder.add_node("retrieve", retrieve)
graph_builder.add_node("generate", generate)

graph_builder.add_edge(START, "retrieve")
graph_builder.add_edge("retrieve", "generate")
graph_builder.add_edge("generate", END)
graph = graph_builder.compile()

response = graph.invoke({"question": "What is Task Decomposition?"})
print(response["answer"])

[ralphy1976]

i just followed the tutorial, i had no issues with the "graph_builder = StateGraph(State).add_sequence([retrieve, generate])" ONCE i remembered to install langgraph....

[kgasioro]

Use of an NVIDIA chat models did not work for me, as returned message indicated model_provider='nvidia' is not supported for init_chat_model() using latest version, even though online documentation states that it is.

[LiuDiliDili]

for step in graph.stream(
{"question": "What does the end of the post say about Task Decomposition?"},
stream_mode="updates",
):
print(f"{step}\n\n----------------\n")
NotImplementedError (When following this tutorial, I encountered an error when executing the last piece of code mentioned above.)
My LangChain version as below：
langchain 0.3.19
langchain-community 0.3.18
langchain-core 0.3.37
langchain-text-splitters 0.3.6
langchain-unstructured 0.1.6
langdetect 1.0.9
langgraph 0.2.74
langgraph-checkpoint 2.0.16
langgraph-sdk 0.1.53
langsmith 0.3.8

[LiuDiliDili]

\langchain\Lib\site-packages\langchain_core\language_models\chat_models.py:1189, in BaseChatModel.bind_tools(self, tools, **kwargs)
1182 def bind_tools(
1183 self,
1184 tools: Sequence[
(...)
1187 **kwargs: Any,
1188 ) -> Runnable[LanguageModelInput, BaseMessage]:
-> 1189 raise NotImplementedError

[LiuDiliDili]

llm = ChatOpenAI(
temperature=0.95,
model="GLM-4-Air",
openai_api_key="***",
openai_api_base="https://open.bigmodel.cn/api/paas/v4/",
model_kwargs={ "response_format": { "type": "json_object" } }
)
ChatSparkLLM still does not support bind_tools. Using ZhiPuAi in this way is useful. I didn't encountered this mistake again.

[mg1094]

loader = WebBaseLoader(
web_paths=("https://lilianweng.github.io/posts/2023-06-23-agent/",),
bs_kwargs=dict(
parse_only=bs4.SoupStrainer(
class_=("post-content", "post-title", "post-header")
)
),
)
Error:HTTPSConnectionPool(host='lilianweng.github.io', port=443): Max retries exceeded with url: /posts/2023-06-23-agent/ (Caused by ProtocolError('Connection aborted.', ConnectionResetError(54, 'Connection reset by peer')))

[devo-q-fontaine]

When running this code:

****async for step in app.astream(
    {"contents": [doc.page_content for doc in split_docs]},
    {"recursion_limit": 10},
):
    print(list(step.keys()))****

I get the following error:

AttributeError                            Traceback (most recent call last)
Cell In[68], line 1
----> 1 async for step in app.astream(
      2     {"contents": [doc.page_content for doc in split_docs]},
      3     {"recursion_limit": 10},
      4 ):
      5     print(list(step.keys()))

File c:\Users\qfontaine\rag_env\Lib\site-packages\langgraph\pregel\__init__.py:2626, in Pregel.astream(self, input, config, stream_mode, output_keys, interrupt_before, interrupt_after, debug, subgraphs)
   2620 # Similarly to Bulk Synchronous Parallel / Pregel model
   2621 # computation proceeds in steps, while there are channel updates
   2622 # channel updates from step N are only visible in step N+1
   2623 # channels are guaranteed to be immutable for the duration of the step,
   2624 # with channel updates applied only at the transition between steps
   2625 while loop.tick(input_keys=self.input_channels):
-> 2626     async for _ in runner.atick(
   2627         loop.tasks.values(),
   2628         timeout=self.step_timeout,
   2629         retry_policy=self.retry_policy,
   2630         get_waiter=get_waiter,
   2631     ):
   2632         # emit output
   2633         for o in output():
   2634             yield o
...
     94             return model_encoding_name
     96 if encoding_name is None:

AttributeError: 'NoneType' object has no attribute 'startswith'
During task with name 'collect_summaries' and id '643a0160-cef7-cc52-4dd8-fc21399daa47'

Could I please get some help with this as I am quite lost to what is going wrong.

[tapegoji]

Do we have the use Langsmith in this tutorial? seems beyond the point.

[ralphy1976]

i found that if you do not use the line: prompt = hub.pull("rlm/rag-prompt")

you won't have a langsmith warning.

You can use your own prompt as shown in the later part of the tutorial.

[gopesh97]

Earlier there was a conversation retrieval chain, which offered map-reduce llm calls. Is there a tutorial that implements rag with map-reduce and doesnt use langsmith?

[palladium123]

It seems the line 'response = llm.invoke(messages)' do not return the correct class.

When I run this block of code (also in the tutorial), I get the correct type for example_messages:

from langchain import hub

prompt = hub.pull("rlm/rag-prompt")

example_messages = prompt.invoke(
{"context": "(context goes here)", "question": "(question goes here)"}
).to_messages()

assert len(example_messages) == 1
print(example_messages[0].content) # returns: <class 'langchain_core.messages.human.HumanMessage'>

But when I run this:

def generate(state:State):
docs_content = "\n\n".join(doc.page_content for doc in state["context"])
messages = prompt.invoke({"question": state["question"], "context": docs_content}).to_messages()
response = llm.invoke(messages)
print(type(response)) # This returns <class 'str>
return {'answer': response.content}

This fits with the error trace I got:

'str' object has no attribute 'content'
File "D:\python\phi4 dictate\RAG_test.py", line 73, in generate
return {"answer": response.content}
^^^^^^^^^^^^^^^^
File "D:\python\phi4 dictate\RAG_test.py", line 80, in
response = graph.invoke({"question": "What is Task Decomposition?"})
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'str' object has no attribute 'content'

Some help would be appreciated.

[ralphy1976]

I had the same problem.

return {"answer": response.content} is not the write syntax.

return {"answer": response} is the write syntax.

to solve it i looked for a similar line of code. luckily there was one 3 lines above: return {"context": retrieved_docs}

after that it worked without any problems.

hope this helps.

[ralphy1976]

right syntax, not write syntax....

[ralphy1976]

If you are interested in being able to type your question directly in the terminal, you can do so by using the following:

Before the chatbot template, i wrote: question = input("ask your question: ")

Then, where i would normally type my question in the code, i wrote: response = graph.invoke({"question": question})

so when i run my code, i can ask whatever question i want.

[clickyman1142]

How can the model decide the "section" in the analyze_query function? Does it look into the documents or randomly pick from those 3 sections? Seems like it is randomly.