A possible inconsistency in the Summarization Tutorial page #9084
IoSonoMarco
started this conversation in
General
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
I'm referring to this page here: https://python.langchain.com/docs/use_cases/summarization, paragraph "Option 2. Map-Reduce".
In case of a long document that exceeds the model context. I found it weird that you first specify that you want to "map each document to an individual summary", and then use a map template like:
map_template = """The following is a set of documents
{docs}
Based on this list of docs, please identify the main themes
Helpful Answer:"""
At the end of the post, you use a text splitter, meaning that every document is passed independently. The map template talks about "a set of documents", but actually is not the case because the mapping is done one by one for every single document split. Am I missing something here?
Beta Was this translation helpful? Give feedback.
All reactions