How to force structured output using load_summarize_chain

[hyoinandout]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

None

Description

How can I force structured output using load_summarize_chain?
I would like to have a structured summarization from a long document.
Such like
output = {"output_text":{"title":"Alpha","Author":"Beta","Summarization":"Gamma foo bar ..."}}

System Info

langchain > 1

[dosubot[bot]]

Hello @hyoinandout! I'm here to help you with any questions or issues you have. I'm a bot designed to provide assistance while you wait for a human maintainer. Feel free to ask me anything!

To force structured output using load_summarize_chain in LangChain, you can use the create_structured_output_runnable function to ensure the output is structured. Here is an example of how you can achieve this by specifying a structured schema:

from typing import Optional
from langchain.chains import create_structured_output_runnable
from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.pydantic_v1 import BaseModel, Field

# Define a structured schema using Pydantic
class DocumentSummary(BaseModel):
    title: str = Field(..., description="The title of the document")
    author: str = Field(..., description="The author of the document")
    summary: str = Field(..., description="The summary of the document")

# Initialize your language model
llm = ChatOpenAI(model="gpt-3.5-turbo-0125", temperature=0)

# Create a structured output runnable
structured_llm = create_structured_output_runnable(DocumentSummary, llm, mode="openai-functions")

# Define the prompt
system = '''You are a world-class assistant for summarizing documents in a structured format.'''
prompt = ChatPromptTemplate.from_messages(
    [("system", system), ("human", "{input}")]
)

# Create the chain
chain = prompt | structured_llm

# Use the chain to summarize a document
document = {
    "input": "This is the content of the document that needs to be summarized."
}
summary = chain.invoke(document)
print(summary)

In this example, the DocumentSummary class is defined to create a structured schema. This schema is then used in the create_structured_output_runnable function to ensure the summarization output is structured according to the specified format. The ChatPromptTemplate is used to define the prompt, and the chain is created by combining the prompt and the structured LLM. Finally, the chain is used to summarize a document, and the structured summary is printed ^[1][2].

Details

About Dosu
This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

To continue the conversation, mention @dosu.

[dosubot[bot]]

Yes, using the RefineChain can be more appropriate for handling large documents while ensuring structured output. Here's how you can achieve structured output in JSON format using the RefineChain:

1. Define the structured schema using Pydantic.
2. Create the initial and refine LLM chains with prompts that ensure JSON output.
3. Combine these chains into a RefineChain.

Here's an example:

from typing import Optional
from langchain.chains import RefineDocumentsChain, LLMChain
from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate, PromptTemplate
from langchain_core.pydantic_v1 import BaseModel, Field

# Define a schema for the JSON output
class SummarySchema(BaseModel):
    title: str = Field(..., description="The title of the document")
    author: str = Field(..., description="The author of the document")
    summary: str = Field(..., description="The summary of the document")

# Initialize the language model
llm = ChatOpenAI(model="gpt-3.5-turbo-0125", temperature=0)

# Create a structured output runnable
structured_llm = create_structured_output_runnable(SummarySchema, llm, mode="openai-json")

# Define the initial prompt
initial_system_message = '''You are a world-class assistant for summarizing documents in a structured JSON format.'''
initial_prompt = ChatPromptTemplate.from_messages(
    [("system", initial_system_message), ("human", "{input}")]
)
initial_llm_chain = LLMChain(llm=structured_llm, prompt=initial_prompt)

# Define the refine prompt
refine_system_message = '''You are a world-class assistant for refining summaries in a structured JSON format.'''
refine_prompt = ChatPromptTemplate.from_messages(
    [("system", refine_system_message), ("human", "{input}")]
)
refine_llm_chain = LLMChain(llm=structured_llm, prompt=refine_prompt)

# Create the RefineDocumentsChain
chain = RefineDocumentsChain(
    initial_llm_chain=initial_llm_chain,
    refine_llm_chain=refine_llm_chain,
    document_variable_name="input",
    initial_response_name="prev_response",
)

# Use the chain to process a long document
document = "Your long document text here"
result = chain.invoke({"input": document})
print(result)

This setup ensures that the RefineChain processes the document in chunks, refining the summary iteratively, and outputs the result in a structured JSON format according to the specified schema ^[1][2].

Details

About Dosu
This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

To continue the conversation, mention @dosu.

[hyoinandout]

Make this compatible to current version

[dosubot[bot]]

To make the provided code compatible with the current version of LangChain, you can use the RefineDocumentsChain implementation. Here is the updated code:

1. Install langchain:

   pip install -qU langchain

2. Define the RefineDocumentsChain implementation:

   from typing import Optional
   from langchain.chains import RefineDocumentsChain, LLMChain
   from langchain_core.prompts import ChatPromptTemplate, PromptTemplate
   from langchain_core.pydantic_v1 import BaseModel, Field
   from langchain_openai import ChatOpenAI
   
   # Define a schema for the JSON output
   class SummarySchema(BaseModel):
       title: str = Field(..., description="The title of the document")
       author: str = Field(..., description="The author of the document")
       summary: str = Field(..., description="The summary of the document")
   
   # Initialize the language model
   llm = ChatOpenAI(model="gpt-3.5-turbo-0125", temperature=0)
   
   # Create a structured output runnable
   structured_llm = create_structured_output_runnable(SummarySchema, llm, mode="openai-json")
   
   # Define the initial prompt
   initial_system_message = '''You are a world-class assistant for summarizing documents in a structured JSON format.'''
   initial_prompt = ChatPromptTemplate.from_messages(
       [("system", initial_system_message), ("human", "{input}")]
   )
   initial_llm_chain = LLMChain(llm=structured_llm, prompt=initial_prompt)
   
   # Define the refine prompt
   refine_system_message = '''You are a world-class assistant for refining summaries in a structured JSON format.'''
   refine_prompt = ChatPromptTemplate.from_messages(
       [("system", refine_system_message), ("human", "{input}")]
   )
   refine_llm_chain = LLMChain(llm=structured_llm, prompt=refine_prompt)
   
   # Create the RefineDocumentsChain
   chain = RefineDocumentsChain(
       initial_llm_chain=initial_llm_chain,
       refine_llm_chain=refine_llm_chain,
       document_variable_name="input",
       initial_response_name="prev_response",
   )
   
   # Use the chain to process a long document
   document = "Your long document text here"
   result = chain.invoke({"input": document})
   print(result)

This setup ensures that the RefineDocumentsChain processes the document in chunks, refining the summary iteratively, and outputs the result in a structured JSON format according to the specified schema ^[1][2].

To continue talking to Dosu, mention @dosu.

[hyoinandout]

This outputs an error

E       AttributeError: 'str' object has no attribute 'page_content'

[dosubot[bot]]

The error 'str' object has no attribute 'page_content' indicates that the RefineDocumentsChain is expecting a Document object with a page_content attribute, but it is receiving a string instead. To resolve this, you need to ensure that the input documents are properly wrapped in Document objects.

Here is the updated code to ensure that the documents are correctly formatted:

from typing import Optional
from langchain.chains import RefineDocumentsChain, LLMChain
from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate, PromptTemplate
from langchain_core.pydantic_v1 import BaseModel, Field
from langchain_core.documents import Document

# Define a schema for the JSON output
class SummarySchema(BaseModel):
    title: str = Field(..., description="The title of the document")
    author: str = Field(..., description="The author of the document")
    summary: str = Field(..., description="The summary of the document")

# Initialize the language model
llm = ChatOpenAI(model="gpt-3.5-turbo-0125", temperature=0)

# Create a structured output runnable
structured_llm = create_structured_output_runnable(SummarySchema, llm, mode="openai-json")

# Define the initial prompt
initial_system_message = '''You are a world-class assistant for summarizing documents in a structured JSON format.'''
initial_prompt = ChatPromptTemplate.from_messages(
    [("system", initial_system_message), ("human", "{input}")]
)
initial_llm_chain = LLMChain(llm=structured_llm, prompt=initial_prompt)

# Define the refine prompt
refine_system_message = '''You are a world-class assistant for refining summaries in a structured JSON format.'''
refine_prompt = ChatPromptTemplate.from_messages(
    [("system", refine_system_message), ("human", "{input}")]
)
refine_llm_chain = LLMChain(llm=structured_llm, prompt=refine_prompt)

# Create the RefineDocumentsChain
chain = RefineDocumentsChain(
    initial_llm_chain=initial_llm_chain,
    refine_llm_chain=refine_llm_chain,
    document_variable_name="input",
    initial_response_name="prev_response",
)

# Wrap the document text in a Document object
document_text = "Your long document text here"
document = Document(page_content=document_text)

# Use the chain to process the document
result = chain.invoke({"input": document})
print(result)

This ensures that the document text is wrapped in a Document object, which has the page_content attribute, resolving the error and allowing the RefineDocumentsChain to process the document correctly ^[1][2].

To continue talking to Dosu, mention @dosu.