Add a string at the end of response using Langchain

[sherlock-tez]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

from typing import Callable

from langchain.chains.combine_documents import create_stuff_documents_chain
from langchain.chains.history_aware_retriever import create_history_aware_retriever
from langchain.chains.retrieval import create_retrieval_chain
from langchain_community.vectorstores import AzureSearch
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.runnables import ConfigurableFieldSpec

from app.biz.chat_history.runnable_with_message_history import RunnableWithMessageHistory
from app.biz.chat_history.session import create_session_factory
from app.core.models.human_input import InputChat
from app.core.prompt.message_history import contextualize_q_prompt, qa_prompt
from app.core.services.openai import llm


def create_message_history_agent(vectorstore: AzureSearch, context_input_func_call: Callable,
                                 prompt: ChatPromptTemplate = qa_prompt,
                                 search_type: str = "hybrid", agent: str = ""):
    history_aware_retriever = create_history_aware_retriever(
        llm, vectorstore.as_retriever(search_type=search_type, k=3), contextualize_q_prompt
    )

    question_answer_chain = create_stuff_documents_chain(llm, prompt)

    return RunnableWithMessageHistory(
        create_retrieval_chain(history_aware_retriever, question_answer_chain),
        create_session_factory(),
        input_messages_key="input",
        history_messages_key="chat_history",
        output_messages_key="answer",
        history_factory_config=[
            ConfigurableFieldSpec(
                id="message_history",
                annotation=str,
                name="Message History",
                description="Json dumps of list of dictionary from "
                            "https://python.langchain.com/v0.2/docs/concepts/#message-types ",
                default="",
                is_shared=True,
            ),
        ],
    ).with_types(input_type=InputChat)

Description

Hi the task is very simple but I can not find the solution. This is my code. I just want to add for example "Thank you for your question" at the end of every response in when streaming. How to do it? Thank you.

System Info

MacOS 14.4
Python 3.11
langchain-openai = "^0.1.14"
azure-search-documents = "=11.6.0b4"
langchain-community = "=0.2.6"
langchain-core = "=0.2.11"

[dosubot[bot]]

To add a string like "Thank you for your question" at the end of every response when streaming using LangChain, you can modify the code to append this string after the streaming loop. Here is how you can do it for both synchronous and asynchronous streaming:

Synchronous Streaming

chunks = []
for chunk in model.stream("what color is the sky?"):
    chunks.append(chunk)
    print(chunk.content, end="|", flush=True)
print("Thank you for your question", end="|", flush=True)

Asynchronous Streaming

chunks = []
async for chunk in model.astream("what color is the sky?"):
    chunks.append(chunk)
    print(chunk.content, end="|", flush=True)
print("Thank you for your question", end="|", flush=True)

Using LangChain with TextGen

If you are using LangChain with the TextGen integration, you can modify the streaming example as follows:

model_url = "ws://localhost:5005"
from langchain.chains import LLMChain
from langchain.globals import set_debug
from langchain_community.llms import TextGen
from langchain_core.callbacks import StreamingStdOutCallbackHandler
from langchain_core.prompts import PromptTemplate

set_debug(True)

template = """Question: {question}

Answer: Let's think step by step."""

prompt = PromptTemplate.from_template(template)
llm = TextGen(
    model_url=model_url, streaming=True, callbacks=[StreamingStdOutCallbackHandler()]
)
llm_chain = LLMChain(prompt=prompt, llm=llm)
question = "What NFL team won the Super Bowl in the year Justin Bieber was born?"

llm_chain.run(question)

# Custom streaming loop
for chunk in llm.stream(question):
    print(chunk, end="", flush=True)
print("Thank you for your question", end="", flush=True)

Using LangChain with PremAI

If you are using LangChain with the PremAI integration, you can modify the streaming example as follows:

import sys
from langchain_community.chat_models import ChatPremAI
from langchain_core.messages import HumanMessage

# Setup PremAI client
chat = ChatPremAI(project_id=1234, model_name="gpt-4o")

# Custom streaming loop
for chunk in chat.stream("hello how are you"):
    sys.stdout.write(chunk.content)
    sys.stdout.flush()
sys.stdout.write("Thank you for your question")
sys.stdout.flush()

In all cases, the string "Thank you for your question" is printed at the end of the response ^[1][2][3].

To continue talking to Dosu, mention @dosu.