Streaming the final answer in Create SQL Agent

[gformcreation]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

agent_executor = create_sql_agent(
    llm,
    db=db,
    agent_type="openai-tools",
    suffix = markdown_prompt + SQL_SUFFIX,
    max_iterations=8,
    # verbose=True
)

async def process_events():
    async for event in agent_executor.astream_events(
        {"input": "Can you get me all the records which consists  of high bp, ejection fraction and a death event for me ?."},
        version="v1",
    ):
        kind = event["event"]
        if kind == "on_chat_model_stream":
            content = event["data"]["chunk"]
            if content:
                print(content.content, end="")

asyncio.run(process_events())

Description

I'm attempting to stream the final answer from the agent_executor, but the output currently includes all the AI-generated responses, such as the SQL queries, in addition to the final answer. Does anyone have any suggestions on how to address this?

System Info

System Information

OS: Windows
OS Version: 10.0.22621
Python Version: 3.9.0 (tags/v3.9.0:9cf6752, Oct 5 2020, 15:34:40) [MSC v.1927 64 bit (AMD64)]

Package Information

langchain_core: 0.2.29
langchain: 0.2.12
langchain_community: 0.2.11
langsmith: 0.1.93
langchain_chroma: 0.1.2
langchain_cli: 0.0.24
langchain_experimental: 0.0.63
langchain_huggingface: 0.0.3
langchain_llm_api: 0.0.1
langchain_nomic: 0.1.2
langchain_openai: 0.1.17
langchain_text_splitters: 0.2.2
langserve: 0.2.1

Optional packages not installed

langgraph

Other Dependencies

aiohttp: 3.9.5
async-timeout: 4.0.2
chromadb: 0.5.0
dataclasses-json: 0.6.7
fastapi: 0.110.1
gitpython: 3.1.42
httpx: 0.27.0
huggingface-hub: 0.23.3
jsonpatch: 1.33
langserve[all]: Installed. No version info available.
libcst: 1.4.0
nomic: 3.1.1
numpy: 1.24.3
openai: 1.36.0
orjson: 3.10.1
packaging: 23.2
pillow: 10.3.0
pydantic: 2.7.1
pyproject-toml: 0.0.10
PyYAML: 6.0.1
requests: 2.32.3
sentence-transformers: 3.0.1
SQLAlchemy: 2.0.31
sse-starlette: 1.8.2
sseclient-py: 1.7.2
tenacity: 8.3.0
tiktoken: 0.7.0
tokenizers: 0.19.1
tomlkit: 0.12.5
transformers: 4.41.2
typer[all]: Installed. No version info available.
typing-extensions: 4.10.0
uvicorn: 0.29.0

[dosubot[bot]]

Hey there, @gformcreation! I'm here to assist you with any questions or issues you have. Let's tackle this together!

To stream only the final answer from agent_executor without including all the AI-generated responses such as SQL queries, you can filter the events to identify the final response. Specifically, you should look for the on_chain_end event with the name "Agent". Here is how you can modify your code to achieve this:

agent_executor = create_sql_agent(
    llm,
    db=db,
    agent_type="openai-tools",
    suffix=markdown_prompt + SQL_SUFFIX,
    max_iterations=8,
    # verbose=True
)

async def process_events():
    final_response = ""
    async for event in agent_executor.astream_events(
        {"input": "Can you get me all the records which consists of high bp, ejection fraction and a death event for me ?."},
        version="v1",
    ):
        kind = event["event"]
        if kind == "on_chain_end" and event["name"] == "Agent":
            final_response = event["data"].get("output", {}).get("output", "")
            break

    print(final_response)

asyncio.run(process_events())

In this code, the on_chain_end event with the name "Agent" is used to capture the final response, and the intermediate responses are ignored ^[1][2].

To continue talking to Dosu, mention @dosu.