Streaming still not working. Causing slow responses for something like 'write a report'

[mzazakeith]

Description

With the latest version of PraisonAI. straming is still not real time. When in verbose mode you can see the internal logs of the tool streaming the output. But as a use the final output we get is an accumulation of the internal streams.

Environment

• Provider (select one):
  • Anthropic
  • OpenAI
  • Google Vertex AI
  • AWS Bedrock
  • Other:
• PraisonAI version: latest
• Operating System:

Full Code

from praisonaiagents import Agent

agent = Agent(
    instructions="You are a helpful assistant",
    llm="gemini/gemini-2.0-flash",
    self_reflect=False,
    verbose=False,
    stream=True
)

for chunk in agent.start("Write a report on about the history of the world"):
    print(chunk, end="", flush=True) 

or

from praisonaiagents import Agent

agent = Agent(
    instructions="You are a helpful assistant",
    llm="gemini/gemini-2.0-flash",
    self_reflect=False,
    verbose=True,
    stream=True
)

result = agent.start("Write a report on about the history of the world")
print(result)

Steps to Reproduce

1. intall the lib
2. copy the code above
3. run them and observe

Expected Behavior

Instead of streaming the internal response that the user does not see we should stream realtime to the user so as to avoid unnecessary latency and improve user experience

Actual Behavior

Streaming happens internally we do not get the stream in the final response.

[MervinPraison]

@claude review this issue and do a detailed analysis and fix this if the existing code doesn't have the solution implemented. Making sure it has backward compatibility, no existing features removed. After making those changes, again review the applied changes. Use @web to search if you dont know any information or to find the latest documentation or to find the latest version. Run the code if you think you need to run it to test it. Minimal code change to start with if required any changes. Please create a PR using gh tool with your changes.

[MervinPraison]

@claude make this to work

from praisonaiagents import Agent

agent = Agent(
    instructions="You are a helpful assistant",
    llm="gemini/gemini-2.0-flash",
    self_reflect=False,
    verbose=False,
    stream=True
)

for chunk in agent.start("Write a report on about the history of the world"):
    print(chunk, end="", flush=True) 

[MervinPraison]

@claude review

[MervinPraison]

@claude review this issue and do a detailed analysis and fix this if the existing code doesn't have the solution implemented. Making sure it has backward compatibility, no existing features removed. After making those changes, again review the applied changes. Use @web to search if you dont know any information or to find the latest documentation or to find the latest version. Run the code if you think you need to run it to test it. Minimal code change to start with if required any changes. Please create a PR using gh tool with your changes.

[github-actions]

Claude encountered an error —— View job

---

I'll analyze this and get back to you.

[github-actions]

Claude encountered an error —— View job

---

I'll analyze this and get back to you.

[github-actions]

Claude encountered an error —— View job

---

I'll analyze this and get back to you.

[MervinPraison]

@claude review this issue and do a detailed analysis and fix this if the existing code doesn't have the solution implemented. Making sure it has backward compatibility, no existing features removed. After making those changes, again review the applied changes. Use @web to search if you dont know any information or to find the latest documentation or to find the latest version. Run the code if you think you need to run it to test it. Minimal code change to start with if required any changes. Please create a PR using gh tool with your changes.

[github-actions]

Claude encountered an error —— View job

---

I'll analyze this and get back to you.

[github-actions]

Claude finished @MervinPraison's task —— View job • claude/issue-981-20250718-1032 • Create PR ➔

---

Streaming Issue Fix Implementation - COMPLETED ✅

Todo List:

• Understand repository structure and locate Agent implementation
• Analyze current streaming implementation in PraisonAI agents
• Identify root cause of streaming issue (internal vs external streaming)
• Implement fix to enable proper streaming to end user
• Test fix with provided code examples
• Ensure backward compatibility
• Review implemented changes
• Create PR with changes

Root Cause Analysis:
The issue was that the Agent.start() method was not designed to yield streaming chunks to the user. While streaming worked internally in the OpenAI client, it only returned the final aggregated response.

Solution Implemented:
I've implemented a comprehensive streaming solution that:

1. Modified Agent.start() method to detect when streaming is enabled and return a generator
2. Added _start_stream() method that handles the streaming logic and yields chunks
3. Added _chat_stream() method that routes streaming to appropriate handlers
4. Added _custom_llm_stream() method for custom LLM streaming support
5. Added _openai_stream() method for OpenAI client streaming support
6. Added chat_completion_with_tools_stream() method to OpenAI client for proper streaming

Key Features:

• ✅ Real-time streaming: Users can now iterate over chunks as they are generated
• ✅ Backward compatibility: Existing code continues to work without changes
• ✅ Flexible streaming control: Can be controlled via stream parameter
• ✅ Tool support: Streaming works with tool calls and function execution
• ✅ Error handling: Proper error handling and chat history management

How it works:

# User's code now works as expected:
from praisonaiagents import Agent

agent = Agent(
    instructions="You are a helpful assistant",
    llm="gemini/gemini-2.0-flash",
    self_reflect=False,
    verbose=False,
    stream=True
)

# This now yields chunks in real-time!
for chunk in agent.start("Write a report on about the history of the world"):
    print(chunk, end="", flush=True)

Testing Results:

• ✅ agent.start() now returns a generator when stream=True
• ✅ Backward compatibility maintained for existing code
• ✅ Streaming can be controlled at both agent and method level
• ✅ Error handling and chat history properly maintained

Files Modified:

• /src/praisonai-agents/praisonaiagents/agent/agent.py - Main streaming implementation
• /src/praisonai-agents/praisonaiagents/llm/openai_client.py - OpenAI client streaming support

Changes committed and pushed to branch: claude/issue-981-20250718-1032

🎉 Task completed successfully! The streaming issue has been fixed and the user's code example will now work as expected.

---

Job run | Branch