Proposal: Streaming Into the Future with Prism Events

[sixlive]

Hey Prism community! 👋

I've been thinking a lot about our streaming implementation lately, and I want to share a proposal for a pretty significant refactor that could make building real-time AI applications way better. But first, I need your feedback.

The Current State (And Why I Think We Can Do Better)

Right now, our streaming system works around "Chunks" – objects that contain mixed data types. If you've built streaming UIs with Prism, you've probably written code like this:

foreach ($stream as $chunk) {
    if ($chunk->text) {
        $text .= $chunk->text;
    }
    if ($chunk->toolCalls) {
        // Handle tool calls
    }
    // Usage data... somewhere?
}

This works, but I've been hearing from developers that it creates some challenges:

1. Usage tracking is inconsistent - Some providers include it, others don't, and it's in different places
2. Mixed data types are confusing - One chunk might have text, tools, and metadata all mixed together
3. No proper SSE format - Building web UIs requires custom formatting
4. State management complexity - Clients have to track what's happening across multiple chunks

Question for you: Does this match your experience? What other pain points have you hit with the current streaming approach?

The Proposal: Event-Driven Streaming

What I'm proposing is a complete refactor to an event-based system. Instead of chunks with mixed data, you'd get specific events for specific things happening in the stream.

Here's what a typical conversation would look like:

event: meta
data: {"id":"msg_123","model":"gpt-4","timestamp":1234567890}

event: text_start
data: {"id":"msg_123","timestamp":1234567891}

event: text_delta
data: {"id":"msg_123","delta":"I am","timestamp":1234567892}

event: text_delta
data: {"id":"msg_123","delta":" Claude","timestamp":1234567893}

event: text_end
data: {"id":"msg_123","timestamp":1234567894}

event: finish
data: {"id":"msg_123","reason":"stop","usage":{"prompt_tokens":10,"completion_tokens":3,"total_tokens":13},"timestamp":1234567895}

Key Changes

1. Proper SSE format with event: fields and snake_case naming
2. Dedicated events for text, tools, thinking, and metadata
3. Consistent usage tracking - Always in the finish event, always there
4. Clear lifecycle events - start/delta/end patterns for everything

The Magic: asEventStreamResponse()

The part I'm most excited about is this new method:

// Current implementation
Route::get('chat', function () {
    return response()->stream(function() use ($request) {
        $stream = Prism::text()
            ->using('openai', 'gpt-5')
            ->withPrompt(request('message'))
            ->asStream();

        foreach ($stream as $chunk) {
            // Manual SSE formatting
            echo "data: " . json_encode(['text' => $chunk->text]) . "\n\n";
            ob_flush();
            flush();
        }
    }, 200, [
        'Content-Type' => 'text/event-stream',
        'Cache-Control' => 'no-cache',
        // ... more headers
    ]);
});

// Proposed approach (one method call)
Route::get('chat', function () {
  return Prism::text()
      ->using('openai', 'gpt-5')
      ->withPrompt(request('message'))
      ->onStreamEnd(function (Collection $responseMessages) {
          // Persist messages or something
      })
      ->asEventStreamResponse();
});

This automatically handles all the SSE formatting, headers, and streaming mechanics.

Tool Calls Get First-Class Treatment

For those of you building tool-enabled apps, tool calls would get proper event sequences:

event: tool_call_start
data: {"id":"call_001","name":"weather","timestamp":1234567891}

event: tool_call_delta
data: {"id":"call_001","arguments":"{\"city\"","timestamp":1234567892}

event: tool_call_delta
data: {"id":"call_001","arguments":":\"Paris\"}","timestamp":1234567893}

event: tool_call_end
data: {"id":"call_001","timestamp":1234567894}

event: tool_result_start
data: {"id":"result_001","call_id":"call_001","timestamp":1234567895}

event: tool_result_end
data: {"id":"result_001","timestamp":1234567896}

You can watch tool arguments being built incrementally and handle the execution lifecycle properly.

Thinking Events for Reasoning Models

With reasoning models becoming more common, we'd have dedicated events for chain-of-thought:

event: thinking_start
data: {"id":"msg_456","timestamp":1234567891}

event: thinking_delta
data: {"id":"msg_456","delta":"Let me analyze this...","timestamp":1234567892}

event: thinking_end
data: {"id":"msg_456","timestamp":1234567893}

The Developer Experience

On the frontend, you could build UIs like this:

const eventSource = new EventSource('/api/chat-stream?message=Hello');

eventSource.addEventListener('text_delta', (event) => {
    const data = JSON.parse(event.data);
    appendText(data.delta);
});

eventSource.addEventListener('tool_call_start', (event) => {
    const data = JSON.parse(event.data);
    showToolExecution(data.name);
});

eventSource.addEventListener('finish', (event) => {
    const data = JSON.parse(event.data);
    updateUsageStats(data.usage); // Always available!
});

No more weird state management or hunting for usage data across different chunk types.

Backward Compatibility

I want to be transparent: this would be a breaking change. The asStream() method would return StreamEvent objects instead of Chunk objects. However, the method signature stays the same, and the migration path is pretty straightforward:

// Old way
foreach ($stream as $chunk) {
    $text .= $chunk->text;
}

// New way
foreach ($stream as $event) {
    if ($event instanceof TextDeltaEvent) {
        $text .= $event->getData()['delta'];
    }
}

The Vision

My goal is to make Prism the best way to build streaming AI applications. This event system would give you:

• Consistent experience across all LLM providers
• Production-ready SSE out of the box
• Granular control over exactly what you want to handle
• Better debugging with clear event sequences

But I don't want to build this in a vacuum. Your feedback will shape how this actually works.

---

This is a proposal, not a commitment. Based on community feedback, the actual implementation might look different. But I wanted to share the direction I'm thinking and get your input before diving in.

[kauffinger]

I like this. We need an end-to-end way to give to developers so they can build great interfaces.

Currently, when using Livewire, I am basically abstracting a way to pass through chunks to the frontend to then process them in JS.
I pass through the type of the chunk & its content in the field it would be in the chunk.

It makes a lot of sense to give the dev an end-to-end way of consuming the events in the frontend. I've struggled myself moving from just streaming the text to streaming whole events. I tried to keep the frontend processing to a minimum, which was wrong. Today, with thinking and tool calls, you really have to process the full stream in the frontend. There is no good in-between imo.

I'd also like to see a default frontend implementation for people to reference in different stacks, kind of like @pushpak1300 or my starter kits.

[uchm4n]

That's an excellent proposal. I completely agree that an event-based system makes a lot more sense and would provide a solid, standardized architecture.
I especially like the asEventStreamResponse() method. It feels very clean and intuitive—a truly "Laravel" way to handle streaming. The dedicated events for text and tools, along with consistent usage tracking, would definitely solve many of the pain points you mentioned. I think it would significantly simplify state management and make building client-side UIs much more straightforward.
This looks like a massive improvement.

[avengerweb]

Streaming-focused feedback from production use

I’m squarely focused on production apps in Laravel, not cloning OpenRouter/LiteLLM in PHP. Prism was a great starting point—I built a small lib on top that focuses only on Streaming and Structured Outputs (kept structured outputs unchanged from Prism).

A few lessons from running this at scale in PHP/Laravel:

• PHP runtime realities: long-running requests + dropped connections shouldn’t kill a workflow (chat turn), but PHP’s request/response model makes that tricky. Agentic workflows can run minutes, which doesn’t mesh well with FPM.
• Scaling: with 1000+ active chats, you’ll exhaust FPM workers quickly. I rely on Reverb + WebSockets to push events instead of holding HTTP workers open.
• Laravel-native patterns: I lean heavily on Laravel events for orchestration/observability, queues for processing, and job batching for concurrent tool calls (e.g., Google + GitHub in parallel). Tool calls fail a lot in the wild; HTTP client retry alone isn’t enough—job-level retry/backoff and idempotency matter.

On the proposal itself: it feels closer to TS/Python streaming patterns than to PHP’s operational constraints. That’s not a knock—just a note that production-ready PHP may need adapted patterns vs. ports.

Questions that would help validate the design for PHP/Laravel:

1. Reasoning streams (multi-track):
   Providers (e.g., OpenAI) can emit multiple thinking_* tracks under the same message.

   • Is there a stable reasoning_id per track?
   • Can thinking_* interleave with text_*/tool_* while preserving ordering guarantees?
   • Does finish summarize all tracks or just the primary?

2. Usage accounting:

   • You note “usage at the end,” but providers also send usage mid-turn or on tool-call messages.
   • Will finish contain a canonical total for the entire turn (incl. tools), or should clients sum partials?
   • Any double-count protection if providers emit partial usage multiple times?

3. Resumption & idempotency:

   • Support for Last-Event-ID so clients can resume an SSE stream after disconnect?
   • Stable event_id + seq for idempotent upserts/replays on the backend?

4. Laravel integration expectations:

   • Guidance for persisting text, tool calls/results, and reasoning tracks (at least a recommended shape/fields).
   • Recommended flow for queues + batching (tool orchestration) and scheduling (delayed/recurring tasks).
   • How do you see asEventStreamResponse() coexisting with Reverb/WebSockets patterns that free FPM workers—any adapter or best practices?

If the design can lock down the above (especially canonical usage and resumability) and provide Laravel-first guidance (queues/events/persistence), it’ll land much closer to what PHP teams need in production.

[sixlive]

Thank you so much! I'll definitely take all of this into consideration

[bdweix]

Great work @sixlive! The proposed asEventStreamResponse looks much cleaner. A few thoughts:

1. Ability to pass streaming templates?
   It would be great for users to pass a streaming template that will format the stream. That way people can easily do custom stream formats if they want, or if not provided it will use the default format (like what you have outlined above). That might be more involved than it sounds though.

2. Vercel AI SDK
   I've been impressed by the AI SDK (https://vercel.com/blog/ai-sdk-5), and they have a section on their stream protocols (https://ai-sdk.dev/docs/ai-sdk-ui/stream-protocol) which is quite close to many of the structures you laid out above. We are building our components and systems to make use of their UI components as you get a lot of out of the box functionality, and at a minimum, an opinionated way to do things (like frontend tool handling). I wonder if it would make sense to more formally try to support/mirror their output format? I suspect we'll see many companies look to use their packages more often, and they support React, Vue, Angular, and Svelte already.

For example, just this week this was released: https://streamdown.ai/, which is built on those streaming standards and AI Elements (https://ai-sdk.dev/elements/overview/usage). If Prisms default worked out of the box with all of those packages I think that would be a boon to all.

[kauffinger]

About 2: If the protocol is good & has many users, this might be a good move.

[sixlive]

I plan on supporting the Vercel data protocol!