AI Response Streaming Implementation in Odoo

[JochenVanSeveren]

Cool to see you're using Odoo's bus.bus for streaming AI responses. I've implemented a different approach using Server-Sent Events (SSE) that looks a bit like this:

@http.route("/ai/stream", type="http", auth="public")
def stream_ai(self, prompt, **kw):
    def event_source():
        for token in ai_model.generate(prompt):
            yield f"data: {json.dumps(token)}\n\n"
        yield "event: done\ndata: [DONE]\n\n"

    return http.Response(
        event_source(),
        headers=[
            ("Content-Type", "text/event-stream"),
            ("Cache-Control", "no-cache"),
            ("Connection", "keep-alive"),
        ],
    )

I'm curious about your choice to use bus.bus instead of direct streaming.
Could you share your thoughts on why you chose bus.bus over SSE as bus.bus requires a lot of writes?

I'm trying to understand the tradeoffs between these approaches for a production environment. Any insights would be helpful!

[yonitjio]

This might be a bit disappointing, but the main focus of the channel is to explore what Odoo offers, preferably, out of the box. That’s why I prioritize using built-in Odoo features before turning to external technologies.

In this case, the bus module is not only sufficient but also relatively straightforward to use, which aligns well with the channel’s goal of being beginner-friendly.

That said, one notable advantage of the bus module—correct me if I'm wrong—is that it temporarily stores messages (until they’re cleared by Odoo’s autovacuum job). This means if the client’s connection is briefly disrupted, the messages aren’t lost; they can still be delivered once the client reconnects.

Of course, if you prefer an alternative approach like Server-Sent Events (SSE), you’re free to go that route—but keep in mind that it requires implementing the entire mechanism from scratch.

[JochenVanSeveren]

Thanks for the explanation. I didn’t realize how much I’d need to implement myself for SSE. That makes total sense now.
The write load from bus.bus is probably negligible.

Also, thanks for your work on this repository. Your nodes editor idea is genius! Makes me see how Odoo could be a real power framework for AI, even accessible to non-devs. I’ll go ahead and use bus.bus for streaming in my React Native app based on your approach.

[yonitjio]

Nice, happy coding!