Feature Request: Add support for streaming results #37
Description
Feature description
Allow registering functions that return iterators as implementation
Use cases
- Send updates for long running processes: For me at least the RPC framework is most useful for long running tasks, otherwise a simple HTTP request would suffice. For long running tasks it would be very useful to be able to send periodic updates on progress/errors etc. to the frontend.
- Streaming responses: Would be great to be able to have a generic interface for streaming responses like the AI Actions do. This would allow for improved user experience when a large number of user facing results are produces sequentially.
- Prevent wasted compute when task abandoned by frontend: since these RPC calls will usually be initiated by page loads/user requests they can be rendered unnecessary during computation if the user closes/refreshes the page or presses "Cancel". It would be useful to be able to abandon any unnecessary work also server side when this happens.
Proposed Implementation
on the retoolrpc side I would either expose this as a parameter when registering an RPC method or just check whether the returned value is an iterator after calling it and iterate over it if it is, streaming the results back to retool.
The advantage of exposing this as a parameter while registering the method is that you can then expose an argument on the frontend to control whether the response should be a stream or the entire list of results from the iterator.