How to handle long running tools

[db0sch]

Hello 👋

First, again, thanks for this gem. It's really a very elegant piece of work.

I have this (possibly stupid) question:

How would you handle a tool taking a long time to execute?
Let's say a tool that will perform a long API call or even calling another agent (that might take time to do its thing).

I would instinctively have this logic in a background job.
But then, I would like either:

• the tool to wait before sending back a response to the LLM
• or have the tool return something that says "processing"... and the LLM would hang in there, until I replace the tool response message with the actual response once it gets there.

If anyone has implemented this, I would be glad to know how you did this?
(maybe there is something obvious I'm not seeing)

I know this is not an issue per se. Sorry about this. Feel free to close it if it's not relevant.
But maybe It could lead to improving the Tool documentation to handle these kind of use cases.

Thanks

[tpaulshippy]

This might be obvious, but isn't your LLM call already in a background job? Usually the LLM takes longer than the tool in my experience. But if I were dealing with slow tools, I would just have even longer running jobs.

[db0sch]

It is running in a background job.
But, let's say my tool is taking sometimes to perform, I would like to be able to display "Searching the web..." (i.e. the tool being performed), and then, when it's done, the conversation can continue.
Maybe i'm not thinking about it the right way 🤔

[tpaulshippy]

Oh sure. Your tool could broadcast a "waiting..." message back to your UI using Turbo Streams.

[tpaulshippy]

Or whatever mechanism you use to stream your LLM responses.

[db0sch]

Thanks. I'll try that.