Anthropic: Input should be a valid string (from: messages.2.content.0.tool_result.content.0.text.text)

[safisa]

Hi,
I built an agent with some tools, I set it to use an Anthropic model, it was working on Theia 1.57, but in Theia 1.63/1.64 it is not, and I'm getting this error when it uses the tool:

ERROR Error handling chat interaction: Error: 400 {"type":"error","error":{"type":"invalid_request_error","message":"messages.2.content.0.tool_result.content.0.text.text: Input should be a valid string"}}
    at s.generate (/browser-app/lib/backend/503.js:2:5565058)
    at _.makeStatusError (/browser-app/lib/backend/503.js:2:1829335)
    at _.makeRequest (/browser-app/lib/backend/503.js:2:1832950)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)

Note that it is working on OpenAI models, and also on Anthropic in Theia 1.57.
Is it now a bug in the latest Theia versions with Anthropic? How can I debug this error or fix it?

Thanks

[sdirix]

Hi @safisa,

It would be interesting how this error can be reproduced.

If you want to debug, these are the two points to look at:

• Here are the messages collected on frontend side. They should already contain "valid" content. These are then sent to the backend.
• On backend side, here the messages are converted to Anthropic specific messages.

At least one of these seems to go wrong.

[safisa]

Hi @sdirix,
I have debugged the issue, it is in the backend in the node_modules/@theia/ai-anthropic/src/node/anthropic-language-model.ts within the formatToolCallResult method.
My tool returns an array of objects, and the method formatToolCallResult does not deal with this type correctly:

theia/packages/ai-anthropic/src/node/anthropic-language-model.ts

Line 213 in 7140fc0

return result.map(r => ({ type: 'text', text: r as string }));

But r is not surely a text, in my case it is an object!!!

Can you fix it?

Thanks

[sdirix]

I agree this code should be more resilient. It's wrong to assume that there are only strings handed over. However I don't think we can do much better than just stringifying each array element if it's not a string already. Is that the behavior you expect?

[safisa]

Hi,
If this is the only way to fix, then yes, stringifying should be okay, since the result will be treated by the LLM, which should not care if it is a JSON object or a string.
I want to know why in Anthropic you are parsing the tool result, and on other LLMs like OpenAI, you are not, and it is working?
Thanks

[eneufeld]

At the moment we implemented the support the apis had different capabilities. OpenAI SDK supported the handling of tool calls whereas the Anthropic SDK didn't. I have to admit I didn't check whether Anthropic SDK now also supports tool handling. If it does, we should remove the manual handling. Happy to review a PR for this.

[safisa]

Hi @sdirix, @eneufeld,
I have changed our tool's response type to string (using JSON.stringify), and now it works with Anthropic. I found also that boolean results are not supported in all models.
So, I have added a function that formats all my tools' result, I am calling it in each tool. Here is the code of this method:

// if the result is a boolean, convert it to 'success' or 'failure' strings, since a boolean is not supported in Theia ToolCallPartRenderer.renderResult
    // (tool handler now returns ToolCallResult = undefined | object | string | ToolCallContent)
    if (typeof toolResult === 'boolean') {
        return toolResult ? 'success' : 'failure';
    }

    // If the result is an object or an array, stringify it with indentation (fixes an issue in Anthropic tool calls)
    if (typeof toolResult === 'object' || Array.isArray(toolResult)) {
        return JSON.stringify(toolResult, null, 2);
    }

Thanks