Tool use and LLMs through official websites.

[gdkrmr]

Regarding #802, another question came up:

If I have the model add a file to context through tool use, it cannot access the file's contents while answering the prompt. This behavior differs from the use through websites, where I only have to do a single prompt. Is this a general limitation of the APIs and models used through e.g. openai.com just do the second call on their own?

[karthink]

Is this a general limitation of the APIs and models used through e.g. openai.com just do the second call on their own?

As discussed in #802, the file needs to be added to the context before starting a request for the LLM to see it. Tool use happens "during" a request, and the payload is not updated. This is a limitation of gptel, not the APIs.

The solution from #802 applies: Either add it to the context before starting a request, or provide the file contents directly as a tool result and don't bother adding it to the context.