How to handle confirmations and follow-up inputs in Spring AI tool-calling with RAG-based tool selection?

[Sarath-Molathoti]

I'm building a web application using Spring Boot 3.4.5 and Spring AI 1.0.0 with Llama3.2(Ollama) model integration. I've implemented tool calling, and because I have many tools in the application, I'm using RAG (Retrieval-Augmented Generation) to filter and pass only relevant tools to the LLM for execution.

Problem: I want to implement a follow-up confirmation step for certain tools. For example:

User: Save user details – Name: John, Email: john@example.com
System: Here is the preview of user details: {…}. Do you want to proceed?
User: Yes → Save operation proceeds

Currently, my tool filtering works based on the user input (using semantic search over tool descriptions or metadata). But when the user replies with a generic confirmation like "yes", "confirm", or "go ahead", the RAG system fails to select the correct tool because these confirmations lack context.

I'm using ChatMemory for maintaining conversation history, but during the confirmation step, since the user input is short/generic, the tool selection logic doesn't work well — it can't infer what tool to call or what operation to complete.

I understand that implementing follow-up questions in a conversational flow is straightforward with RAG and ChatMemory. However, my challenge arises specifically with tool calling.

When tools are selected dynamically through RAG, and the user enters multiple follow-up or unexpected clarification messages (like "change the name", "what email did I give?", "add address too"), it becomes difficult to manage — especially because a single tool must handle all such follow-up queries gracefully after the initial invocation.

This becomes even more complex when a tool requires many input parameters, and the user may provide them incrementally, modify them mid-flow, or confirm them across multiple turns. In such cases, managing tool context, partial input collection, and intent preservation becomes challenging.

I'm looking for best practices, design patterns, or architectural suggestions to handle this cleanly in Spring AI 1.0.0 — especially when tools are filtered dynamically via RAG and invoked through LLMs.