Bug: Inconsistent Behavior with AISearch Tool when Using Multiple Tools

[Pavan-Microsoft]

We’re seeing inconsistent behavior when using both AISearch and SQL tools together within the agent framework.

When using only AISearch, it responds correctly.

When using both AISearch and SQL tools, we observe the following inconsistent outcomes:

Sometimes AISearch responds correctly

Sometimes it says "cannot answer the query"

Sometimes we get an error: no tool or function named file_search

We also need details on how to retrieve citations from AISearch responses.

Steps to Reproduce

Create and register an agent using the provided script below.

Define both tools — SQLTool and HostedFileSearchTool.

Run multiple user inputs through the agent.

Observe inconsistent behavior in AISearch responses.

Sample Script used:

async with (
    AzureCliCredential() as credential,
    AIProjectClient(endpoint=config.ai_project_endpoint, credential=credential) as client
):
    agent_i = await create_agent(client, config)

    created_thread = await client.agents.threads.create()

    try:
        custom_tool = SQLTool(conn=await get_sqldb_connection())

        search_tool = HostedFileSearchTool(
            additional_properties={
                "index_name": config.azure_ai_search_index,
                "query_type": "vector_semantic_hybrid",
                "top_k": 5
            }
        )

        my_tools = [search_tool, custom_tool.get_sql_response]

        agent_ai_client = AzureAIAgentClient(
            project_client=client,
            agent_id=agent_i.id,
            project_endpoint=config.ai_project_endpoint
        )

        truncation_strategy = TruncationObject(type="last_messages", last_messages=4)
        async with ChatAgent(
            chat_client=agent_ai_client,
            tools=my_tools,
            tool_choice="auto"
        ) as agent:
            for user_input in USER_INPUTS:
                thread = agent.get_new_thread(service_thread_id=created_thread.id)
                print(f"# User: {user_input}")
                print("# Host: ", end="")
                async for response in agent.run_stream(
                    messages=user_input,
                    thread=thread,
                    truncation_strategy=truncation_strategy
                ):
                    print(response.text, end="")
                print()

How can we retrieve citations from AISearch responses?

Is there any configuration needed when multiple tools are registered together?