Skip to content

Web UI can't run multiple eval cases with stdio MCP tools #3161

@jedick

Description

@jedick

Describe the bug
"Run Evaluation" doesn't run multiple eval cases for agents using stdio MCP tools.

To Reproduce

  • Run adk create my_agent
  • Edit my_agent/agent.py:
from google.adk.agents.llm_agent import Agent
from google.adk.tools.mcp_tool.mcp_toolset import MCPToolset
from google.adk.tools.mcp_tool.mcp_session_manager import StdioConnectionParams
from mcp import StdioServerParameters

root_agent = Agent(
    model="gemini-2.5-flash",
    name="root_agent",
    description="Add two numbers.",
    instruction="You are a helpful assistant that can add two numbers. Use the 'add_two_numbers' tool for this purpose.",
    tools=[
        MCPToolset(
            connection_params=StdioConnectionParams(
                server_params=StdioServerParameters(
                    command="python",
                    args=[
                        "my_agent/add.py",
                    ],
                ),
            ),
        )
    ],
)
  • Create my_agent/add.py:
from fastmcp import FastMCP

# Initialize the MCP server
mcp = FastMCP("Addition Server")


# Define a tool to add two numbers
@mcp.tool()
def add_two_numbers(a: int, b: int) -> int:
    """
    Adds two numbers together.
    """
    return a + b


# Run the MCP server
if __name__ == "__main__":
    mcp.run()
  • Run adk web
  • In the web UI:
  1. Eval - Create Evaluation Set: add_numbers
  2. Enter prompt: What is 2+2?
  3. Add current session to evalset - Case name: 2+2
  4. Create New Session
  5. Enter prompt: What is 21+21?
  6. Add current session to evalset - Case name: 21+21
  7. Eval - Select all cases - Run Evaluation

Console log:

2025-10-14 11:44:25,936 - WARNING - runners.py:1049 - Toolset MCPToolset cleanup cancelled: Cancelled by cancel scope 7f8a8fce7dd0
ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "/home/jedick/.local/lib/python3.12/site-packages/uvicorn/protocols/http/httptools_impl.py", line 409, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

[...]

  File "/home/jedick/.local/lib/python3.12/site-packages/aiohttp/streams.py", line 671, in read
    await self._waiter
asyncio.exceptions.CancelledError: Cancelled by cancel scope 7f8acd4f9eb0
INFO:     127.0.0.1:34552 - "POST /apps/my_agent/eval_sets/add_numbers/run_eval HTTP/1.1" 500 Internal Server Error
Warning: Error during MCP session cleanup for stdio_session: Attempted to exit cancel scope in a different task than it was entered in

Expected behavior
Both eval cases should be run and the results shown in the Web UI.

Screenshots

Evals NOT WORKING if both cases are selected:

Image

Evals WORKING if one case is selected:

Image

Desktop (please complete the following information):

  • OS: Linux
  • Python version(python -V): 3.12.12
  • ADK version(pip show google-adk): 1.16.0

Model Information:

  • Are you using LiteLLM: No
  • Which model is being used: gemini-2.5-flash

Additional context
Might be related to #2980

Metadata

Metadata

Assignees

Labels

mcp[Component] Issues about MCP support

Type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions