-
Notifications
You must be signed in to change notification settings - Fork 2k
Open
Labels
mcp[Component] Issues about MCP support[Component] Issues about MCP support
Description
Describe the bug
"Run Evaluation" doesn't run multiple eval cases for agents using stdio MCP tools.
To Reproduce
- Run
adk create my_agent
- Edit my_agent/agent.py:
from google.adk.agents.llm_agent import Agent
from google.adk.tools.mcp_tool.mcp_toolset import MCPToolset
from google.adk.tools.mcp_tool.mcp_session_manager import StdioConnectionParams
from mcp import StdioServerParameters
root_agent = Agent(
model="gemini-2.5-flash",
name="root_agent",
description="Add two numbers.",
instruction="You are a helpful assistant that can add two numbers. Use the 'add_two_numbers' tool for this purpose.",
tools=[
MCPToolset(
connection_params=StdioConnectionParams(
server_params=StdioServerParameters(
command="python",
args=[
"my_agent/add.py",
],
),
),
)
],
)
- Create my_agent/add.py:
from fastmcp import FastMCP
# Initialize the MCP server
mcp = FastMCP("Addition Server")
# Define a tool to add two numbers
@mcp.tool()
def add_two_numbers(a: int, b: int) -> int:
"""
Adds two numbers together.
"""
return a + b
# Run the MCP server
if __name__ == "__main__":
mcp.run()
- Run
adk web
- In the web UI:
- Eval - Create Evaluation Set: add_numbers
- Enter prompt: What is 2+2?
- Add current session to evalset - Case name: 2+2
- Create New Session
- Enter prompt: What is 21+21?
- Add current session to evalset - Case name: 21+21
- Eval - Select all cases - Run Evaluation
Console log:
2025-10-14 11:44:25,936 - WARNING - runners.py:1049 - Toolset MCPToolset cleanup cancelled: Cancelled by cancel scope 7f8a8fce7dd0
ERROR: Exception in ASGI application
Traceback (most recent call last):
File "/home/jedick/.local/lib/python3.12/site-packages/uvicorn/protocols/http/httptools_impl.py", line 409, in run_asgi
result = await app( # type: ignore[func-returns-value]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[...]
File "/home/jedick/.local/lib/python3.12/site-packages/aiohttp/streams.py", line 671, in read
await self._waiter
asyncio.exceptions.CancelledError: Cancelled by cancel scope 7f8acd4f9eb0
INFO: 127.0.0.1:34552 - "POST /apps/my_agent/eval_sets/add_numbers/run_eval HTTP/1.1" 500 Internal Server Error
Warning: Error during MCP session cleanup for stdio_session: Attempted to exit cancel scope in a different task than it was entered in
Expected behavior
Both eval cases should be run and the results shown in the Web UI.
Screenshots
Evals NOT WORKING if both cases are selected:

Evals WORKING if one case is selected:

Desktop (please complete the following information):
- OS: Linux
- Python version(python -V): 3.12.12
- ADK version(pip show google-adk): 1.16.0
Model Information:
- Are you using LiteLLM: No
- Which model is being used: gemini-2.5-flash
Additional context
Might be related to #2980
Metadata
Metadata
Assignees
Labels
mcp[Component] Issues about MCP support[Component] Issues about MCP support