-
Notifications
You must be signed in to change notification settings - Fork 2.1k
fix(dspy): Fix MCP tool conversion when schema has no input params #8566
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
fix(dspy): Fix MCP tool conversion when schema has no input params #8566
Conversation
84757ee
to
c4464c8
Compare
dspy/utils/mcp.py
Outdated
@@ -44,4 +44,8 @@ async def func(*args, **kwargs): | |||
result = await session.call_tool(tool.name, arguments=kwargs) | |||
return _convert_mcp_tool_result(result) | |||
|
|||
return Tool(func=func, name=tool.name, desc=tool.description, args=args, arg_types=arg_types, arg_desc=arg_desc) | |||
async def _no_arg_func(): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this just for not setting has_kwargs=False?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not totally sure... this change to having a separate _no_arg_func() does result in has_kwargs=False on the resulting Tool. I'm not sure if that is reason the Tool ends up working as expected or if it's correlated side effect, but not the reason. I'll dig into Tool to see what's going on and report back.
Thanks for your review!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think the signature of the function that gets passed into Tool(func=)
is the main issue after all. The Tool constructor inspects the function and creates its own args
and arg_types
based on what it inspects, and that's what results in the incorrect input argument schema on the resulting Tool. It also ends up setting has_kwargs=True, but afaict that doesn't impact the input argument schema.
These are the steps I took to get to that conclusion:
To walk through it in a little more detail, Tool._parse_function
is called from Tool.__init__
and provided the func
and arg_desc
variables that were passed into the constructor:
dspy/dspy/adapters/types/tool.py
Line 72 in 2e60022
self._parse_function(func, arg_desc) |
Tool._parse_function
uses inspect to see what arguments are on the provided func
here (annotations_func = func
in the case we're looking at):
dspy/dspy/adapters/types/tool.py
Lines 86 to 92 in 2e60022
# Use inspect.signature to get all arg names | |
sig = inspect.signature(annotations_func) | |
# Get available type hints | |
available_hints = get_type_hints(annotations_func) | |
# Build a dictionary of arg name -> type (defaulting to Any when missing) | |
hints = {param_name: available_hints.get(param_name, Any) for param_name in sig.parameters.keys()} | |
default_values = {param_name: sig.parameters[param_name].default for param_name in sig.parameters.keys()} |
hints={"args": Any, "kwargs": Any}
at the end of the block for a function like the one that was being passed in from the mcp conversion tool. I double check that by running this code in a python repl (uv run python
):
>>> import inspect
>>> from typing import TYPE_CHECKING, Any, Callable, Type, get_origin, get_type_hints
>>> def func(*args, **kwargs):
... return "yay kwargs"
...
>>> annotations_func = func
... sig = inspect.signature(annotations_func)
... # Get available type hints
... available_hints = get_type_hints(annotations_func)
... # Build a dictionary of arg name -> type (defaulting to Any when missing)
... hints = {param_name: available_hints.get(param_name, Any) for param_name in sig.parameters.keys()}
... default_values = {param_name: sig.parameters[param_name].default for param_name in sig.parameters.keys()}
... print(f"{hints=}\n{default_values=}")
...
hints={'args': typing.Any, 'kwargs': typing.Any}
default_values={'args': <class 'inspect._empty'>, 'kwargs': <class 'inspect._empty'>}
The for-loop that follows populates arg_types
with the same key/values as hints
. It also creates an entry in args
for each key in hints
. This is the line that executes for the function defined above:
dspy/dspy/adapters/types/tool.py
Line 106 in 2e60022
args[k] = _resolve_json_schema_reference(TypeAdapter(v).json_schema()) |
And this is what args and arg_types look like after the for-loop runs:
>>> from pydantic import BaseModel, TypeAdapter, create_model
>>> from dspy.adapters.types.tool import _resolve_json_schema_reference
>>> args = {}
>>> arg_types = {}
>>> for k, v in hints.items():
... arg_types[k] = v
... args[k] = _resolve_json_schema_reference(TypeAdapter(v).json_schema())
...
>>> args
{'args': {}, 'kwargs': {}}
>>> arg_types
{'args': typing.Any, 'kwargs': typing.Any}
Finally, this is where the args variable computed above is set as the args instance variable on the Tool:
dspy/dspy/adapters/types/tool.py
Lines 114 to 116 in 2e60022
self.args = self.args or args | |
self.arg_types = self.arg_types or arg_types | |
self.has_kwargs = any(param.kind == param.VAR_KEYWORD for param in sig.parameters.values()) |
The self.args
and self.arg_types
are False-y values since they are empty dicts.
This becomes a problem later specifically for the mcp tool, because the func that the DSPy Tool has is:
# Convert the MCP tool and Session to a single async method
async def func(*args, **kwargs):
result = await session.call_tool(tool.name, arguments=kwargs)
return _convert_mcp_tool_result(result)
And callers will set input arguments such that kwargs={"args": {}, "kwargs": {}}
, because those are the input args the DSPy Tool says to use. That then results in errors like this from the mcp tool:
RuntimeError: Failed to call a MCP tool: Error calling tool 'current_datetime': 2 validation errors for call[current_datetime]
args
Unexpected keyword argument [type=unexpected_keyword_argument, input_value={}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.11/v/unexpected keyword argument
kwargs
Unexpected keyword argument [type=unexpected_keyword_argument, input_value={}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.11/v/unexpected keyword argument
The change I made to pass in a different function when there are no input arguments results in args={}
on the Tool, which then communicates the right input argument schema and we get valid calls to the mcp tool.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you so much for the detailed investigation! I think the root issue is that we use self.args = self.args or args
instead
of self.args = self.args if args is not None else args
(same for arg_types). Can you try to see if this change ifself can solve the issue?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good idea! I just tried that out and it does seem to work, yay! 🎉
I updated this pr accordingly. Thanks for your help!
Update `convert_input_schema_to_tool_args()` to return `None` when the schema does not have properties. Then, use a no-argument function instead of an arbitrary argument function in `convert_mcp_tool()` to ensure the resulting `Tool` has `has_kwargs=False`. Fixes stanfordnlp#8565
9e2fa4f
to
a5b7061
Compare
📝 Changes Description
Update
convert_input_schema_to_tool_args()
to returnNone
when the schema does not have properties.Then, use a no-argument function instead of an arbitrary argument function in
convert_mcp_tool()
to ensure the resultingTool
hashas_kwargs=False
.Fixes #8565
✅ Contributor Checklist
This is my first PR on DSPy; please lmk if anything is weird/missing. Thanks!