Metadata exclusion for embedded resource tool response

[tradeqvest]

Question

Hello!

When returning an embedded resource from an MCP Tool using the MCPServerStreamableHTTP, pydantic ai currently only returns the text due to this code.

However, information besides the text may be interesting for further processing.

embedded_resource = EmbeddedResource(
                    type="resource",
                    resource=TextResourceContents(
                        uri=f"resource://{to_uri(result.id)}",
                        text=result.document.text,
                        mimeType="text/plain",
                        _meta=result.document.metadata or {},
                    ),
                    _meta={
                        "relevance_score": round(result.score, 3),
                        "document_last_modified": document.metadata.get(
                            "modifiedDate", None
                        ),
                    },
                )

In this example, only the text would be returned, which may be insufficient. Is this behavior intended for embedded resources? Other clients from fastmcp return all available content. I feel that at least having control over this behavior would be very beneficial.

Thank you in advance for your support.

Additional Context

No response

[DouweM]

@tradeqvest Do I understand correctly that you'd like the relevance_score and document_last_modified to be passed to the model alongside the resource text? Or are you intending to use this data in code?

[tradeqvest]

Yes, that would be one aspect. Even more important would be all the content in the resource itself, e.g. uri and metadata. Right now, I do not see a way of providing this information to the model nor obtaining it for usage in code.

[DouweM]

@tradeqvest OK, so the entire EmbeddedResource should just be serialized as JSON and sent to the model like that? With the full object available in result.all_messages()?

What do you think we should do when the MCP server returns a ResourceLink, which we're currently handling similarly by looking up the resource and sending just that text/binary to the model, losing any metadata:

pydantic-ai/pydantic_ai_slim/pydantic_ai/mcp.py

Lines 286 to 295 in fc6a2b2

	elif isinstance(part, mcp_types.EmbeddedResource):
	resource = part.resource
	return self._get_content(resource)
	elif isinstance(part, mcp_types.ResourceLink):
	resource_result: mcp_types.ReadResourceResult = await self._client.read_resource(part.uri)
	return (
	self._get_content(resource_result.contents[0])
	if len(resource_result.contents) == 1
	else [self._get_content(resource) for resource in resource_result.contents]
	)

In that case, the model wouldn't know what to do with the the resource URI unless it has a read_resource tool (which is not provided by default -- should it be?).

The handling of directly returned TextContent, ImageContent, and AudioContent also currently loses the metadata, as we pass it to the model as BinaryContent:

pydantic-ai/pydantic_ai_slim/pydantic_ai/mcp.py

Lines 270 to 285 in fc6a2b2

	if isinstance(part, mcp_types.TextContent):
	text = part.text
	if text.startswith(('[', '{')):
	try:
	return pydantic_core.from_json(text)
	except ValueError:
	pass
	return text
	elif isinstance(part, mcp_types.ImageContent):
	return messages.BinaryContent(data=base64.b64decode(part.data), media_type=part.mimeType)
	elif isinstance(part, mcp_types.AudioContent):
	# NOTE: The FastMCP server doesn't support audio content.
	# See <https://github.com/modelcontextprotocol/python-sdk/issues/952> for more details.
	return messages.BinaryContent(
	data=base64.b64decode(part.data), media_type=part.mimeType
	) # pragma: no cover

If we wanted to include the metadata there, we'd need to pass it in a separate "user" message.

Do you know how other AI libraries that act as MCP clients handle this? Unfortunately LLM APIs don't have a native way of providing metadata alongside a file.