Model Context Protocol
New Content Type for "UI" #287
Replies: 17 comments 36 replies
-
|
Hey @kentcdodds while we're at it, we should probably drop |
Beta Was this translation helpful? Give feedback.
-
|
Agreed. Updated! |
Beta Was this translation helpful? Give feedback.
-
|
Good idea. Indeed there are security concerns with using HTML. There are several frameworks to standardize declaring UI components. Perhaps we could encourage these as custom options for that At Microsoft when working on Microsoft Copilot for consumers and enterprise users, we used Adaptive Cards and that worked great. The new consumer version doesn't use them anymore because they're not needed, but the M365 (Office) Copilot still uses them. Here are some more details that I wrote about it and how standardizing on Adaptive Cards helped us scale to many UIs and types of clients: https://devblogs.microsoft.com/dotnet/building-ai-powered-bing-chat-with-signalr-and-other-open-source-tools/ There's also Fluid Framework. |
Beta Was this translation helpful? Give feedback.
-
I would treat the chat client the same way we treat browsers today - we send a bunch of html & js their way and it's up to the client, not the protocol, to decide what of that to allow/disallow. |
Beta Was this translation helpful? Give feedback.
-
|
Since authors aren't necessarily writing these, don't you think the default should be the browser's native component model? |
Beta Was this translation helpful? Give feedback.
-
|
I would like this to be one of several different Here are some thoughts from Claude that I think are good.Model Context Protocol UI Extension (MCP-UI) 1. AbstractThis RFC proposes an extension to the Model Context Protocol (MCP) to standardize the integration of rich UI components in chat-based AI applications. The MCP-UI Extension enables MCP servers to expose interactive UI elements ranging from simple cards to full-page applications, along with a hook system to support rich interaction lifecycles. 2. Introduction and Motivation2.1 BackgroundThe Model Context Protocol (MCP) has successfully established a standard for AI models to access context, tools, and resources. However, the current specification focuses on textual or basic content types, with limited support for rich interactive experiences. Applications like Convex's Chef and Stackblitz's Bolt.new demonstrate the power of combining LLM interactions with rich UIs powered by WebContainers. These experiences remain application-specific, without a standard protocol for interoperability. 2.2 Problem StatementDevelopers building chat applications with AI models encounter several challenges:
2.3 GoalsThis extension aims to:
3. Terminology
4. Specification4.1 Capability NegotiationMCP servers that support UI components MUST declare relevant capabilities during initialization: {
"capabilities": {
"ui": {
"components": true, // Support for UI components
"fullView": true, // Support for expanded view components
"hooks": true, // Support for lifecycle hooks
"frameworksSupported": ["html", "react", "vue"] // Optional frameworks list
}
}
}Clients supporting the UI extension MUST include UI capabilities in their initialization request: {
"capabilities": {
"ui": {
"formats": ["html", "react"], // UI formats the client can render
"hooksEnabled": true // Client supports hooks execution
}
}
}4.2 Component Registry4.2.1 Listing ComponentsClients can discover available UI components by sending a {
"jsonrpc": "2.0",
"id": 1,
"method": "ui/listComponents"
}Servers respond with: {
"jsonrpc": "2.0",
"id": 1,
"result": {
"components": [
{
"id": "data-chart",
"name": "Data Visualization Chart",
"description": "Interactive visualization of tabular data",
"type": "card",
"supportedFormats": ["html", "react"],
"triggers": ["tools/call:data-analyze"],
"inputSchema": {
"type": "object",
"properties": {
"data": {
"type": "array",
"description": "Data points to visualize"
},
"chartType": {
"type": "string",
"enum": ["bar", "line", "pie"]
}
}
}
},
{
"id": "code-editor",
"name": "Code Editor",
"description": "Full-featured code editor with syntax highlighting",
"type": "fullView",
"supportedFormats": ["html", "react"],
"triggers": ["tools/call:edit-code"],
"inputSchema": {
"type": "object",
"properties": {
"language": {
"type": "string"
},
"code": {
"type": "string"
},
"readOnly": {
"type": "boolean"
}
}
}
}
]
}
}4.2.2 Component Change NotificationsServers SHOULD notify clients when the component registry changes: {
"jsonrpc": "2.0",
"method": "notifications/ui/components_changed"
}4.3 Rendering Components4.3.1 Request to Render a ComponentClients request component rendering with: {
"jsonrpc": "2.0",
"id": 2,
"method": "ui/renderComponent",
"params": {
"id": "data-chart",
"format": "react", // Requested format from supportedFormats
"data": {
"data": [
{"month": "Jan", "value": 10},
{"month": "Feb", "value": 15},
{"month": "Mar", "value": 8}
],
"chartType": "bar"
},
"sessionId": "ui-session-123" // Optional client-generated session ID
}
}4.3.2 Component Rendering ResponseFor HTML format: {
"jsonrpc": "2.0",
"id": 2,
"result": {
"format": "html",
"content": {
"html": "<div class=\"chart-container\">...</div>",
"scripts": [
"https://cdn.example.com/charts.js",
"const chart = new Chart(...);"
],
"styles": [
".chart-container { height: 300px; }",
"https://cdn.example.com/charts.css"
]
},
"metadata": {
"height": 300,
"width": "100%",
"interactivity": "high"
},
"sessionId": "ui-session-123"
}
}For React format: {
"jsonrpc": "2.0",
"id": 2,
"result": {
"format": "react",
"content": {
"module": "https://cdn.example.com/components/DataChart.js",
"props": {
"data": [...],
"chartType": "bar",
"onDataPointClick": {"__handler": "dataPointSelected"}
}
},
"metadata": {
"height": 300,
"width": "100%",
"interactivity": "high"
},
"sessionId": "ui-session-123"
}
}For ESM module format (framework agnostic): {
"jsonrpc": "2.0",
"id": 2,
"result": {
"format": "module",
"content": {
"main": "https://cdn.example.com/components/chart-component.js",
"dependencies": [
"https://cdn.example.com/components/utils.js"
],
"props": {
"data": [...],
"chartType": "bar"
}
},
"metadata": {
"height": 300,
"width": "100%"
},
"sessionId": "ui-session-123"
}
}4.3.3 Component EmbeddingThe UI components can be embedded in two ways:
The client determines the rendering approach based on:
4.4 Component Interaction4.4.1 Event HandlingComponents can define event handlers that trigger specific actions: {
"jsonrpc": "2.0",
"method": "ui/componentEvent",
"params": {
"sessionId": "ui-session-123",
"componentId": "data-chart",
"event": "dataPointSelected",
"data": {
"point": {"month": "Feb", "value": 15}
}
}
}4.4.2 State UpdatesServers can push state updates to rendered components: {
"jsonrpc": "2.0",
"method": "notifications/ui/stateUpdate",
"params": {
"sessionId": "ui-session-123",
"updates": {
"selectedPoint": {"month": "Feb", "value": 15},
"highlightedSeries": "revenue"
}
}
}4.4.3 Form SubmissionComponents can submit form data through a standardized method: {
"jsonrpc": "2.0",
"id": 3,
"method": "ui/submitForm",
"params": {
"sessionId": "ui-session-123",
"formId": "config-form",
"data": {
"chartType": "line",
"timeRange": "last-30-days",
"metrics": ["revenue", "users"]
}
}
}4.5 Lifecycle HooksHooks provide integration points at critical moments in the chat interaction flow. 4.5.1 Hook Types
4.5.2 Hook RegistrationServers register hooks through the component registry extension: {
"jsonrpc": "2.0",
"id": 4,
"method": "ui/listHooks"
}Response: {
"jsonrpc": "2.0",
"id": 4,
"result": {
"hooks": [
{
"id": "code-formatter",
"type": "lint",
"description": "Formats code according to style guidelines",
"triggers": ["postGen"],
"inputSchema": {
"type": "object",
"properties": {
"language": { "type": "string" },
"code": { "type": "string" }
}
}
},
{
"id": "dependency-validator",
"type": "test",
"description": "Validates that all dependencies are properly defined",
"triggers": ["userAccepted"],
"inputSchema": {
"type": "object",
"properties": {
"packageJson": { "type": "string" },
"imports": { "type": "array" }
}
}
}
]
}
}4.5.3 Hook ExecutionClients execute hooks by calling a specialized tool: {
"jsonrpc": "2.0",
"id": 5,
"method": "tools/call",
"params": {
"name": "hook:code-formatter",
"arguments": {
"language": "javascript",
"code": "function hello() {console.log('world')}"
}
}
}Response: {
"jsonrpc": "2.0",
"id": 5,
"result": {
"content": [
{
"type": "text",
"text": "function hello() {\n console.log('world');\n}"
}
],
"metadata": {
"lintChanges": 2,
"issues": []
}
}
}4.6 WebContainer IntegrationThe MCP-UI extension is designed to work seamlessly with WebContainer-based environments:
Example WebContainer integration: {
"jsonrpc": "2.0",
"id": 6,
"method": "ui/renderComponent",
"params": {
"id": "react-editor",
"format": "react",
"data": {
"projectPath": "/workspace/my-project",
"openFile": "src/App.js"
},
"webContainerOptions": {
"serverPort": 3000,
"filesystemAccess": ["read", "write"],
"mountPoints": ["/workspace"]
}
}
}5. Security Considerations5.1 Content Security PolicyClients MUST implement appropriate Content Security Policies when rendering arbitrary HTML and JavaScript from MCP servers. This includes:
5.2 User ConsentClients SHOULD obtain user consent before:
5.3 Data PrivacyComponents SHOULD:
5.4 Resource LimitationsClients MUST implement resource limitations to prevent abuse:
6. Examples6.1 Simple Chart CardUser: Show me a chart of the last 6 months of revenue Assistant's Process:
Implementation: // Tool call
{
"method": "tools/call",
"params": {
"name": "data-analytics",
"arguments": {
"metric": "revenue",
"period": "6-months"
}
}
}
// Tool response
{
"content": [
{
"type": "text",
"text": "I've analyzed the last 6 months of revenue data."
}
],
"data": {
"revenue": [
{"month": "Nov", "value": 120000},
{"month": "Dec", "value": 150000},
{"month": "Jan", "value": 130000},
{"month": "Feb", "value": 160000},
{"month": "Mar", "value": 140000},
{"month": "Apr", "value": 180000}
]
}
}
// UI component rendering
{
"method": "ui/renderComponent",
"params": {
"id": "data-chart",
"format": "react",
"data": {
"data": [response.data.revenue],
"chartType": "line",
"title": "Monthly Revenue (Last 6 Months)"
}
}
}6.2 Code Editor Full ViewUser: Help me refactor my React component to use hooks Assistant's Process:
Implementation: // UI component rendering
{
"method": "ui/renderComponent",
"params": {
"id": "code-editor",
"format": "react",
"data": {
"language": "javascript",
"code": "class MyComponent extends React.Component { ... }",
"action": "refactor",
"refactorType": "convert-to-hooks"
},
"type": "fullView"
}
}6.3 Lint Hook ExampleUser: Generate a React component that fetches data from an API Assistant's Process:
Implementation: // Generate code using LLM
// Before displaying, run the code-formatter hook
{
"method": "tools/call",
"params": {
"name": "hook:code-formatter",
"arguments": {
"language": "javascript",
"code": "function DataFetcher(){const[data,setData]=useState(null);useEffect(()=>{fetch('/api/data').then(r=>r.json()).then(setData)},[]);return <div>{data?JSON.stringify(data):'Loading...'}</div>}"
}
}
}
// Formatted result displayed to user
{
"content": [
{
"type": "text",
"text": "```jsx\nfunction DataFetcher() {\n const [data, setData] = useState(null);\n\n useEffect(() => {\n fetch('/api/data')\n .then(r => r.json())\n .then(setData);\n }, []);\n\n return <div>{data ? JSON.stringify(data) : 'Loading...'}</div>;\n}\n```"
}
]
}7. Implementation Considerations7.1 Compatibility LayerFor compatibility with existing MCP implementations, the UI extension can be introduced incrementally:
7.2 Framework SupportThe protocol is designed to be framework-agnostic but includes optimizations for popular frameworks:
7.3 Progressive EnhancementImplementation should follow progressive enhancement principles:
8. Adoption Roadmap8.1 Phase 1: Draft Specification (Current)
8.2 Phase 2: Implementation and Testing
8.3 Phase 3: Standardization
9. References
10. AcknowledgmentsThis proposal was inspired by innovative applications like Convex's Chef and Stackblitz's Bolt.new, as well as feedback from the broader AI developer community seeking to create more interactive and productive AI-assisted experiences. 11. Appendix11.1 Schema DefinitionsSee accompanying JSON Schema documents for formal definitions of all protocol extensions. 11.2 WebContainer Configuration Options
For Instance, in the screenshot I would imagine that the MCP server for convex emitted that it could render a full app. And Chef decided to render that as a new tab in it's bolt.new like interface. |
Beta Was this translation helpful? Give feedback.
-
|
Interesting idea. I think that what you just posted should probably be in its own discussion. It's a much bigger thing than what I'm asking for here. |
Beta Was this translation helpful? Give feedback.
-
|
I've been thinking about this too. What I've been imagining is more design systems/tokens based + generative UI. Provide context about brand, voice, tone, etc. context. Then specify display expectations for the UI. Then let the LLM produce the UI in the systems it has available. Then return resources can also specify preferred display expectations. What's led me to this thinking is trying to find a way that doesn't assume the capabilities of an agent or what it's using this information for. If it's going to use this for displaying, it can decide if it can use this information to produce the necessary UI based on what it has available to it. If it's going to use this information for doing more data processing, then it won't need to concern itself with the UI parts. Just throwing HTML into an arbitrary isolated webview with lots of things turned off is certainly the easiest route but this route might offer some really interesting mixed experiences with what the agent branding is and the upstream data/mcp |
Beta Was this translation helpful? Give feedback.
-
|
This is interesting. How do you think it would represent more complex UI components, like a map (ie: "show me directions to the nearest taco stand")? |
Beta Was this translation helpful? Give feedback.
-
|
It depends on the level you're asking about. If this is person -> agent: "show me directions to the nearest taco stand." The agent that the person is interacting with is in charge of how to display this information. It will need to have its support for various different components and such. If this was person -> agent: "show me directions to nearest taco stand." -> MCP tool call for restaurant finder then I would expect the restaurant finder to have already provided the design system/tokens. Then it would return the necessary structured data for a map coordinates and/or address information with any additional instructions it might want to have a branded experience. That example might be a little odd as I'd expect one of two things: 1) agent can display maps (and can thus probably generate brandable sections) or 2) they provide an address that's linkable to a map service like google maps. If I think of other experiences that might be complex (in the sense that there can be lots of components with different needs).. I can imagine a shopping cart experience with delegated UI. My agent finds a shop that I'd like to shop from (or many). Using their MCPs they can delegate information about branding, imagery, etc. that I can then present. So site logo, call to action buttons, etc. can all look and feel similar to what you'd expect from the site. If I'm presenting more than one site's products, the agent can normalize it and use distinct tokens that allow for lightly branded experiences (colors, etc.). If this is a specific store/product experience, it can go much further in what/how it shows content to users based on the available context (sizes, animation, etc.). Ultimately, again, I think the agent will determine when/how the UI based on what it's doing with it. Like, what are the truly imporant parts that one needs to describe to make up a branded experience and can those be codified in a way that allows any agent to do this. There's security/fraud considerations abound here. But I think we will need more cues that don't exist currently to ensure these things are legit regardless of mechanism. What's the identification and "green lock" in this world? Admittedly, I'm still thinking through this but I feel like, directionally, this is what true generative UI is vs function calling selecting internal components on a site. |
Beta Was this translation helpful? Give feedback.
-
|
I think this direction is interesting, however one hesitation I have with it (versus just directly returning HTML) is that we lose the possibility of instantaneous rendering. One of the biggest pain points I have with, say, Claude Artifacts' flow is how long it takes to render a complex UI. It's a major blocker on having it be useful in the kind of workflows I imagine this proposal to enable. The idea of non-generative strict templating with HTML on the server side solves the speed problem. Secondarily / related: if I'm setting up some route admin-type workflow that benefits from having a UI, I'd probably want the UI to be quite deterministic. It would be potentially undesirable to get a slightly different UI every time I go through the workflow. Not to say I don't think there is room to consider this generative UI approach, but it may be worth a separate exploration. IMO it will be an easier path to get something like this HTML content type added, and if it becomes a popular use case with ecosystem adoption, expanding it to include design tokens and generative rendering could be a way to level it up. |
Beta Was this translation helpful? Give feedback.
-
|
One more additional concern with the generative UI approach: that starts to feel like a predominantly host-app concern. The idea of design tokens for branding and stylistic purposes is interesting, but ultimately the functionality to generate a UI falls to the host app, which then begs the question: why aren't more host apps today already taking e.g. free form TextContent + ImageContent responses and pulling them together to generate helpful UI's to their users? I'd posit that it's at least partially the "speed" problem above (though it's possible that simply the fact that there isn't enough context to "hint" to the LLM+host that generating a UI would be appropriate is enough of a blocker). This is in contrast with the original proposal, where structuring the UI is squarely a server-side concern, which I think unlocks more interesting possibilities that altogether aren't possible today. |
Beta Was this translation helpful? Give feedback.
-
|
I think @sean-roberts has made some interesting points here. I especially think it is important to think about how to support the two possibilities- one where the agent wants to render something for a user, and another where the agent just wants the data for further processing. As it relates to showing things to the user, I don't think generating a UI makes a lot of sense. To me, a huge reason for doing this is to inject branding capabilities into MCP where there is no branding at all right now. I actually think this is vital for the success of the whole MCP project because it gets to the motivations of server implementers... If I am a company and I have no way to associate my brand with the value a user is getting from my MCP server, why would I ever invest in moving my customers in that direction? I think everybody here understands that well enough, but what I'm getting to is that I don't think brands want a generated UI... they want to tightly dictate what the user sees and the experience the user has. All that to say, I'm a nobody but my vote is for deterministic UI. |
Beta Was this translation helpful? Give feedback.
-
|
I do really like the idea of UI being a possible content type as a response from a tool call. Wanted to flag that this is probably related and has a new PR open. It probably doesn't make sense to try to get the concept of UI in that PR, but would likely build on top of that work. |
Beta Was this translation helpful? Give feedback.
-
|
I like your examples for your cases, but I also think there are similar opportunities for |
Beta Was this translation helpful? Give feedback.
-
|
With this in mind, having a set of UI components or tokens as @sean-roberts suggests seems like the best approach here. The LLM can put these things together in different ways depending on the needs of the user. For example:
The UI generated can be branded to the service if provided with the proper components and/or tokens from the MCP server metadata and/or tool response. This is in contrast to the current possible experience:
I think having something in the spec for clients to reliably generate UI when appropriate would be quite useful. |
Beta Was this translation helpful? Give feedback.
-
|
@tadasant if you were returning a list of contacts then you should be returning a list of resources with the mime-type as vCards and the UI would know how to visualize vCard resources. So in that case you wouldn't need to visualize the response using a custom render. It would just visualize the resources that were returned based on the mime-type. So given this you still only need components which match then JSON schema of your tool calls (for confirmation cards) and components which visualize resources (for rich attachments and responses) |
Beta Was this translation helpful? Give feedback.
-
I don't think this is necessarily true. I think tool calls can return data that is much more ephemeral than a Resource, and in many cases doesn't stand alone (and might have context relevant to that specific tool call). For example, say the tool call was to
I added some thoughts on the generative UI approach above -- not totally on sold on that, but I would agree with the before/after examples you gave as something this proposal (in its original, non-generative form) does help solve. One interesting dynamic I am observing as we talk through use cases is the idea that the user will often want to interact with the UI to continue the workflow. i.e.
I think it'd be worthwhile to think through how to make this HTML content type "interactive", such that a button click can basically trigger something like a
|
Beta Was this translation helpful? Give feedback.
-
|
Why can't the per-distance not be represented in a resource? Resources follow
Would load the resource passing in the users lat long so that it can calculate the distance. You can go full oData if you wanted to support filtering and things like that allowing your resources to represent collections. There are also GIS datatypes that would make for a good mime type such as geojson |
Beta Was this translation helpful? Give feedback.
-
|
Writing to also flag this PR and this comment on the use of EmbeddedResource. The spec already supports delivery of I'd normally expect the provision of dynamic UI rendering to be a Host application feature. Is the proposal to require Hosts to support certain UX primitives? (Removed comment on Claude Artifacts, it's the other way round :) ). |
Beta Was this translation helpful? Give feedback.
-
|
Good flag on that PR; certainly relevant w.r.t. advertising support for this content type (if we were to add it).
Maybe I'm misunderstanding the scope of the
Indeed, I'm not sure how to think about a URI if we're using Resource to represent some point in time data structure. |
Beta Was this translation helpful? Give feedback.
-
Is there a reason you think it needs to work this way? |
Beta Was this translation helpful? Give feedback.
-
You're right, I'm probably wrong here. There's nothing explicitly in the spec that enforces that, so that was just my own misinterpretation. I like your and @evalstate's thinking better as it makes Resources more flexible and allows us to support ideas like the one that kicked off this thread without new work on the spec. |
Beta Was this translation helpful? Give feedback.
-
|
Sorry, my previous answer was unhelpful - I was in spec review mode! Working through a couple of example workflows: LLM calls "generate ui" tool, and the MCP Server responds with a CallToolResult containing a TextEmbeddedResource with the mimeType of The Host would then recognise the URI scheme as a custom Resource Template that required UI rendering and display it. It MAY choose to subscribe to updates if the Server supports it. It's also fine for a URI to be disposable. A Host application that did this would be really really cool, as you could use the generated ui to send new messages to the LLM etc. It could also for example update based on new version numbers of the Sophisticated Client/Server pairs may choose to use Sampling for this kind of UI generation if appropriate. Again, there's nothing stopping the promotion of a URI scheme and commonly accepted practices for Hosts to do this kind of rendering. Another alternative would be to take the mpc-webcam approach. There's no reason why an MCP Server can't show it own UI, or generate it's UI via Sampling requests. Since those features are in mcp-webcam, forking it for a POC should be straightforward (sadly, time doesn't permit me to do this right now). There is a little guidance on some specific URI types here that is worth reviewing. It opens up implementation questions like So I think the specification enables this kind of workflow at the moment, would be interested in learning more on the req's - happy to assist in any POCs. |
Beta Was this translation helpful? Give feedback.
-
|
That's fair, I had a chat with @evalstate and he convinced me this is a valid use of Resources. My position now is that the original proposal can be accomplished using |
Beta Was this translation helpful? Give feedback.
-
|
@evalstate ya, in the content type discussion we had I got hung up on this is as well - here's the exact wording from the spec:
I would say this wording implies that the URI must be non-ephemeral. Maybe we can change this to make it more clear the URI is optional on the embedded resource, or the server MAY expose embedded resource uris |
Beta Was this translation helpful? Give feedback.
-
|
Agreed - I'll explain the reasoning behind my position:
I think it is worth updating that text as this could be inhibiting the take-up of one of the more powerful aspects of MCP. |
Beta Was this translation helpful? Give feedback.
-
|
Coming to this thread from modelcontextprotocol/modelcontextprotocol#356 (comment). I agree it would be nice to have more guidance in the spec about the usage of I also agree that URIs should be allowed to be ephemeral. Specifically, I would like to see URIs that are scoped to a live context window. So you might have multiple items in a tool result |
Beta Was this translation helpful? Give feedback.
-
|
@jonathanhefner - @evalstate just opened a PR modelcontextprotocol/modelcontextprotocol#415 :) |
Beta Was this translation helpful? Give feedback.
-
|
I like this proposal, although I think this issue is more on the HTMX side rather than on the React/Vue/Web Components/React Native side. For my requirements, I am more on https://github.com/BLamy side, envisioning making something like https://github.com/21st-dev/magic-mcp be done in a standard way. So I created a separate issue here for the components way: Let me know if it makes sense to merge here, or keep it separated. |
Beta Was this translation helpful? Give feedback.
-
|
I guess the question for me would be is this a product you're building on top of MCP (which is the kind of innovation MCP can drive) - or a need to change the specification to meet a goal. Bear in mind from the Specification front page:
It's quite possible that people will be using MCP in quite different environments for which the requirement (in the specification) to support a Rich UI wouldn't make sense. |
Beta Was this translation helpful? Give feedback.
-
|
I agree that it wouldn't make sense, but there should be a way to clients/servers that DO support such rendering, have a standard way to communicate its way. In its simple form it would be like Accept headers, in its complex form like Media Capabilities API |
Beta Was this translation helpful? Give feedback.
-
|
@tadasant I think we may need to make this more prominent as these discussions keep popping up. |
Beta Was this translation helpful? Give feedback.
-
|
Honestly, this was part of the long discussion we had around content types - I don't think it's at all obvious in the documentation that an embedded resource is just a free form data field, and by putting data in it you don't pickup any additional hosting requirements. There's probably a quick documentation fix or naming fix for this (not overloading the term resource would help, since you're sort of trained by the rest of the protocol AND by REST in general that a resource is an addressable thing. |
Beta Was this translation helpful? Give feedback.
-
|
I wrote up a little bit more about my thoughts on why I think this is important here: https://www.epicai.pro/the-future-of-ai-interaction-beyond-just-text-w22ps And made a video explaining it too: future-interaction.gh.mp4 |
Beta Was this translation helpful? Give feedback.
-
|
Sharing MCP UI, an SDK that implements both the server and client side of UI over MCP. It's meant to be used as a playground to bootstrap this initiative in practice. Hopefully, the community can test ideas until we find what works best. Thoughts, changes, and contributions are obviously more than welcome. Under the hood, the SDK enables any MCP server to respond with an Embedded Resource with a "ui://" or "ui-app://" URI. The client SDK allows hosts to render it in a supported method (currently raw HTML or external app) and handles follow-up interactions through events (e.g., tool calls). We need to find a better delivery method, perhaps RSC/remotedom/... mcpui-x.mp4 |
Beta Was this translation helpful? Give feedback.
-
|
Yes - this would be fantastic! On our MyLife platform we most definitely ship UI requests (totally prototype draft in our case awaiting more focused attention by frontend enthusiasts, of which I am not) to the frontend in order for UI elements (input and other) to be constructed, altered or destroyed based on intelligence rendering on the backend. |
Beta Was this translation helpful? Give feedback.
-
|
I really like this and feel like it's the direction LLM-infused software is heading. But, I don't think it's really necessary to specify, nor is it secure to return UI code that should be rendered. Rather, clients (who already have LLMs to use) can interpret any data returned by a server with the intent to render UI. What could be specified is a signal that the server may want to send along a hint that the response could trigger a generated UI. But, that should be entirely up to the client to handle independent of what the server says. As a client, it would not be safe to simply accept code to use for rendering from a server. Use the browser as an equivalent story here - we have frames to allow a similar experience but it's largely unused because of poor security and consistency across host-delivered UI. Instead, a browser client fetches data, and then uses it to conditionally render UI owned by the client. I think this is a prescient idea, which we will see happen without any specification -- and don't need to specify the behavior here because it adds a lot of surface area to the protocol that ultimately is more of an emergent outcome, vs. one defined in the spec. I'd be happy to know where there are entirely secure cases to take untrusted, executable code for rendering, or where we think today's client-side MCP implementations with access to LLMs can't already be prompted into rendering UI based on data from server responses. |
Beta Was this translation helpful? Give feedback.
-
|
I agree, although I'm not sure there's a need for the server to specify to the client that the response could generate a UI. Do you have an example? It feels like all UI generation will ultimately end up in the host because the host will have an agentic loop using a model fine tuned to generate the UI dynamically from low level primitives. This is really a generalisation of what we are starting to see with Claude Artifacts or Grok's UI. In that scenario, only data is returned from the tool calls. The host then moulds that data into whatever UI it sees fit, based on its understanding of the user’s preferences and the current task context. Given that, I’d prefer to keep the UI out of MCP and instead allow the host to optionally specify an output schema that the data should conform to, similar to what Google’s A2A protocol allows. |
Beta Was this translation helpful? Give feedback.
-
|
Let's imagine that your MCP server is really smart and what it wants to receive from the hosting environment is a (sandboxed) UI surface that it can do anything with -- WebGL visualizations, audio, javascript execution -- the whole thing. Maybe the protocol should be inverted -- the agent simply constrains this UI surface in some way when it calls the MCP server, and the server renders into the sandbox with those constraints/preferences. As always, if I can't enable my user agent to re-render your brand's font choices as Papyrus then it's not really my agent. TL;DR maybe start with iframes? |
Beta Was this translation helpful? Give feedback.
-
|
Very much agree with the sentiment. |
Beta Was this translation helpful? Give feedback.
-
|
Having the clients deciding on when and how to render UI (like today) might be enough for simple data visualization, but won't be enough for interactive UI snippets (for example choosing an option) - which
Currently |
Beta Was this translation helpful? Give feedback.
-
|
I agree, the best example here is picking an airplane seat. There's just no way to really handle that "elegantly" without some sort of structure like this, because you need to show where bathrooms are and stuff. There's no way to tell the llm how to render that. |
Beta Was this translation helpful? Give feedback.
-
|
Good example. Ultimately I think that can still be achieved without server rendering by having the MCP server expose resources with details about different airplane layouts, including for example an annotated SVG diagram of the plane. But it's a nice test case. |
Beta Was this translation helpful? Give feedback.
-
|
How about this |
Beta Was this translation helpful? Give feedback.
-
|
This is an interesting POC with mcp-ui and Shopify web components (public). Very rough around the edges of course, but note the flow being affected by user interaction (including opening a tab). This should be given solely to the client's decision of course. VID-20250526-WA0047.mp4 |
Beta Was this translation helpful? Give feedback.
-
|
This is the way 👆 |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
Pre-submission Checklist
Your Idea
There are some cases where text input and output is a great user experience, and other times the user may want to interact with a generated UI with regular buttons that they're used to (for example, a simple stopwatch), or even a dynamic component like a map or graph.
Making it so tools can respond with this kind of UI would increase the utility of the MCP spec.
Desired Response (updated in the spirit of modelcontextprotocol/modelcontextprotocol#180):
{ "mimeType": "text/html", "data": "<marquee>Look! UI!</marquee>" }There are some significant security concerns here for clients that don't render this in an isolated environment like an iframe (or potentially a realm in the future). There could be potentially a reason to only support iframe elements to enforce that isolation (or even a response type of
{ "type": "iframe", "src": "..." }) . But I think that it would be better if clients would manage this isolation automatically.I also think that a relatively powerful model for this would be React Server Components (
{ "type": "rsc", "data": "..." }), but we can leave that discussion for another time. Just starting with HTML would be pretty powerful, even if it was limited to iframe only.UPDATE: More on this here https://www.epicai.pro/the-future-of-ai-interaction-beyond-just-text-w22ps
Scope
Beta Was this translation helpful? Give feedback.
All reactions