Best Practises to Build an MCP Server

[ma3u]

The YouTube video by David Gomes of Neon, discussing Best Practises to build MCP servers:

Summary of the Source

• What is MCP? is a new, open protocol developed by Anthropic. It standardizes how applications provide context to LLMs and how LLMs can interface with real-world apps and services.
• Importance and Adoption: MCP is relatively new but gaining significant adoption. Windows Copilot, Google's Gemini, OpenAI (like ChatGPT), and Anthropic's Claude all support or will soon support MCP. If you want LLMs to use your application or service, you need an MCP server.
• Components of an MCP Server: An MCP server exposes tools, resources, and prompts to the LLM (which acts as the MCP client).
• Tools: Tools are the most important component. They can be thought of as actions that an LLM might want to perform, such as buying an item on an e-commerce site or creating a PostgreSQL database.
• Building MCP Servers: Many companies are building MCP servers to make their services available to LLMs. A common approach is to auto-generate an MCP server from an existing OpenAPI schema. While easy and quick, the source argues this is not the right approach.

Why User-Centric Design (Designing for LLMs) is Important

The source strongly emphasizes designing the MCP server specifically for the LLM 'user', which aligns with a user-centric approach.

• LLMs and Tool Choice: Existing APIs are often very extensive, having many endpoints. LLMs are really, really terrible at choosing from a long list of tools; they get confused if given too much choice. LLMs perform much better with a reduced context size. Auto-generating from an OpenAPI schema exposes all API endpoints as MCP tools, leading to poor performance. Therefore, you must make a choice and cut down the number of tools you expose.
• API Descriptions vs. LLM Needs: Existing API descriptions are typically written for humans who can reason and search for information. LLMs, however, need you to be a lot more direct with them and require examples much more than humans do. Writing tool descriptions for an LLM is different than writing for a human. Tool descriptions for LLMs should be organized and provide ample context on when to use the tool. The source's own MCP server uses XML for tool descriptions written in a specific, organized manner to give the LLM maximum context.
• Tasks vs. Low-Level Operations: Most APIs are designed for low-level resource management and automation for developers. LLMs, however, need tasks, actions, and tools; they are more human-like and care about achieving a specific goal rather than low-level creation. The design of MCP tools needs to be done with the LLM's goal in mind, which is not the typical approach for designing APIs. This fundamental difference is part of why MCP was created.

Why User Journey (Multi-Step Workflows) for MCP is Important

Thinking about how an LLM will use a sequence of tools to complete a task – essentially its 'user journey' – is highlighted as a key benefit of designing MCP tools purpose-built for LLMs.

• The source gives an example of database migrations. A naive approach would be to expose a single "Run SQL" tool and expect the LLM to figure out how to use it for migrations. While they do have a "Run SQL" tool, they also expose purpose-built MCP tools for database migrations: "Prepare Database Migration" and "Complete Database Migration".
• LLMs are eager to use these purpose-built tools over the generic "Run SQL" tool. This allows for a more structured workflow (or journey): the "Prepare" tool stages the migration on a temporary branch, the server tells the LLM to test it, and then teaches the LLM to call the "Complete" tool when ready.
• This kind of complex, multistep workflow doesn't make sense for a standard API but is perfectly suited for LLMs, especially since LLMs may not be great at SQL. This demonstrates designing the tools around a specific task flow that guides the LLM through a process.

How to Test These Best Practices

Testing is crucial because LLMs are non-deterministic.

• The source uses "Evals," which are tests in the AI world.
• These evals are used to make sure that LLMs are calling the right tool for the right job and to ensure the descriptions for the exposed tools are good.
• Evals should be run multiple times (100, 1,000, 10,000 times) due to the non-deterministic nature of LLMs.
• Evals evolve over time to help refine tool descriptions. If you are building an MCP server, you should have evals and tests to ensure LLMs can use it correctly.

Examples and TypeScript

• The source provides conceptual examples of tools like "Buy Item", "Create PostgreSQL Database", "Run SQL", "Prepare Database Migration", and "Complete Database Migration".
• It mentions their tool descriptions are written in XML.

In summary, building an effective MCP server requires deliberately designing the tools and workflows specifically for LLMs, providing clear, direct, and focused descriptions, and exposing a limited, task-oriented set of actions, rather than simply exposing an existing API. These practices should be validated through extensive testing using "Evals".

[ma3u]

I created a Epic to describe the user journey:

• User Journey 1: Claude Desktop access via MCP to Dust.tt (STDIO) #14

and created a test strategy:

• [Epic] Implement Comprehensive Test Strategy for MCP Server #8

[ma3u]

developers should take several deliberate actions rather than simply exposing an existing API. Here are the recommended actions:

1. Make a conscious choice and significantly reduce the number of tools exposed. Existing APIs tend to be very extensive, and LLMs are "really, really terrible at choosing from a long list of tools". Giving LLMs too much choice confuses them and leads to poor performance, even with large context windows. You must "cut down as many tools as possible" and remove any tools that are not considered essential for an LLM to consume.

2. Write tool descriptions specifically for LLMs. Descriptions in existing APIs are usually written for humans who can reason and search for information. LLMs, however, "need you to be a lot more direct with them" and require examples "much more than humans do". Tool descriptions for LLMs should be very organized and provide ample context on when each tool should be used. This is a different writing style than for humans. You should evaluate and refine these descriptions.

3. Design the MCP tools around tasks, actions, and the LLM's specific goals, rather than the low-level resource management focus of typical APIs. LLMs are "much more human-like" and care about achieving a goal, not just low-level creation. The design of the list of tools needs to be done with the LLM's purpose in mind. This fundamental difference is part of why MCP was created.

4. Consider creating purpose-built MCP tools for multi-step workflows or "journeys". Simply exposing a generic tool like "Run SQL" might be a naive approach for complex tasks like database migrations. Instead, take the opportunity to develop more interesting tools that guide the LLM through a process, even if such a "complex, multistep workflow" doesn't make sense in a standard API. This allows for capabilities like staging changes and guiding the LLM to test before completion.

5. Implement comprehensive testing using "Evals". Since LLMs are non-deterministic, testing is crucial. Evals are used to ensure that LLMs correctly identify and call the right tool for a given job. These evals should be run multiple times (hundreds, thousands, or even tens of thousands of times). Evals also help in refining and getting the tool descriptions right over time. If you are building an MCP server, you "should have evals and tests".

[ma3u]

Overview of the Proposed Multi-Agent MCP Architecture

The integration between Claude Desktop and Dust.tt through MCP creates a powerful multi-step workflow system where Claude Desktop acts as the orchestrating client, leveraging specialized Dust agents through standardized protocol communications. Dust's multi-tool agents can handle advanced workflows with multiple sub-steps, while the MCP Dust Serverprovides the bridge between these systems using JSON-RPC 2.0 protocol.

The workflow architecture follows the MCP specification for transport mechanisms, supporting both stdio and HTTP streaming transports. Claude Desktop connects to the MCP Dust Server through configuration files, establishing persistent sessions that maintain context across multi-step operations.

Multi-Step Workflow Examples

Enterprise Knowledge Discovery Workflow

Step 1: Initialize Research Context
Claude Desktop sends an initialization request through the MCP protocol to establish a research session with the Dust agent. The MCP Dust Server handles this via the dust-query tool, setting up proper authentication and workspace context using the configured Dust API credentials.

Step 2: Multi-Source Knowledge Retrieval
The Dust agent utilizes its semantic search capabilities to query multiple connected data sources simultaneously. This leverages Dust's ability to access Notion, Slack, GitHub, and other integrated platforms, with the MCP server streaming responses back to Claude Desktop through Server-Sent Events.

Step 3: Cross-Reference and Analysis
Dust's multi-tool agents automatically select appropriate tools for data analysis, potentially using Query Tables for structured data and web navigation for external context. Each tool execution generates structured responses that flow back through the MCP protocol to Claude Desktop.

Step 4: Synthesis and Reporting
Claude Desktop receives the aggregated data and performs final synthesis, potentially triggering additional Dust agent calls for specialized formatting or data extraction using Dust's Extract Data tool.

Customer Support Automation Workflow

Step 1: Context Establishment
Claude Desktop initiates the workflow by sending customer identification data through the MCP protocol. The Dust agent system processes this using its semantic search tools to retrieve relevant customer history and support context.

Step 2: Multi-Agent Coordination
Following patterns similar to LlamaIndex's AgentWorkflow orchestration-solution, different specialized Dust agents handle specific aspects: a concierge agent for initial triage, a technical agent for issue analysis, and a resolution agent for solution generation. The MCP server coordinates these handoffs through structured JSON-RPC messages.

Step 3: Real-Time Information Gathering
Dust agents use their Latest Data tool to fetch real-time information from support systems, while the Web Navigation tool gathers external context about known issues or solutions. The MCP protocol ensures streaming responses maintain session continuity.

Step 4: Solution Delivery and Follow-up
The final agent in the chain generates comprehensive responses, with the MCP server handling the delivery back to Claude Desktop through the established SSE connection, maintaining full audit trails of the multi-step process.

Technical Architecture Flow

MCP Protocol Layer

The communication between Claude Desktop and Dust follows the JSON-RPC 2.0 specification, with messages structured as requests, responses, and notifications.

Session Management

The MCP server maintains persistent sessions with Redis-based storage, ensuring context preservation across multi-step workflows. This addresses the challenge of maintaining state when multiple Dust agents are involved in a single user journey, with each step building upon previous results.

Transport Mechanisms

Following the MCP transport specifications, the system supports both stdio transport for local development and HTTP-based transports for production deployments. The SSE transport enables real-time streaming of agent responses, crucial for multi-step workflows where intermediate results need immediate visibility.

Implementation Considerations

Tool Registration and Discovery

Claude Desktop discovers available Dust capabilities through the MCP tool listing mechanism.

Error Handling and Recovery

Multi-step workflows require robust error handling, with the MCP server implementing automatic reconnection mechanisms and session recovery capabilities. When Dust agents encounter failures, the system can resume from the last successful step rather than restarting the entire workflow.

Security and Authentication

The workflow maintains security through secure API key handling with PII masking in logs, following enterprise-grade security practices.

Configuration Example

For Claude Desktop integration, the configuration follows this pattern based on the MCP quickstart guide:

{
  "mcpServers": {
    "dust-mcp": {
      "command": "npm",
      "args": ["run", "start:server"],
      "env": {
        "DUST_API_KEY": "${YOUR_API_KEY}",
        "DUST_WORKSPACE_ID": "${YOUR_WORKSPACE_ID}",
        "DUST_AGENT_ID": "${YOUR_AGENT_ID}"
      },
      "host": "127.0.0.1",
      "port": 5001,
      "timeout": 30000
    }
  }
}

This configuration enables Claude Desktop to establish persistent connections with the MCP Dust Server, facilitating complex multi-step workflows that leverage the full capabilities of [Dust's multi-agent system](https://blog.dust.tt/using-multiple-ai-assistants-a-pioneers-perspective/).

Workflow Orchestration Benefits

The combination of Claude Desktop's reasoning capabilities with Dust's specialized agents through MCP creates workflows that are more sophisticated than either system could achieve independently. [Dust's multi-agent approach](https://blog.dust.tt/using-multiple-ai-assistants-a-pioneers-perspective/) provides domain expertise and company context, while Claude Desktop offers advanced reasoning and orchestration capabilities, with the MCP protocol ensuring seamless integration and real-time collaboration between these AI systems.

This architecture represents a significant advancement in AI workflow capabilities, enabling organizations to build complex, multi-step processes that combine the strengths of different AI platforms while maintaining transparency, control, and enterprise-grade reliability throughout the entire workflow execution.

[ma3u]

Here are some sample scenarios and what you would be testing for in your evals, based on the you tube video provided:

• Scenario 1: Testing a basic tool like "Create a Postgres Database"

  • Prompt: "Build me a simple to-do list app using Neon postgres." (or a similar prompt asking the LLM to create a database)
  • Eval Goal: To confirm that the LLM correctly identifies and calls the "Create a Postgres Database" tool.
  • Sample Test Result (Expected): The eval would record that the LLM invoked the "Create a Postgres Database" tool with the necessary parameters (e.g., database name, potentially region or other configurations depending on your tool's requirements). The test would pass if the correct tool was called.
  • Sample Test Result (Failure): The eval would record that the LLM called an incorrect tool (e.g., a tool for deleting a database, or a tool for running SQL) or failed to call any tool. The test would fail in this case.

• Scenario 2: Testing the prioritization of purpose-built tools

  • Prompt: "I need to perform a database migration on my Neon database." (or a similar prompt indicating the need for a migration)
  • Eval Goal: To confirm that the LLM prioritizes calling the "Prepare Database Migration" tool over the more general "Run SQL" tool, as encouraged by Neon [5, 6].
  • Sample Test Result (Expected): The eval would record that the LLM called the "Prepare Database Migration" tool. The test would pass.
  • Sample Test Result (Failure): The eval would record that the LLM called the "Run SQL" tool instead of the "Prepare Database Migration" tool. The test would fail.

• Scenario 3: Testing a multi-step workflow

  • Prompt (Initial): "I need to perform a database migration on my Neon database."
  • Response from MCP server (simulated): The MCP server successfully calls the "Prepare Database Migration" tool, and then responds to the LLM indicating the migration is staged and to call the "Complete Database Migration" tool to finish.
  • Prompt (Follow-up by LLM): Based on the MCP server's previous response, the LLM would ideally issue a prompt that leads to calling the "Complete Database Migration" tool (e.g., "Complete the database migration").
  • Eval Goal: To confirm that the LLM correctly follows the multi-step workflow by calling the "Complete Database Migration" tool after the "Prepare Database Migration" tool has been used and the MCP server has provided instructions.
  • Sample Test Result (Expected): The eval would record a sequence of tool calls: first "Prepare Database Migration," followed by "Complete Database Migration." The test would pass.
  • Sample Test Result (Failure): The eval would record that the LLM did not call the "Complete Database Migration" tool after the initial step, or called an irrelevant tool. The test would fail.

These sample evals demonstrate the principle of testing whether the LLM selects the correct tool based on the user's intent, especially when there are similar or related tools available. The key is to design prompts that clearly indicate the desired task and then verify that the LLM interacts with your MCP server by calling the tool that is best suited for that task, as described by your tool descriptions.