MCPVals – Comprehensive Usage & Reference Guide

A comprehensive evaluation library for Model Context Protocol (MCP) servers. Test and validate your MCP servers with complete MCP specification coverage including Tools, Resources, Prompts, and Sampling with deterministic metrics, security validation, and optional LLM-based evaluation.

Status: MVP – API stable, minor breaking changes possible before 1.0.0

---

0. Quick Start

1. Installation

# Install – pick your favourite package manager
pnpm add -D mcpvals            # dev-dependency is typical

2. Create a config file

Create a config file (e.g., mcp-eval.config.ts):

import type { Config } from "mcpvals";

export default {
  server: {
    transport: "stdio",
    command: "node",
    args: ["./example/simple-mcp-server.js"],
  },

  // Test individual tools directly
  toolHealthSuites: [
    {
      name: "Calculator Health Tests",
      tests: [
        {
          name: "add",
          args: { a: 5, b: 3 },
          expectedResult: 8,
          maxLatency: 500,
        },
        {
          name: "divide",
          args: { a: 10, b: 0 },
          expectedError: "division by zero",
        },
      ],
    },
  ],

  // Test resources (data and context)
  resourceSuites: [
    {
      name: "Data Access Tests",
      tests: [
        {
          name: "read-config",
          uri: "config://settings",
          expectedMimeType: "application/json",
          maxLatency: 1000,
        },
      ],
    },
  ],

  // Test prompts (templates and workflows)
  promptSuites: [
    {
      name: "Prompt Template Tests",
      tests: [
        {
          name: "user-greeting",
          args: { name: "Alice", role: "admin" },
          expectedContent: ["Welcome", "Alice"],
        },
      ],
    },
  ],

  // Test sampling (LLM requests from server)
  samplingSuites: [
    {
      name: "AI Capability Tests",
      capabilityTests: [
        {
          name: "sampling-supported",
          expectedCapability: true,
        },
      ],
    },
  ],

  // Test OAuth 2.1 authentication flows
  oauth2Suites: [
    {
      name: "OAuth Security Tests",
      authorizationCodeTests: [
        {
          name: "Authorization Code with PKCE",
          flow: "authorization_code",
          server: {
            authorizationEndpoint: "https://auth.example.com/authorize",
            tokenEndpoint: "https://auth.example.com/token",
            supportedGrantTypes: ["authorization_code"],
            supportedScopes: ["read", "write"],
            pkceRequired: true,
          },
          client: {
            clientId: "test-client-id",
            responseType: "code",
            scope: ["read", "write"],
            redirectUri: "https://app.example.com/callback",
            pkce: { enabled: true, codeChallengeMethod: "S256" },
          },
          simulateUserConsent: true,
          expectedResult: "success",
        },
      ],
    },
  ],

  // Test multi-step, LLM-driven workflows
  workflows: [
    {
      name: "Multi-step Calculation",
      steps: [
        {
          user: "Calculate (5 + 3) * 2, then divide by 4",
          expectedState: "4",
        },
      ],
      expectTools: ["add", "multiply", "divide"],
    },
  ],

  // Optional LLM judge
  llmJudge: true,
  openaiKey: process.env.OPENAI_API_KEY,
  passThreshold: 0.8,
} satisfies Config;

3. Run Evaluation

# Required for workflow execution
export ANTHROPIC_API_KEY="sk-ant-..."

# Optional for LLM judge
export OPENAI_API_KEY="sk-..."

# Run everything
npx mcpvals eval mcp-eval.config.ts

# Run only tool health tests
npx mcpvals eval mcp-eval.config.ts --tool-health-only

# Run only resource evaluation tests
npx mcpvals eval mcp-eval.config.ts --resources-only

# Run only prompt evaluation tests
npx mcpvals eval mcp-eval.config.ts --prompts-only

# Run only sampling evaluation tests
npx mcpvals eval mcp-eval.config.ts --sampling-only

# Run only OAuth 2.1 authentication tests
npx mcpvals eval mcp-eval.config.ts --oauth-only

# Run with LLM judge and save report
npx mcpvals eval mcp-eval.config.ts --llm-judge --reporter json > report.json

---

1. Core Concepts

MCPVals provides comprehensive testing for all MCP specification primitives:

1. Tool Health Testing: Directly calls individual tools with specific arguments to verify their correctness, performance, and error handling. This is ideal for unit testing and regression checking.

2. Resource Evaluation: Tests MCP resources (data and context) including discovery, access validation, URI template testing, content validation, and subscription capabilities.

3. Prompt Evaluation: Validates MCP prompts (templates and workflows) with argument testing, template generation, security validation, and dynamic content evaluation.

4. Sampling Evaluation: Tests MCP sampling capabilities (server-initiated LLM requests) including capability negotiation, human-in-the-loop workflows, model preferences, and security controls.

5. Workflow Evaluation: Uses a large language model (LLM) to interpret natural language prompts and execute a series of tool calls to achieve a goal. This tests the integration of your MCP primitives from an LLM's perspective.

6. OAuth 2.1 Testing: Comprehensive OAuth 2.1 authentication flow testing with PKCE, resource indicators (RFC 8707), multi-tenant support, and security validation. Tests authorization code flows, client credentials, device flows, token management, and security controls.

---

2. Installation & Runtime Requirements

1. Node.js ≥ 18 – we rely on native fetch, EventSource, and fs/promises.
2. pnpm / npm / yarn – whichever you prefer, MCPVals is published as an ESM‐only package.
3. MCP Server – a local stdio binary or a remote Streaming-HTTP endpoint.
4. Anthropic API Key – Required for workflow execution (uses Claude to drive tool calls). Set via ANTHROPIC_API_KEY environment variable.
5. (Optional) OpenAI key – Only required if using the LLM judge feature. Set via OPENAI_API_KEY.

ESM-only: You cannot require("mcpvals") from a CommonJS project. Either enable "type": "module" in your package.json or use dynamic import().

---

3. CLI Reference

Usage: mcpvals <command>

Commands:
  eval <config>   Evaluate MCP servers using workflows and/or tool health tests
  list <config>   List workflows in a config file
  help [command]  Show help                                [default]

Evaluation options:
  -d, --debug              Verbose logging (child-process stdout/stderr is piped)
  -r, --reporter <fmt>     console | json | junit (JUnit coming soon)
  --llm-judge              Enable LLM judge (requires llmJudge:true + key)
  --tool-health-only       Run only tool health tests, skip others
  --resources-only         Run only resource evaluation tests, skip others
  --prompts-only           Run only prompt evaluation tests, skip others
  --sampling-only          Run only sampling evaluation tests, skip others
  --oauth-only             Run only OAuth 2.1 authentication tests, skip others
  --workflows-only         Run only workflows, skip other test types

3.1 eval

Runs tests specified in the config file. It will run all configured test types (toolHealthSuites, resourceSuites, promptSuites, samplingSuites, and workflows) by default. Use flags to run only specific types. Exits 0 on success or 1 on any failure – perfect for CI.

3.2 list

Static inspection – prints workflows without starting the server. Handy when iterating on test coverage.

---

4. Configuration

MCPVals loads either a .json file or a .ts/.js module that export default an object. Any string value in the config supports Bash-style environment variable interpolation ${VAR}.

4.1 server

Defines how to connect to your MCP server.

• transport: stdio, shttp (Streaming HTTP), or sse (Server-Sent Events).
• command/args: (for stdio) The command to execute your server.
• env: (for stdio) Environment variables to set for the child process.
• url/headers: (for shttp and sse) The endpoint and headers for a remote server.
• reconnect/reconnectInterval/maxReconnectAttempts: (for sse) Reconnection settings for SSE connections.

Example shttp with Authentication:

{
  "server": {
    "transport": "shttp",
    "url": "https://api.example.com/mcp",
    "headers": {
      "Authorization": "Bearer ${API_TOKEN}",
      "X-API-Key": "${API_KEY}"
    }
  }
}

Example sse with Reconnection:

{
  "server": {
    "transport": "sse",
    "url": "https://api.example.com/mcp/sse",
    "headers": {
      "Accept": "text/event-stream",
      "Cache-Control": "no-cache",
      "Authorization": "Bearer ${API_TOKEN}"
    },
    "reconnect": true,
    "reconnectInterval": 5000,
    "maxReconnectAttempts": 10
  }
}

4.2 toolHealthSuites[]

An array of suites for testing tools directly. Each suite contains:

• name: Identifier for the test suite.
• tests: An array of individual tool tests.
• parallel: (boolean) Whether to run tests in the suite in parallel (default: false).
• timeout: (number) Override the global timeout for this suite.

Tool Test Schema

Field	Type	Description
name	string	Tool name to test (must match an available MCP tool).
description	string?	What this test validates.
args	object	Arguments to pass to the tool.
expectedResult	any?	Expected result. Uses deep equality for objects, contains for strings.
expectedError	string?	Expected error message if the tool should fail.
maxLatency	number?	Maximum acceptable latency in milliseconds.
retries	number?	Retries on failure (0-5, default: 0).

4.3 resourceSuites[]

An array of suites for testing MCP resources. Each suite contains:

• name: Identifier for the resource test suite.
• discoveryTests: Tests for resource discovery and enumeration.
• tests: Resource access and content validation tests.
• templateTests: URI template instantiation tests.
• subscriptionTests: Resource subscription and update tests.
• parallel: (boolean) Whether to run tests in parallel (default: false).
• timeout: (number) Override the global timeout for this suite.

Resource Test Types

Discovery Tests: Validate listResources functionality

{
  name: "resource-discovery",
  expectedCount: { min: 1, max: 10 }, // Optional count validation
  expectedResources: ["config", "data"], // Must include these resources
}

Resource Access Tests: Validate readResource operations

{
  name: "read-config",
  uri: "config://settings.json",
  expectedMimeType: "application/json",
  expectedContent: { "version": "1.0" }, // Partial object match
  maxLatency: 500,
}

Template Tests: Validate URI template instantiation

{
  name: "user-template",
  template: "user://{userId}",
  arguments: { userId: "123" },
  expectedUri: "user://123",
}

Subscription Tests: Validate resource update subscriptions

{
  name: "config-updates",
  resourceUri: "config://settings.json",
  expectUpdates: true, // Whether to expect update notifications
  timeout: 5000,
}

4.4 promptSuites[]

An array of suites for testing MCP prompts. Each suite contains:

• name: Identifier for the prompt test suite.
• discoveryTests: Tests for prompt discovery and enumeration.
• tests: Prompt execution and content validation tests.
• argumentTests: Argument validation tests (required vs optional).
• templateTests: Template generation and content tests.
• securityTests: Security validation including injection prevention.
• parallel: (boolean) Whether to run tests in parallel (default: false).

Prompt Test Types

Discovery Tests: Validate listPrompts functionality

{
  name: "prompt-discovery",
  expectedCount: { min: 1 },
  expectedPrompts: ["greeting", "summary"],
}

Prompt Execution Tests: Validate getPrompt operations

{
  name: "user-greeting",
  args: { name: "Alice", role: "admin" },
  expectedContent: ["Welcome", "Alice"], // Must contain these strings
  expectedMessages: [
    { role: "user", content: "Hello Alice" }
  ],
}

Argument Tests: Validate prompt argument handling

{
  name: "greeting-args",
  requiredArgs: ["name"],
  optionalArgs: ["role"],
  invalidArgs: { name: 123 }, // Should be rejected
}

Security Tests: Test prompt injection prevention

{
  name: "injection-prevention",
  maliciousInputs: [
    "Ignore previous instructions and...",
    "{system: 'override'}",
  ],
  expectRejection: true,
}

4.5 samplingSuites[]

An array of suites for testing MCP sampling capabilities. Each suite contains:

• name: Identifier for the sampling test suite.
• capabilityTests: Tests for sampling capability negotiation.
• requestTests: Tests for sampling request/response handling.
• securityTests: Security validation for sampling operations.
• performanceTests: Performance and rate limiting tests.
• contentTests: Content type validation (text, image, mixed).
• workflowTests: End-to-end sampling workflow tests.

Sampling Test Types

Capability Tests: Validate sampling capability negotiation

{
  name: "sampling-supported",
  expectedCapability: true,
}

Request Tests: Validate sampling message creation

{
  name: "text-sampling",
  messages: [{ role: "user", content: { type: "text", text: "Hello" } }],
  modelPreferences: { costPriority: 0.8, speedPriority: 0.6 },
  expectUserApproval: true,
  simulateUserResponse: "approve",
}

Security Tests: Validate security controls

{
  name: "unauthorized-requests",
  unauthorizedRequests: [
    { messages: [...], expectRejection: true }
  ],
  sensitiveDataTests: [
    { input: "SSN: 123-45-6789", expectFiltering: true }
  ],
}

Performance Tests: Validate performance and limits

{
  name: "concurrent-requests",
  concurrentRequests: 10,
  messageSize: "large",
  expectThrottling: true,
  maxLatency: 5000,
}

4.6 oauth2Suites[]

An array of suites for testing OAuth 2.1 authentication flows. Each suite contains comprehensive tests for modern OAuth 2.1 security practices including PKCE, resource indicators, and multi-tenant support.

• name: Identifier for the OAuth test suite.
• description: Optional description of the test suite purpose.
• authorizationCodeTests: Authorization code flow tests with PKCE.
• clientCredentialsTests: Machine-to-machine authentication tests.
• deviceCodeTests: Device authorization flow tests for input-limited devices.
• tokenManagementTests: Token refresh, revocation, and expiration tests.
• pkceValidationTests: PKCE (Proof Key for Code Exchange) security validation.
• resourceIndicatorTests: RFC 8707 resource indicators for audience restriction.
• multiTenantTests: Multi-tenant isolation and access control tests.
• parallel: (boolean) Whether to run tests in parallel (default: false).
• timeout: (number) Override the global timeout for this suite.

OAuth 2.1 Test Types

Authorization Code Flow Tests: Complete OAuth 2.1 authorization code flow with PKCE

{
  name: "Authorization Code with PKCE",
  flow: "authorization_code",
  server: {
    authorizationEndpoint: "https://auth.example.com/authorize",
    tokenEndpoint: "https://auth.example.com/token",
    supportedGrantTypes: ["authorization_code"],
    supportedScopes: ["read", "write"],
    pkceRequired: true
  },
  client: {
    clientId: "test-client-id",
    responseType: "code",
    scope: ["read", "write"],
    redirectUri: "https://app.example.com/callback",
    pkce: {
      enabled: true,
      codeChallengeMethod: "S256"
    }
  },
  simulateUserConsent: true,
  expectedResult: "success",
  securityChecks: {
    validatePKCE: true,
    validateState: true,
    checkTokenExpiration: true
  }
}

Token Management Tests: Refresh, revocation, and expiration validation

{
  name: "Token Refresh Test",
  testType: "refresh",
  token: {
    accessToken: "your-jwt-token-here",
    refreshToken: "refresh-token-example",
    tokenType: "Bearer",
    expiresIn: 3600,
    scope: ["read", "write"]
  },
  expectedResult: "success",
  validateTokenClaims: true
}

PKCE Validation Tests: Security validation for Proof Key for Code Exchange

{
  name: "Valid PKCE S256 Challenge",
  codeChallengeMethod: "S256",
  invalidChallenge: false,
  expectedResult: "success",
  securityLevel: "high"
}

Resource Indicator Tests: RFC 8707 audience restriction validation

{
  name: "Resource-Specific Token",
  resourceUri: "https://api.example.com",
  requestedScopes: ["api:read", "api:write"],
  expectedAudience: "https://api.example.com",
  validateAudienceRestriction: true,
  expectedResult: "success"
}

Multi-Tenant Tests: Tenant isolation and cross-tenant access control

{
  name: "Cross-Tenant Access Blocked",
  primaryTenant: {
    tenantId: "tenant-123",
    isolationLevel: "strict",
    crossTenantAccess: false
  },
  secondaryTenant: {
    tenantId: "tenant-456",
    isolationLevel: "strict",
    crossTenantAccess: false
  },
  testScenario: "cross_tenant_blocked",
  expectedResult: "success",
  validateTenantIsolation: true
}

4.7 workflows[]

An array of LLM-driven test workflows. Each workflow contains:

• name: Identifier for the workflow.
• steps: An array of user interactions (usually just one for a high-level goal).
• expectTools: An array of tool names expected to be called during the workflow.

Workflow Step Schema

Field	Type	Description
user	string	High-level user intent. The LLM will plan how to accomplish this.
expectedState	string?	A sub-string the evaluator looks for in the final assistant message or tool result.

Workflow Best Practices

1. Write natural prompts: Instead of micro-managing tool calls, give the LLM a complete task (e.g., "Book a flight from SF to NY for next Tuesday and then find a hotel near the airport.").
2. Use workflow-level expectTools: List all tools you expect to be used across the entire workflow to verify the LLM's plan.

4.8 Global Options

• timeout: (number) Global timeout in ms for server startup and individual tool calls. Default: 30000.
• llmJudge: (boolean) Enables the LLM Judge feature. Default: false.
• openaiKey: (string) OpenAI API key for the LLM Judge.
• judgeModel: (string) The model to use for judging. Default: "gpt-4o".
• passThreshold: (number) The minimum score (0-1) from the LLM Judge to pass. Default: 0.8.

---

5. Evaluation & Metrics

5.1 Tool Health Metrics

When running tool health tests, the following is assessed for each test:

• Result Correctness: Does the output match expectedResult?
• Error Correctness: If expectedError is set, did the tool fail with a matching error?
• Latency: Did the tool respond within maxLatency?
• Success: Did the tool call complete without unexpected errors?

5.2 Resource Evaluation Metrics

For resource tests, the following is assessed:

• Discovery Metrics: Resource count validation, expected resource presence
• Access Metrics: Successful resource reading, MIME type validation, content correctness
• Template Metrics: URI template instantiation, parameter substitution accuracy
• Subscription Metrics: Update notification handling, subscription lifecycle management
• Performance Metrics: Response latency, retry success rates

5.3 Prompt Evaluation Metrics

For prompt tests, the following is assessed:

• Discovery Metrics: Prompt availability and enumeration
• Execution Metrics: Prompt generation success, content validation, message structure
• Argument Metrics: Required/optional parameter handling, validation correctness
• Template Metrics: Dynamic content generation, parameter substitution
• Security Metrics: Injection prevention, input sanitization effectiveness

5.4 Sampling Evaluation Metrics

For sampling tests, the following is assessed:

• Capability Metrics: Sampling support detection and negotiation
• Request Metrics: Message creation, model preference handling, approval workflows
• Security Metrics: Unauthorized request blocking, sensitive data filtering, privacy protection
• Performance Metrics: Concurrent request handling, rate limiting, latency management
• Content Metrics: Text/image/mixed content validation, format handling

5.5 OAuth 2.1 Evaluation Metrics

For OAuth 2.1 authentication tests, the following is assessed:

• Flow Completion Metrics: Successful completion of authorization code, client credentials, and device code flows
• PKCE Security Metrics: Code challenge/verifier validation, S256 method enforcement, replay attack prevention
• Token Management Metrics: Token refresh success, revocation effectiveness, expiration validation
• Security Validation Metrics: State parameter validation, nonce verification, audience restriction compliance
• Multi-Tenant Metrics: Tenant isolation enforcement, cross-tenant access blocking, tenant switching validation
• Resource Indicator Metrics: RFC 8707 compliance, audience restriction, scope validation
• Performance Metrics: Token endpoint latency, authorization flow completion time, concurrent request handling

5.6 Workflow Metrics (Deterministic)

For each workflow, a trace of the LLM interaction is recorded and evaluated against 3 metrics:

#	Metric	Pass Criteria
1	End-to-End Success	expectedState is found in the final response.
2	Tool Invocation Order	The tools listed in expectTools were called in the exact order specified.
3	Tool Call Health	All tool calls completed successfully (no errors, HTTP 2xx, etc.).

The overall score is an arithmetic mean. The evaluation fails if any metric fails.

5.7 LLM Judge (Optional)

Add subjective grading when deterministic checks are not enough (e.g., checking tone, or conversational quality).

• Set "llmJudge": true in the config and provide an OpenAI key.
• Use the --llm-judge CLI flag.

The judge asks the specified judgeModel for a score and a reason. A 4th metric, LLM Judge, is added to the workflow results, which passes if score >= passThreshold.

---

6. Library API

You can run evaluations programmatically.

import { evaluate } from "mcpvals";

const report = await evaluate("./mcp-eval.config.ts", {
  debug: process.env.CI === undefined,
  reporter: "json",
  llmJudge: true,
});

if (!report.passed) {
  process.exit(1);
}

7.1 Re-exported Types

The library exports all configuration and result types for use in TypeScript projects:

Configuration Types:

• Config, Workflow, WorkflowStep, ToolTest, ToolHealthSuite
• ResourceSuite, ResourceTest, ResourceDiscoveryTest, ResourceTemplateTest, ResourceSubscriptionTest
• PromptSuite, PromptTest, PromptArgumentTest, PromptTemplateTest, PromptSecurityTest
• SamplingSuite, SamplingCapabilityTest, SamplingRequestTest, SamplingSecurityTest, SamplingPerformanceTest, SamplingContentTest, SamplingWorkflowTest
• OAuth2TestSuite, AuthorizationCodeTest, ClientCredentialsTest, DeviceCodeTest, TokenManagementTest, PKCEValidationTest, ResourceIndicatorTest, MultiTenantTest

Result Types:

• EvaluationReport, WorkflowEvaluation, EvaluationResult
• ToolHealthResult, ToolTestResult
• ResourceSuiteResult, ResourceDiscoveryResult, ResourceTestResult, ResourceTemplateResult, ResourceSubscriptionResult
• PromptSuiteResult, PromptDiscoveryResult, PromptTestResult, PromptArgumentResult, PromptTemplateResult, PromptSecurityResult
• SamplingSuiteResult, SamplingCapabilityResult, SamplingRequestResult, SamplingSecurityResult, SamplingPerformanceResult, SamplingContentResult, SamplingWorkflowResult
• OAuth2SuiteResult, OAuth2TestResult, TokenManager, PKCEUtils, SecurityUtils
• runLlmJudge, LlmJudgeResult

---

8. Extensibility & Troubleshooting

• Custom Reporters: Import ConsoleReporter for reference and implement your own .report() method.
• Server Hangs: Increase the timeout value in your config. Ensure your server writes MCP messages to stdout.
• LLM Judge Fails: Use --debug to inspect the raw model output for malformed JSON.

---

9. Roadmap

✅ Completed (v0.1.0):

• Complete MCP specification coverage (Tools, Resources, Prompts, Sampling)
• Resource evaluation with discovery, access, templates, and subscriptions
• Prompt evaluation with execution, arguments, templates, and security testing
• Sampling evaluation with capability, requests, security, and performance testing
• OAuth 2.1 authentication testing with PKCE, resource indicators, and multi-tenant support
• Comprehensive security validation framework
• Enhanced console reporting for all evaluation types
• Server-Sent Events (SSE) transport support with automatic reconnection

🚧 In Progress:

• JUnit XML reporter for CI integration
• Advanced security testing extensions
• Performance benchmarking and comparison tools

📋 Planned (v0.2.0):

• Fluent API alongside configuration files
• Interactive CLI for test generation
• Output-schema validation for tool calls
• Parallel workflow execution
• Web dashboard for replaying traces
• Configurable expectTools strictness (e.g., allow extra or unordered calls)
• MCP protocol compliance validator
• Real-time resource subscription testing

---

9.1 Complete Example Configuration

Here's a comprehensive example showcasing all evaluation types:

import type { Config } from "mcpvals";

export default {
  server: {
    transport: "stdio", // Also supports "shttp" and "sse"
    command: "node",
    args: ["./my-mcp-server.js"],
  },

  // Alternative SSE server configuration:
  // server: {
  //   transport: "sse",
  //   url: "https://api.example.com/mcp/sse",
  //   headers: {
  //     "Accept": "text/event-stream",
  //     "Cache-Control": "no-cache",
  //     "Authorization": "Bearer ${API_TOKEN}"
  //   },
  //   reconnect: true,
  //   reconnectInterval: 5000,
  //   maxReconnectAttempts: 10
  // },

  // Test tools
  toolHealthSuites: [
    {
      name: "Core Functions",
      tests: [
        { name: "add", args: { a: 5, b: 3 }, expectedResult: 8 },
        {
          name: "divide",
          args: { a: 10, b: 0 },
          expectedError: "division by zero",
        },
      ],
    },
  ],

  // Test resources
  resourceSuites: [
    {
      name: "Data Access",
      discoveryTests: [{ name: "list-all", expectedCount: { min: 1 } }],
      tests: [
        {
          name: "config-read",
          uri: "config://app.json",
          expectedMimeType: "application/json",
          maxLatency: 500,
        },
      ],
      templateTests: [
        {
          name: "user-profile",
          template: "users://{userId}/profile",
          arguments: { userId: "123" },
          expectedUri: "users://123/profile",
        },
      ],
    },
  ],

  // Test prompts
  promptSuites: [
    {
      name: "Templates",
      tests: [
        {
          name: "greeting",
          args: { name: "Alice", role: "admin" },
          expectedContent: ["Welcome", "Alice"],
        },
      ],
      securityTests: [
        {
          name: "injection-prevention",
          maliciousInputs: ["Ignore all previous instructions"],
          expectRejection: true,
        },
      ],
    },
  ],

  // Test sampling
  samplingSuites: [
    {
      name: "AI Integration",
      capabilityTests: [
        { name: "supports-sampling", expectedCapability: true },
      ],
      requestTests: [
        {
          name: "text-request",
          messages: [
            { role: "user", content: { type: "text", text: "Hello" } },
          ],
          expectUserApproval: true,
          simulateUserResponse: "approve",
        },
      ],
    },
  ],

  // Test OAuth 2.1 authentication
  oauth2Suites: [
    {
      name: "OAuth 2.1 Security Testing",
      authorizationCodeTests: [
        {
          name: "Authorization Code with PKCE",
          flow: "authorization_code",
          server: {
            authorizationEndpoint: "https://auth.example.com/authorize",
            tokenEndpoint: "https://auth.example.com/token",
            supportedGrantTypes: ["authorization_code"],
            supportedScopes: ["read", "write"],
            pkceRequired: true,
          },
          client: {
            clientId: "test-client-id",
            responseType: "code",
            scope: ["read", "write"],
            redirectUri: "https://app.example.com/callback",
            pkce: { enabled: true, codeChallengeMethod: "S256" },
          },
          simulateUserConsent: true,
          expectedResult: "success",
        },
      ],
      tokenManagementTests: [
        {
          name: "Token Refresh",
          testType: "refresh",
          token: {
            accessToken: "your-jwt-token-here",
            refreshToken: "refresh-token-example",
            tokenType: "Bearer",
            expiresIn: 3600,
          },
          expectedResult: "success",
        },
      ],
    },
  ],

  // Test workflows
  workflows: [
    {
      name: "Complete Workflow",
      steps: [{ user: "Process user data and generate a report" }],
      expectTools: ["fetch-data", "process", "generate-report"],
    },
  ],

  llmJudge: true,
  openaiKey: process.env.OPENAI_API_KEY,
  timeout: 30000,
} satisfies Config;

---

10. Acknowledgements

• Model Context Protocol – for the SDK
• Vercel AI SDK – for LLM integration
• chalk – for terminal colors

Enjoy testing your MCP servers – PRs, issues & feedback welcome! ✨