Skip to content

Elimitate duplication between client_config_permutations and server.mcp_config #59

@iskhakov

Description

@iskhakov

The MCP catalog evaluation script currently extracts and stores two overlapping configurations for each MCP server:

  1. archestra_config.client_config_permutations.mcpServers - Contains multiple ways to run a server (docker, uvx, npx, etc.). This is displayed in the "Configuration" block on our website catalog (example:
    https://www.archestra.ai/mcp-catalog/upstash__context7)
  2. server.mcp_config - Contains one DXT-formatted configuration for running the server, where env variables reference values from user_config. This is what Archestra app uses to actually start MCP servers:
    Image

Both are extracted independently from READMEs using AI (via separate prompts) and located in /app/app/mcp-catalog/scripts/evaluate-catalog.ts.

Problem

  1. mcpServers extraction fails - LLM non-deterministic output causes incorrect config names, missed commands, or wrong parsing
  2. server.mcp_config extraction fails - LLM mixes different configurations, misses required env variables, or adds incorrect ones. MCP servers need exact env variables as documented, otherwise they crash

Solution

  1. Use regex patterns to extract archestra_config.client_config_permutations.mcpServers from documentation instead of relying on LLM interpretation, or propose your own approach
    1.1. Scan all markdown files in the repository, not just README (example: https://github.com/korotovsky/slack-mcp-server/blob/master/docs/03-configuration-and-usage.md#using-npx)
  2. Try to extract server.mcp_config from the original dxt manifest if available (https://github.com/korotovsky/slack-mcp-server/blob/master/manifest-dxt.json) with fallback to generating one from one of the archestra_config.client_config_permutations.mcpServers

Test

We should be able to use generated configs in Archestra or other MCP client to run MCP servers

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions