Skip to content

Commit 52c6ea3

Browse files
rdimitrovrdimitrov
authored
Add registry documentation - heuristics and management (#555)
Signed-off-by: Radoslav Dimitrov <radoslav@stacklok.com>
1 parent 43b53f6 commit 52c6ea3

File tree

3 files changed

+163
-0
lines changed

3 files changed

+163
-0
lines changed

‎README.md

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -295,6 +295,16 @@ server you'd like to see included, feel free to
295295
[open an issue](https://github.com/stacklok/toolhive/issues) or submit a pull
296296
request to update the [registry.json](pkg/registry/data/registry.json) file.
297297

298+
#### Registry curation
299+
300+
Our registry curation process uses a set of heuristics to evaluate MCP servers for quality,
301+
security, and compatibility before inclusion. We prioritize servers that demonstrate good
302+
documentation, active maintenance, and follow security best practices.
303+
304+
For detailed information about our curation process, see:
305+
- [Registry Heuristics](./docs/registry/heuristics.md)
306+
- [Registry Management](./docs/registry/management.md)
307+
298308
### Manage MCP servers
299309

300310
To list the running MCP servers:

‎docs/registry/heuristics.md

Lines changed: 122 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,122 @@
1+
# MCP Server Registry Inclusion Heuristics
2+
3+
## Overview
4+
5+
This document defines the criteria for including MCP (Model Context Protocol) servers in the ToolHive Registry. The goal is to establish a curated, community-auditable list of high-quality MCP servers through clear, observable, and objective criteria.
6+
7+
## Heuristics
8+
9+
### Open Source Requirements
10+
- Must be fully open source with no exceptions
11+
- Source code must be publicly accessible
12+
- Must use an acceptable open source license (see [Acceptable Licenses](#acceptable-licenses) below)
13+
14+
### Security
15+
- Software provenance verification (Sigstore, GitHub Attestations)
16+
- SLSA compliance level assessment
17+
- Pinned dependencies and GitHub Actions
18+
- Published Software Bill of Materials (SBOMs)
19+
20+
### Continuous Integration
21+
- Automated dependency updates (Dependabot, Renovate, etc.)
22+
- Automated security scanning
23+
- CVE monitoring
24+
- Code linting and quality checks
25+
26+
### Repository Metrics
27+
- Repository stars and forks
28+
- Commit frequency and recency
29+
- Contributor activity
30+
- Issue and PR statistics
31+
32+
### API Compliance
33+
- Full MCP API specification support
34+
- Implementation of all required endpoints (tools, resources, etc.)
35+
- Protocol version compatibility
36+
37+
### Tool Stability
38+
- Version consistency
39+
- Breaking change frequency
40+
- Backward compatibility maintenance
41+
42+
### Code Quality
43+
- Presence of automated tests
44+
- Test coverage percentage
45+
- Quality CI/CD implementation
46+
- Code review practices
47+
48+
### Documentation
49+
- Basic project documentation
50+
- API documentation
51+
- Deployment and operation guides
52+
- Regular documentation updates
53+
54+
### Release Process
55+
- Established CI-based release process
56+
- Regular release cadence
57+
- Semantic versioning compliance
58+
- Maintained changelog
59+
60+
### Community Health
61+
62+
#### Responsiveness
63+
- Active maintainer engagement
64+
- Regular commit activity
65+
- Timely issue and PR responses (issues open 3-4 weeks without response is a red flag)
66+
- Bug resolution rate
67+
- User support quality
68+
69+
#### Community Strength
70+
- Project backing (individual vs. organizational)
71+
- Number of active maintainers
72+
- Contributor diversity
73+
- Corporate or foundation support
74+
- Governance model maturity
75+
76+
### Security Requirements
77+
78+
#### Authentication & Authorization
79+
- Secure authentication mechanisms
80+
- Proper authorization controls
81+
- Standard security protocol support (OAuth, TLS)
82+
83+
#### Data Protection
84+
- Encryption for data in transit and at rest
85+
- Proper sensitive information handling
86+
87+
#### Security Practices
88+
- Clear incident response channels
89+
- Security issue reporting mechanisms (email, GHSA, etc.)
90+
91+
## Future Considerations
92+
93+
### Automated vs Manual Checks
94+
- Balance between automated checks (e.g., CI/CD, security scans) and manual reviews (e.g., community health, documentation quality)
95+
- Automated checks for basic compliance (e.g., license, API support)
96+
- Manual reviews for nuanced aspects (e.g., community strength, documentation quality)
97+
98+
### Scoring System
99+
- **Required**: Essential attributes (significant penalty if missing)
100+
- **Expected**: Typical well-executed project attributes (moderate score impact)
101+
- **Recommended**: Good practice indicators (positive contribution)
102+
- **Bonus**: Excellence demonstrators (pure positive, no penalty for absence)
103+
104+
### Tiered Classifications
105+
- "Verified" vs "Experimental/Community" designations
106+
- Minimum threshold requirements (stars, maintainers, community indicators)
107+
- Regular re-evaluation frequency for automated checks
108+
109+
## Acceptable Licenses
110+
111+
The following open source licenses are accepted for MCP servers in the ToolHive registry:
112+
113+
### Permissive Licenses
114+
Licenses such as Apache-2.0, MIT, BSD-2-Clause, BSD-3-Clause allow maximum flexibility
115+
for integration, modification, and redistribution with minimal restrictions,
116+
making MCP servers accessible across all project types and commercial applications.
117+
118+
### Excluded Licenses
119+
120+
Copyleft and restrictive licenses such as AGPL, GPL2 and 3 are excluded to ensure MCP servers can be
121+
freely integrated into various commercial and open source projects without legal
122+
complications or viral licensing requirements.

‎docs/registry/management.md

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
# MCP Server Registry Management Process
2+
3+
## Overview
4+
This document outlines the processes for managing MCP (Model Context Protocol) servers within the ToolHive registry, covering adding, removing, appealing decisions, and handling duplicate submissions.
5+
6+
## Adding MCP Servers
7+
1. Submit PR with required registry.json content
8+
2. Automated technical verification
9+
3. Manual review by registry maintainers
10+
4. Final approval or rejection decision
11+
12+
## Removing MCP Servers
13+
1. Automated non-compliance detection
14+
2. Notification to registry maintainers
15+
3. Grace period for remediation
16+
4. Final review and decision
17+
5. Public notification with reasoning
18+
19+
## Appeals Process
20+
- Open to MCP server users and maintainers
21+
- Based on objective criteria
22+
- Transparent communication of outcomes
23+
24+
## Handling Duplicates
25+
- Assess functional differentiation from existing entries
26+
- Prioritize based on:
27+
- Community adoption and activity levels
28+
- Overall code quality
29+
- Long-term viability and backing
30+
- Add deprecation notices before removal (1-2 month transition period)
31+
- Document rationale for decisions

0 commit comments

Comments
 (0)