RFC: Migrating MCP Registry from MongoDB to PostgreSQL

[sambhav]

Pre-submission Checklist

• I have verified this would not be more appropriate as a feature request in a specific repository
• I have searched existing discussions to avoid duplicates

Your Idea

RFC: Migrating MCP Registry from MongoDB to PostgreSQL

Summary

This RFC proposes migrating the MCP Registry from MongoDB to PostgreSQL.

Background

The MCP Registry is currently implemented using MongoDB as its database backend. The project is in early development stages with no production deployments, making this an opportune moment to reconsider our database technology choice before wider adoption.

Motivation

Several factors motivate this proposed change:

1. Licensing Concerns: MongoDB's SSPL license restricts third-party service offerings, potentially limiting our hosting options.
2. Deployment Flexibility: PostgreSQL is available on more cloud platforms with consistent behavior across providers.
3. Data Integrity Requirements: As a registry, data consistency and ACID compliance are critical.
4. Performance for Our Workload: PostgreSQL may provide better query performance for our specific access patterns.
5. Go Language Ecosystem: Strong PostgreSQL support in the Go ecosystem.
6. Timing Advantage: With no existing users, we can make this change now without complex migration procedures.

Technical Details

Data Modeling Options in PostgreSQL

PostgreSQL offers multiple approaches that can accommodate our needs:

1. JSONB for Document Storage:

   • Store JSON documents similar to MongoDB's approach
   • Binary format improves query performance
   • Supports indexing on document properties
   • Allows for schema flexibility

2. Traditional Relational Model:

   • Normalized tables with foreign key relationships
   • Strong type checking and constraints
   • Optimized for complex queries

3. Hybrid Approach:

   • Core entities in relational tables
   • Extensible or variable attributes in JSONB columns
   • Best of both worlds for evolving schemas

Scaling and Performance

• Vertical scaling for initial growth
• Horizontal scaling via table partitioning for larger deployments
• Connection pooling for optimizing concurrent connections
• Read replicas for distributing query load

Go Implementation Considerations

• Multiple mature drivers available (pgx, pq)
• Code generation tools for type-safe queries (sqlc)
• ORM options if preferred (ent, gorm)
• Strong JSON handling aligns with Go's capabilities

Benefits

1. Better Hosting Options: More cloud providers support PostgreSQL
2. Data Integrity: Strong ACID compliance ensures registry reliability
3. Query Performance: Potentially better performance for complex queries
4. Schema Flexibility: Option to use JSONB for document-like storage or traditional tables
5. Go Ecosystem: Better tooling and type safety with Go drivers
6. Future-Proofing: Protection against licensing changes
7. Simplified Architecture: Single database rather than potentially multiple systems
8. Timing Advantage: Making this change now avoids future migration complexity

Risks and Mitigations

Risk	Mitigation
Schema design challenges	Prototype both JSONB and relational approaches
Development timeline impact	Integrate changes with current development cycle

Open Questions

1. Which PostgreSQL data modeling approach is most appropriate for the registry?

   • Primary JSONB document storage
   • Traditional relational tables
   • Hybrid approach

2. What specific performance characteristics are most important for the registry?

   • Read performance
   • Write performance
   • Query complexity support

3. What deployment environments must we support?

   • Self-hosted
   • Specific cloud providers
   • Kubernetes environments

4. Who will be responsible for implementing this change?

References

1. PostgreSQL Documentation: https://www.postgresql.org/docs/
2. PostgreSQL JSONB: https://www.postgresql.org/docs/current/datatype-json.html
3. PostgreSQL Go Driver (pgx): https://github.com/jackc/pgx

Scope

• Protocol Specification
• SDK Features
• Documentation
• Developer Experience
• Other

[toby]

I agree that Postgres should ultimately be the database for the registry. I really like how Musicbrainz does their db and would opt for traditional RDBS tables. They also do a dump twice a week, which I think would be a cool idea for the registry as well.

I think for now, Mongo is ok while we work on the data that needs to be exposed to MCP Hosts for successful installation. Maybe once #3 is settled, we can model it in Postgres. We're also in discussions with a couple of platforms about hosting options, so it would be good to decide on Postgres to ensure it's available in the environment we go with.

[sambhav]

I'm not sure if the example was coincidental or contextual lol. I was a maintainer on Musicbrainz in the past 🤣 if it was coincidental, that's a small world.

[toby]

Oh wow, amazing!! That was purely coincidental. I was at Last.fm and we worked with Musicbrainz data. I see it as a heavy inspiration for this project.

[sambhav]

Hehe, it's interesting how the worlds connect. I was responsible for building their real-time relevance search system https://blog.metabrainz.org/2018/06/15/musicbrainz-search-overhaul/ so am intimately familiar with the whole DB setup and dumps.

[toby]

Very cool, glad to have your expertise involved!

[indranilr]

+1 for this feature, this would at least help my organization to onboard the implementation.

[kidehen]

Why is the suggestion to move from one DBMS to another. How about a DBMS-independent approach?