Architecture Proposal: Secure Enterprise Multi-Agent System (MAS) Platform Design

[sriaradhyula]

Overview

This proposal outlines a modular, secure, and framework-agnostic architecture for building AI-native multi-agent systems in enterprise environments. It serves as a community-driven foundation for designing interoperable, scalable, and observable platforms.

Goals

• Standardize secure communication between heterogeneous agents
• Enable interoperability across LangGraph, LlamaIndex, Strands, etc.
• Integrate LLMs for autonomous and collaborative agent behaviors
• Adopt enterprise-grade identity, access control, and token propagation
• Provide built-in hooks for agent observability and evaluation

Architecture Foundation

Core Principles

Principle	Description
Security First	JWT-based auth via Keycloak across A2A and MCP
Extensibility	Framework agnostic and pluggable
Latency-aware	Messaging bus for fast intra-agent communications like AGNTCY SLIM
Unified Protocols	A2A for agent comms, MCP for backend APIs
Enterprise Ready	Support for OAuth based AuthN/AuthZ for RBAC using OSS KeyCloak reference IDP
Observability Native	Use OSS tools like OpenLLMetry, LangFuse, AGNTCY Observe. Compatible with LangSmith

Proposed Layered Architecture

1. Authentication & Identity Layer

• Keycloak as the OSS Identity Provider (IDP)
• End User Identity and Agent Identity to generate OAuth JWT Token
• JWT Token validation and scope/claims inspection at every

2. Agent Interface Layer

• Supervisor Agent receives user input via:
  • Backstage plugins
  • Chat interfaces
  • CLI clients
• Routes requests via A2A (Agent-to-Agent protocol)

3. Sub-Agent Execution Layer

• Agents can be implemented in any framework like LangGraph, LlamaIndex, Autogen, CrewAI, Strands, etc.
• Each Agent exposes A2A interfaces and uses bearer token to authenticate the request.
• Note: Request authorization TBD

4. Messaging Layer

• Resilient async message delivery (low latency)

• Retry, circuit breaking, and timeout strategies

• Example: - AGNTCY SLIM Pub/Sub SLIM Docs

5. MCP Gateway Layer

• Unified reverse proxy for MCP protocol
• Centralized auth translation and token forwarding
• MCP Gateway Registry

6. LLM Access Layer

• Unified LLM provider interface via LLM proxy (e.g., LiteLLM)
• Framework-friendly LLM abstraction for reasoning, synthesis, and reflection
• Setup Budget restrictions per agent
• Use LLMs that can be hosted local to the environment like using Olama
• Quickly switch LLMs

7. Observability

• Add support for:
  • LangSmith traces
  • LangFuse logs
  • OpenLLMetry metrics
  • Arize.ai/Phoenix - https://github.com/Arize-ai/phoenix
  • AGNTCY Observe pipelines

8. Evaluations

• Offline and Online MAS evaluations
• Evaluation rubrics

What We’re Asking the Community

We’d love your feedback and contributions on:

• OAuth2 / token validation and authorization scopes
• Choice of AGNTCY Slim and MCP Gateway. Can they be the same component? Should we explore other options
• Distribution packaging for local/k8s deployment - helm, test bench
• Define a standard setup of rubrics to evaluate agents

Reference: https://drive.google.com/file/d/1jCDKLNBgWyzRmiU3w9KUz1VJjKBj9GDH/view?usp=sharing

[madhurprash]

This is good, thanks for creating this. Is there a reference link here you could add for A2A? We can re use what we demo'd at the kick off with A2A, but I think there are good components here...

[sriaradhyula]

Reference implementation - https://github.com/cnoe-io/ai-platform-engineering/tree/main