Skip to content

08. Architecture Overview

Jump to bottom Edit New page
ad1107 edited this page May 24, 2025 · 1 revision

Architecture Overview

Comprehensive system architecture documentation for UtilsBot+, covering design principles, component interactions, and data flow.

📋 Table of Contents

System Architecture

High-Level Architecture Diagram

┌─────────────────────────────────────────────────────────────────┐
│                         External Services                      │
├─────────────────┬─────────────────┬─────────────────────────────┤
│   Discord API   │   Google AI     │     External APIs           │
│   (discord.py)  │   (Gemini)      │  (Screenshot, IP, etc.)     │
└─────────┬───────┴─────────┬───────┴─────────┬───────────────────┘
          │                 │                 │
          │    ┌────────────┼─────────────────┼──────────────┐
          │    │            │                 │              │
          ▼    ▼            ▼                 ▼              │
    ┌─────────────────────────────────────────────────────────────┐
    │                    Bot Core Layer                          │
    │                  (core/bot.py)                             │
    │  ┌─────────────┐ ┌─────────────┐ ┌─────────────────────┐  │
    │  │   Event     │ │  Command    │ │    Error Handler    │  │
    │  │  Handler    │ │  Router     │ │   & Middleware      │  │
    │  └─────────────┘ └─────────────┘ └─────────────────────┘  │
    └──────────────────────┬──────────────────────────────────────┘
                           │
        ┌──────────────────┼──────────────────┐
        │                  │                  │
        ▼                  ▼                  ▼
┌───────────────┐  ┌───────────────┐  ┌───────────────┐
│    Cogs       │  │   Database    │  │   Utilities   │
│  (Features)   │  │ (SQLAlchemy)  │  │  (Helpers)    │
│               │  │               │  │               │
│ • AI          │  │ • Users       │  │ • Embeds      │
│ • Games       │  │ • Guilds      │  │ • Checks      │
│ • Network     │  │ • Stats       │  │ • Health      │
│ • Tools       │  │ • API Usage   │  │ • Screenshot  │
│ • System      │  │ • Logs        │  │ • Cache       │
└───────────────┘  └───────────────┘  └───────────────┘
        │                  │                  │
        └──────────────────┼──────────────────┘
                           │
                    ┌──────▼──────┐
                    │  Monitoring │
                    │ & Logging   │
                    │             │
                    │ • Structlog │
                    │ • Sentry    │
                    │ • Metrics   │
                    └─────────────┘

Component Interaction Flow

User Input (Slash Command)
         │
         ▼
┌─────────────────┐
│ Discord Gateway │ ← WebSocket connection
└─────────┬───────┘
          │
          ▼
┌─────────────────┐
│  Bot Core       │ ← Event processing
│  Event Handler  │
└─────────┬───────┘
          │
          ▼
┌─────────────────┐
│ Command Router  │ ← Slash command routing
│ & Middleware    │
└─────────┬───────┘
          │
          ▼
┌─────────────────┐
│ Permission      │ ← Security checks
│ Checks          │
└─────────┬───────┘
          │
          ▼
┌─────────────────┐
│ Cog Handler     │ ← Feature execution
│ (Specific Cog)  │
└─────────┬───────┘
          │
    ┌─────┼─────┐
    │     │     │
    ▼     ▼     ▼
┌─────┐ ┌─────┐ ┌─────┐
│ DB  │ │ API │ │Utils│ ← Resource access
└─────┘ └─────┘ └─────┘
    │     │     │
    └─────┼─────┘
          │
          ▼
┌─────────────────┐
│ Response        │ ← User feedback
│ Generation      │
└─────────────────┘

Design Principles

1. Slash Commands Only Architecture

Why Slash Commands:

  • Better UX: Built-in help, autocomplete, and parameter validation
  • Performance: No need to scan every message for prefixes
  • Security: Better permission handling and user identification
  • Future-Proof: Discord's preferred command system

Implementation:

# No prefix commands, only slash commands
command_prefix="$disabled$"  # Ensures no prefix command processing

# All commands use app_commands decorator
@app_commands.command(name="example", description="Example command")
async def example_command(self, interaction: discord.Interaction):
    pass

2. Async-First Design

Async Patterns:

  • Non-blocking I/O: All database and API calls are async
  • Concurrent Operations: Multiple requests handled simultaneously
  • Resource Efficiency: Single thread handles thousands of operations

Example Implementation:

# Async database operations
async with self.db.async_session() as session:
    user = await session.get(User, user_id)

# Async HTTP requests
async with aiohttp.ClientSession() as session:
    async with session.get(url) as response:
        data = await response.json()

3. Modular Cog Architecture

Benefits:

  • Separation of Concerns: Each cog handles specific functionality
  • Hot Reloading: Cogs can be reloaded without bot restart
  • Easy Testing: Individual cogs can be tested in isolation
  • Maintainability: Clear code organization and boundaries

Cog Structure:

class ExampleCog(commands.Cog):
    def __init__(self, bot):
        self.bot = bot
        self.logger = structlog.get_logger(__name__)
    
    @app_commands.command()
    async def example(self, interaction: discord.Interaction):
        # Command implementation
        pass

4. Error-First Design

Comprehensive Error Handling:

  • Graceful Degradation: Features fail safely without crashing
  • User-Friendly Messages: Clear error communication
  • Logging: Detailed error tracking for debugging
  • Recovery: Automatic retry and fallback mechanisms

5. Security by Design

Multi-Layer Security:

  • Input Validation: All user inputs sanitized and validated
  • Permission Checks: Role-based access control
  • Rate Limiting: Prevents abuse and API quota exhaustion
  • Audit Logging: Complete trail of user actions

Component Overview

Bot Core Layer

core/bot.py - Main bot class and lifecycle management

Responsibilities:

  • Discord connection management
  • Event handling and routing
  • Cog loading and management
  • Global error handling
  • Database initialization
  • Health monitoring

Key Features:

class UtilsBot(commands.Bot):
    def __init__(self):
        # Configure intents
        intents = discord.Intents.default()
        intents.message_content = False  # Not needed for slash commands
        
        # Initialize bot
        super().__init__(
            command_prefix="$disabled$",
            intents=intents,
            help_command=None
        )

Cog System

Modular Feature Implementation:

Cog Purpose Key Commands
AI Google Gemini integration /ask, /chat
Games Interactive games /wordle
Network Web utilities /screenshot, /ip, /unshorten
Tools Security & utility tools /totp, /qr, /hash
Info Bot information & help /info, /help, /ping
System Developer commands /sync, /eval, /reload

Cog Lifecycle:

# Automatic cog loading
async def load_cogs(self):
    for filename in os.listdir('./cogs'):
        if filename.endswith('.py') and not filename.startswith('_'):
            await self.load_extension(f'cogs.{filename[:-3]}')

Database Layer

SQLAlchemy 2.0 with Async Support:

Architecture:

# Async engine setup
engine = create_async_engine(DATABASE_URL, echo=False)

# Session factory
AsyncSessionLocal = async_sessionmaker(
    engine, 
    class_=AsyncSession,
    expire_on_commit=False
)

# Context manager for sessions
async with AsyncSessionLocal() as session:
    # Database operations
    pass

Models:

  • User: Discord user data and preferences
  • Guild: Server settings and configuration
  • GameStats: User game performance tracking
  • APIUsage: External API usage monitoring
  • CommandLogs: Audit trail for all commands

Utility Layer

Helper Functions and Services:

utils/embeds.py - Discord embed generation

def create_success_embed(title: str, description: str) -> discord.Embed:
    return discord.Embed(
        title=f"✅ {title}",
        description=description,
        color=0x00ff00
    )

utils/checks.py - Permission decorators

def requires_whitelist():
    async def predicate(interaction: discord.Interaction) -> bool:
        return await check_user_whitelist(interaction.user.id)
    return app_commands.check(predicate)

utils/screenshot.py - Screenshot service abstraction utils/health.py - Health monitoring utilities

Data Flow

Command Execution Flow

  1. User Input: User types slash command in Discord
  2. Discord Gateway: Command sent via WebSocket to bot
  3. Event Processing: on_interaction event triggered
  4. Command Routing: Command mapped to appropriate cog method
  5. Permission Checks: Security decorators validate access
  6. Business Logic: Command-specific functionality executed
  7. Database Operations: Data read/written as needed
  8. External APIs: Third-party services called if required
  9. Response Generation: Discord embed or message created
  10. User Feedback: Response sent back to Discord user

Database Transaction Flow

# Typical database operation pattern
async def update_user_stats(user_id: str, command: str):
    async with self.db.async_session() as session:
        try:
            # Get or create user
            user = await get_or_create_user(session, user_id)
            
            # Update statistics
            user.total_commands_used += 1
            user.last_seen_at = datetime.utcnow()
            
            # Log command execution
            log_entry = CommandLog(
                user_discord_id=user_id,
                command_name=command,
                executed_at=datetime.utcnow()
            )
            session.add(log_entry)
            
            # Commit transaction
            await session.commit()
            
        except Exception as e:
            await session.rollback()
            raise e

API Integration Flow

# External API call pattern
async def call_external_api(endpoint: str, params: dict):
    async with aiohttp.ClientSession() as session:
        try:
            async with session.get(endpoint, params=params) as response:
                if response.status == 200:
                    return await response.json()
                else:
                    raise APIError(f"API returned {response.status}")
        except asyncio.TimeoutError:
            raise APIError("Request timed out")
        except Exception as e:
            raise APIError(f"API call failed: {e}")

Technology Stack

Core Technologies

Component Technology Version Purpose
Runtime Python 3.11+ Main programming language
Discord discord.py 2.4+ Discord API wrapper
Database SQLAlchemy 2.0+ ORM and database toolkit
HTTP aiohttp/httpx Latest Async HTTP client
AI google-generativeai 0.8+ Google Gemini integration
Logging structlog Latest Structured logging
Config pydantic 2.8+ Configuration validation

External Services

Service Purpose Tier Limits
Google Gemini AI responses Free 15 req/min
ip-api.com IP geolocation Free 1000 req/month
ScreenshotOne Website screenshots Paid Varies
Sentry Error tracking Free/Paid Event limits

Development Tools

Tool Purpose
Black Code formatting
isort Import sorting
mypy Type checking
pytest Testing framework
pre-commit Git hooks

Performance Considerations

Async Optimization

Connection Pooling:

# Database connection pooling
engine = create_async_engine(
    DATABASE_URL,
    pool_size=20,
    max_overflow=0,
    pool_pre_ping=True
)

# HTTP session reuse
self.http_session = aiohttp.ClientSession()

Concurrent Operations:

# Process multiple operations concurrently
tasks = [process_user(user) for user in users]
results = await asyncio.gather(*tasks, return_exceptions=True)

Caching Strategy

In-Memory Caching:

from functools import lru_cache

@lru_cache(maxsize=1000)
def get_user_permissions(user_id: str) -> dict:
    # Cached permission lookup
    pass

Database Query Optimization:

# Eager loading to prevent N+1 queries
query = select(User).options(selectinload(User.game_stats))
users = await session.execute(query)

Resource Management

Memory Usage:

  • Response caching for repeated queries
  • Automatic cleanup of expired data
  • Connection pool size limits

CPU Usage:

  • Async I/O for non-blocking operations
  • Background tasks for maintenance
  • Rate limiting to prevent overload

Scalability & Extensibility

Horizontal Scaling

Multi-Instance Deployment:

  • Stateless bot design allows multiple instances
  • Database handles concurrent connections
  • Redis for shared state (if needed)

Load Balancing:

# Docker Compose scaling
services:
  bot:
    scale: 3  # Run 3 bot instances

Vertical Scaling

Resource Optimization:

  • Database indexing for query performance
  • Connection pooling for efficiency
  • Memory usage optimization

Adding New Features

New Command Addition:

  1. Create command in appropriate cog
  2. Add database models if needed
  3. Update permissions/security checks
  4. Add tests and documentation
  5. Deploy with hot reload

New Cog Creation:

# Template for new cog
class NewFeatureCog(commands.Cog):
    def __init__(self, bot):
        self.bot = bot
        self.logger = structlog.get_logger(__name__)
    
    @app_commands.command()
    async def new_command(self, interaction: discord.Interaction):
        # Implementation
        pass

async def setup(bot):
    await bot.add_cog(NewFeatureCog(bot))

Integration Points

External Service Integration:

  • Add new API clients to utils/
  • Create configuration options
  • Implement error handling and fallbacks
  • Add usage tracking

Database Extensions:

  • Create new models in models/database.py
  • Write migration scripts
  • Update repository methods
  • Add indexes for performance

🔗 Related Pages

📝 What's Next?

Add a custom footer
Add a custom sidebar
Clone this wiki locally