Braindler Assistant Metrics 📊

Real-time performance and usage monitoring system for Braindler Assistants using Prometheus and Grafana.

Overview

Braindler Assistant Metrics provides comprehensive observability for your AI assistant deployments. The system collects, stores, visualizes, and alerts on critical metrics from your Braindler Assistants across all environments.

Features

• Real-time Monitoring: Live dashboards showing assistant performance metrics
• Multi-dimensional Data: Filter metrics by assistant ID, tenant, language, platform, and more
• Alerting: Proactive notifications when metrics exceed thresholds
• Historical Analysis: Track performance trends over time
• Custom Dashboards: Create specialized views for different stakeholders

Metrics Collected

Conversation Metrics

Metric	Type	Description
braindler_conversations_started_total	Counter	Total number of conversations initiated
braindler_conversations_completed_total	Counter	Total number of successfully completed conversations
braindler_conversations_abandoned_total	Counter	Total number of abandoned conversations
braindler_conversation_duration_seconds	Histogram	Duration of conversations from start to end
braindler_conversation_steps_count	Histogram	Number of interaction steps in completed conversations

Response Metrics

Metric	Type	Description
braindler_response_time_seconds	Histogram	Time taken to generate assistant responses
braindler_llm_token_usage_total	Counter	Total tokens consumed by LLM models
braindler_llm_cost_total	Counter	Cost of LLM API calls in USD
braindler_response_failure_total	Counter	Failed response generations by error type

User Interaction Metrics

Metric	Type	Description
braindler_user_satisfaction_score	Gauge	User satisfaction ratings (1-5)
braindler_conversation_goals_achieved_total	Counter	Number of conversation goals successfully achieved
braindler_user_inputs_per_conversation	Histogram	Distribution of number of user inputs per conversation
braindler_user_wait_time_seconds	Histogram	Time users spend waiting for responses

System Performance Metrics

Metric	Type	Description
braindler_system_memory_bytes	Gauge	Memory usage of assistant instances
braindler_system_cpu_usage_percent	Gauge	CPU utilization by assistant instances
braindler_system_errors_total	Counter	Count of system errors by type
braindler_script_execution_time_seconds	Histogram	Time taken to process conversation scripts

Architecture

graph TD
    A[Braindler Assistants] -->|Export metrics| B[Prometheus Exporters]
    B -->|Scrape metrics| C[Prometheus Server]
    C -->|Query metrics| D[Grafana]
    D -->|Display| E[Dashboards]
    D -->|Notify| F[Alerts]
    
    G[Platform Services] -->|Export metrics| B
    H[External APIs] -->|Export metrics| B

Loading

Getting Started

Prerequisites

• Docker and Docker Compose
• Kubernetes (for production deployments)
• Access to your Braindler Assistant instances

Quick Start

1. Clone this repository:

   git clone https://github.com/braindler/braindler-assistant-metrics.git
   cd braindler-assistant-metrics

2. Configure your environment:

   cp .env.example .env
   # Edit .env with your specific configuration

3. Start the monitoring stack:

   docker-compose up -d

4. Access Grafana at http://localhost:3000 (default credentials: admin/admin)

Instrumenting Your Assistants

Add the Prometheus client library to your assistant code:

const { Counter, Histogram } = require('prom-client');

// Initialize metrics
const conversationsStarted = new Counter({
  name: 'braindler_conversations_started_total',
  help: 'Total number of conversations started',
  labelNames: ['assistant_id', 'platform']
});

// Track conversation start
function startConversation(assistantId, platform) {
  conversationsStarted.inc({ assistant_id: assistantId, platform });
  // Your conversation logic
}

Dashboards

Pre-configured Grafana dashboards are available in the dashboards/ directory:

• Overview Dashboard: High-level metrics across all assistants
• Conversation Analytics: Detailed conversation metrics and patterns
• Performance Monitoring: System performance and response times
• Cost Analysis: Token usage and associated costs
• User Experience: Satisfaction scores and interaction patterns

Alert Rules

Example alert rules can be found in prometheus/alerts.yml:

• High response latency
• Elevated error rates
• Unusual conversation abandonment rates
• Excessive token consumption
• Low user satisfaction scores

Deployment Options

Docker Compose (Development)

Ideal for local development or small deployments:

docker-compose up -d

Kubernetes (Production)

For production environments, use the provided Helm charts:

helm install braindler-metrics ./helm/braindler-metrics -n monitoring

Cloud Managed Services

Instructions for managed solutions available in docs/cloud-deployment.md.

Integration with Braindler Ecosystem

This metrics system integrates with:

• Braindler Assistant Core

• Braindler Chat Script Editor

• Braindler Management Console

• GitHub: Braindler

---

📊 Braindler Assistant Metrics - Insight-driven AI assistants.