Skip to content

A Kubernetes service that monitors workloads for specific annotations and sends filtered logs to Slack channels based on configurations

Notifications You must be signed in to change notification settings

ElementTech/slaking

Folders and files

NameName
Last commit message
Last commit date

Latest commit

ย 

History

24 Commits
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 

Repository files navigation

๐Ÿฆฅ Slaking

"The laziest Pokรฉmon that monitors your Kubernetes workloads while you take a nap!"

Slaking Pokรฉmon

    โ•ญโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ•ฎ
    โ”‚                                                             โ”‚
    โ”‚  ๐Ÿฆฅ Slaking - Kubernetes Log Monitoring Service            โ”‚
    โ”‚                                                             โ”‚
    โ”‚  "When your logs need attention, Slaking wakes up!"        โ”‚
    โ”‚                                                             โ”‚
    โ•ฐโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ•ฏ

A Kubernetes service that monitors workloads for specific annotations and sends filtered logs to Slack channels based on configurations - just like how Slaking watches over its territory!

๐ŸŽฏ Features

  • ๐Ÿฆฅ Workload Monitoring: Like Slaking's keen senses, watches Kubernetes workloads (Pods, Deployments, StatefulSets, etc.) for specific annotations
  • ๐Ÿ” Log Filtering: Filters logs based on configurable patterns and criteria - Slaking knows exactly what to look for!
  • ๐Ÿ“ฑ Slack Integration: Sends filtered logs to designated Slack channels - Slaking's way of communicating
  • โš™๏ธ Configuration Management: Supports multiple configurations for different workloads and channels
  • โค๏ธ Health Monitoring: Built-in health checks and metrics - Slaking stays healthy!
  • ๐Ÿ“ฆ Helm Chart: Complete Helm chart for easy deployment and management

๐Ÿ—๏ธ Architecture

โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”    โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”    โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
โ”‚   Kubernetes    โ”‚    โ”‚    ๐Ÿฆฅ Slaking    โ”‚    โ”‚     Slack       โ”‚
โ”‚   Workloads     โ”‚โ”€โ”€โ”€โ–ถโ”‚   Service        โ”‚โ”€โ”€โ”€โ–ถโ”‚   Channels      โ”‚
โ”‚   (with         โ”‚    โ”‚                  โ”‚    โ”‚                 โ”‚
โ”‚   annotations)  โ”‚    โ”‚  - Watcher       โ”‚    โ”‚  - #alerts      โ”‚
โ”‚                 โ”‚    โ”‚  - Filter        โ”‚    โ”‚  - #errors      โ”‚
โ”‚                 โ”‚    โ”‚  - Forwarder     โ”‚    โ”‚  - #debug       โ”‚
โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜    โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜    โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜

Slaking's territory spans from your Kubernetes cluster to your Slack workspace!

๐Ÿš€ Deployment Options

Option 1: Helm Chart (Recommended) ๐Ÿฆฅ

The easiest way to deploy Slaking is using the provided Helm chart from the GitHub Pages repository:

Watch All Namespaces (Recommended)

# Add the Helm repository
helm repo add slaking https://elementtech.github.io/slaking
helm repo update

# Install the chart - Wake up Slaking to watch ALL namespaces!
helm install slaking slaking/slaking \
  --namespace slaking \
  --create-namespace \
  --set env.SLACK_TOKEN="xoxb-your-token" \
  --set env.SLACK_DEFAULT_CHANNEL="#alerts" \
  --set env.K8S_WATCH_ALL_NAMESPACES="true"

Watch Specific Namespaces Only

# Install the chart to watch specific namespaces only
helm install slaking slaking/slaking \
  --namespace slaking \
  --create-namespace \
  --set env.SLACK_TOKEN="xoxb-your-token" \
  --set env.SLACK_DEFAULT_CHANNEL="#alerts" \
  --set env.K8S_WATCH_ALL_NAMESPACES="false" \
  --set env.K8S_NAMESPACES="production,staging,monitoring"

Alternative: Install from local chart directory (if you have the source code):

# Quick deployment with automated script (watches all namespaces by default)
./helm-deploy.sh

# Manual Helm deployment from local chart
helm install slaking ./charts/slaking \
  --namespace slaking \
  --create-namespace \
  --set env.SLACK_TOKEN="xoxb-your-token" \
  --set env.SLACK_DEFAULT_CHANNEL="#alerts" \
  --set env.K8S_WATCH_ALL_NAMESPACES="true"

Benefits of Helm deployment:

  • โœ… Easy configuration management
  • โœ… Environment-specific values files
  • โœ… Automatic RBAC setup
  • โœ… Prometheus ServiceMonitor integration
  • โœ… Horizontal Pod Autoscaler support
  • โœ… Ingress configuration
  • โœ… Comprehensive health checks
  • โœ… Easy upgrades and rollbacks

Option 2: Vanilla Kubernetes Manifests

For users who prefer direct Kubernetes manifests:

# Deploy using vanilla manifests
./deploy.sh

# Or manually
kubectl apply -f k8s/

โš™๏ธ Configuration

Workload Annotations

Add these annotations to your Kubernetes workloads - Slaking will watch for these signals:

metadata:
  annotations:
    slaking.enabled: "true"
    slaking.channel: "#alerts"
    slaking.filters: "error|exception|fatal"
    slaking.level: "error"
    slaking.include-labels: "app=myapp,environment=prod"

Configuration Options

Annotation Description Default
slaking.enabled Enable log forwarding for this workload false
slaking.channel Slack channel to send logs to #general
slaking.filters Regex patterns to filter logs .*
slaking.level Minimum log level (debug, info, warn, error) info
slaking.include-labels Comma-separated key=value pairs to include ""
slaking.exclude-labels Comma-separated key=value pairs to exclude ""
slaking.max-lines Maximum lines per message 10
slaking.cooldown Cooldown period between messages (seconds) 60

Namespace Configuration

Slaking supports two modes for watching Kubernetes namespaces:

๐Ÿฆฅ Watch All Namespaces (Recommended)

This is the default and recommended configuration. Slaking will monitor all namespaces in your cluster and only process logs from workloads that have the slaking.enabled: "true" annotation.

Environment Configuration:

# Set to watch all namespaces
K8S_WATCH_ALL_NAMESPACES=true
# Leave K8S_NAMESPACES empty or unset

Helm Configuration:

helm install slaking slaking/slaking \
  --set env.K8S_WATCH_ALL_NAMESPACES="true"

Benefits:

  • โœ… No need to specify namespaces manually
  • โœ… Automatically picks up new namespaces
  • โœ… Works with any workload regardless of namespace
  • โœ… Simpler configuration and maintenance

๐ŸŽฏ Watch Specific Namespaces Only

For environments where you want to limit Slaking's scope to specific namespaces (e.g., for security or performance reasons).

Environment Configuration:

# Disable watching all namespaces
K8S_WATCH_ALL_NAMESPACES=false
# Specify namespaces to watch
K8S_NAMESPACES=production,staging,monitoring

Helm Configuration:

helm install slaking slaking/slaking \
  --set env.K8S_WATCH_ALL_NAMESPACES="false" \
  --set env.K8S_NAMESPACES="production,staging,monitoring"

Use Cases:

  • ๐Ÿ”’ Multi-tenant clusters where you want to isolate monitoring
  • ๐Ÿš€ Performance optimization for large clusters
  • ๐Ÿ›ก๏ธ Security requirements that limit cross-namespace access

๐ŸŽฎ Installation

Prerequisites

  • Kubernetes cluster (1.19+)
  • kubectl configured and connected to your cluster
  • Docker installed and running
  • Slack workspace with API access
  • Helm 3.0+ (for Helm deployment)

Quick Start

  1. Install from Helm repository (Recommended):

    # Add the Helm repository
    helm repo add slaking https://elementtech.github.io/slaking
    helm repo update
    
    # Install with your Slack configuration - Time to wake up Slaking!
    # This will watch ALL namespaces by default
    helm install slaking slaking/slaking \
      --namespace slaking \
      --create-namespace \
      --set env.SLACK_TOKEN="xoxb-your-token" \
      --set env.SLACK_DEFAULT_CHANNEL="#alerts" \
      --set env.K8S_WATCH_ALL_NAMESPACES="true"
  2. Alternative: Local development setup:

    # Clone and install dependencies
    git clone https://github.com/ElementTech/slaking.git
    cd slaking
    npm install
    
    # Set up environment variables
    cp env.example .env
    # Edit .env with your Slack token and other settings
    # By default, it will watch all namespaces
    
    # Deploy using automated script
    ./helm-deploy.sh
  3. Configure your workloads with annotations and watch logs flow to Slack! ๐Ÿฆฅ

Updating the Helm Chart

# Update the repository
helm repo update

# Check available versions
helm search repo slaking/slaking

# Upgrade to latest version (maintains current namespace configuration)
helm upgrade slaking slaking/slaking \
  --namespace slaking \
  --reuse-values

# Upgrade and change to watch all namespaces
helm upgrade slaking slaking/slaking \
  --namespace slaking \
  --set env.K8S_WATCH_ALL_NAMESPACES="true" \
  --reuse-values

# Upgrade and change to watch specific namespaces only
helm upgrade slaking slaking/slaking \
  --namespace slaking \
  --set env.K8S_WATCH_ALL_NAMESPACES="false" \
  --set env.K8S_NAMESPACES="production,staging" \
  --reuse-values

# Or upgrade to a specific version
helm upgrade slaking slaking/slaking \
  --namespace slaking \
  --version 1.0.1 \
  --reuse-values

๐Ÿ”Œ API Endpoints

  • GET /health - Health check (Slaking's vital signs)
  • GET /metrics - Prometheus metrics (Slaking's stats)
  • POST /config - Update configuration
  • GET /config - Get current configuration
  • GET /status - Service status (includes namespace configuration)

๐Ÿ“Š Monitoring

The service exposes Prometheus metrics at /metrics for monitoring - Slaking's performance stats:

  • slaking_logs_processed_total
  • slaking_logs_filtered_total
  • slaking_slack_messages_sent_total
  • slaking_errors_total

Grafana Dashboard

Create a Grafana dashboard with these queries:

# Log processing rate
rate(slaking_logs_processed_total[5m])

# Error rate
rate(slaking_errors_total[5m])

# Active streams
slaking_active_streams

# Slack message rate
rate(slaking_slack_messages_sent_total[5m])

๐ŸŽฏ Advanced Configuration

Helm Chart Configuration

The Helm chart supports extensive configuration:

# values-production.yaml
replicaCount: 3
hpa:
  enabled: true
  maxReplicas: 5
serviceMonitor:
  enabled: true
config:
  slack:
    defaultChannel: "#prod-alerts"
  kubernetes:
    # Watch all namespaces (recommended)
    watchAllNamespaces: true
    namespaces: []
    # OR watch specific namespaces only
    # watchAllNamespaces: false
    # namespaces: ["production", "staging"]
env:
  SLACK_TOKEN: "xoxb-your-token"
  K8S_WATCH_ALL_NAMESPACES: "true"
  # K8S_NAMESPACES: "production,staging"  # Only needed if watchAllNamespaces: false

Multi-Environment Setup

Create separate values files for different environments:

  • values-production.yaml - Production settings
  • values-staging.yaml - Staging settings
  • values-development.yaml - Development settings

๐Ÿ†˜ Troubleshooting

Common Issues

  1. No logs being sent to Slack

    • Check if annotations are properly set
    • Verify Slack token and channel permissions
    • Check service logs for errors
  2. Too many messages

    • Adjust cooldown period
    • Refine filter patterns
    • Set appropriate log levels
  3. Permission denied

    • Ensure proper RBAC configuration
    • Check service account permissions
  4. Helm repository not found (404 error)

    • Ensure GitHub Pages is enabled for the gh-pages branch
    • Check that the chart-releaser action has run successfully
    • Verify the repository URL is correct: https://elementtech.github.io/slaking
    • Wait a few minutes after pushing changes for GitHub Pages to update

Debug Mode

Repository-based Helm deployment:

helm upgrade slaking slaking/slaking \
  --namespace slaking \
  --set env.LOG_LEVEL=debug \
  --reuse-values

Local Helm deployment:

helm upgrade slaking ./charts/slaking \
  --namespace slaking \
  --set env.LOG_LEVEL=debug

Testing Configuration

# Test health endpoint
kubectl port-forward -n slaking svc/slaking 3000:3000
curl http://localhost:3000/health

# Test metrics endpoint
kubectl port-forward -n slaking svc/slaking-metrics 9090:9090
curl http://localhost:9090/metrics

๐ŸŽฎ Management Commands

Helm Deployment (Repository-based)

# Check status
helm list -n slaking
kubectl get pods -n slaking

# View logs
kubectl logs -n slaking -l app=slaking

# Test deployment
kubectl port-forward -n slaking svc/slaking 3000:3000
curl http://localhost:3000/health

# Upgrade deployment
helm upgrade slaking slaking/slaking \
  --namespace slaking \
  --reuse-values

# Uninstall
helm uninstall slaking -n slaking
kubectl delete namespace slaking

Helm Deployment (Local chart)

# Check status
./helm-deploy.sh status

# View logs
./helm-deploy.sh logs

# Test deployment
./helm-deploy.sh test

# Upgrade deployment
./helm-deploy.sh upgrade

# Uninstall
./helm-deploy.sh uninstall

๐Ÿ”’ Security Considerations

RBAC Permissions

The service requires minimal permissions:

  • Read access to pods and their logs
  • Watch access to deployments, statefulsets, daemonsets
  • Read access to namespaces

Network Security

  • Service runs on ClusterIP by default
  • Metrics endpoint is separate for Prometheus scraping
  • No external access required

Secret Management

  • Slack token is stored in Kubernetes Secret
  • Consider using external secret management (HashiCorp Vault, AWS Secrets Manager, etc.)

    ๐Ÿฆฅ Slaking says: "Thanks for choosing me as your Kubernetes monitor!"
    
    "When your logs need attention, I'll be there to help!"

๐Ÿ“„ License

MIT


Made with โค๏ธ by the Slaking team - Because even the laziest Pokรฉmon can be the most reliable! ๐Ÿฆฅ

About

A Kubernetes service that monitors workloads for specific annotations and sends filtered logs to Slack channels based on configurations

Topics

Resources

Stars

Watchers

Forks

Packages