Test Analytics Dashboard

A modern web application for visualizing and analyzing test execution results across your testing suite. This dashboard provides insights into test performance, stability, and distribution across different testing layers.

Table of Contents

1. Quick Start
2. Features
3. Tech Stack
4. Project Structure
5. Development Setup
6. Testing
7. Feature Setup Guides
8. Feature Ownership Mapping
9. AWS S3 Integration
10. Deployment
11. Secrets Management
12. Monitoring and Maintenance
13. Feature Flags
14. API Response Examples
15. Troubleshooting
16. Contributors
17. Development Tools
18. License

Quick Start

Run in Development Mode (Local Environment)

# Install all dependencies and set up API compatibility layer
npm run install:all

# Start the development server with hot reloading
npm run dev

Run with Docker (Containerized Environment)

# Build and start containers
docker-compose up --build

Features

• 📈 Pass rate trends and stability analysis - Track test success rates over time
• ⏱️ Test duration distribution - Identify slow tests and performance bottlenecks
• 🔍 Detailed test suite breakdown - Drill down into individual test suites and cases
• 🏗️ Testing pyramid visualization - Monitor test distribution across layers
• 🎯 Flaky test detection - Identify and track unstable tests
• 🔧 Feature ownership mapping - Link tests to teams and feature codes
• 📊 Multi-layer test analytics - E2E, Integration, and Backend test analysis
• 🚀 Real-time data from S3 - Direct integration with AWS S3 for test reports
• 🎨 Modern responsive UI - Built with React, TypeScript, and TailwindCSS
• ⚙️ Feature flags system - Enable/disable features dynamically

Tech Stack

Frontend

• React 18 - A JavaScript library for building user interfaces
• TypeScript - For type-safe code and better developer experience
• TailwindCSS - For utility-first styling and rapid UI development
• Chart.js & react-chartjs-2 - For interactive and responsive data visualization
• Lucide React - For modern, customizable icons
• React Query (TanStack Query) - For efficient data fetching and caching
• React Router - For client-side routing
• Jest & Testing Library - For unit and integration testing

Backend

• Node.js & Express - For building a scalable REST API
• TypeScript - For type safety and improved maintainability
• AWS SDK v3 - For S3 integration and data fetching for the reports
• Jest - For backend unit testing with mocking

Infrastructure

• Docker & Docker Compose - For containerized development and deployment
• AWS S3 - For storing and retrieving test reports
• AWS IAM - For secure credential management
• Kubernetes - For production deployment (optional)

Project Structure

.
├── client/                 # Frontend React application
│   ├── public/            # Static files
│   └── src/
│       ├── components/    # React components
│       │   ├── bug-tracker/      # Bug tracking components
│       │   ├── common/           # Shared UI components
│       │   ├── test-analytics/   # Test analytics components
│       │   └── test-pyramid/     # Test pyramid visualization
│       ├── hooks/         # Custom React hooks
│       ├── pages/         # Page components
│       ├── types/         # TypeScript type definitions
│       ├── utils/         # Utility functions
│       └── __tests__/     # Frontend tests
├── api/                   # OpenAPI specification and generated client
│   ├── src/
│   │   └── generated/     # Auto-generated TypeScript client
│   └── openapi.yaml      # API specification
├── server/               # Backend Node.js application
│   └── src/
│       ├── controllers/  # Request handlers
│       ├── services/     # Business logic
│       ├── routes/       # API routes
│       ├── middleware/   # Express middleware
│       ├── utils/        # Utility functions
│       └── __tests__/    # Backend tests
├── deploy/               # Deployment configurations
│   ├── deployment.yaml   # Kubernetes deployment
│   ├── iam.tf           # AWS IAM configuration
│   └── kustomization.yaml # Kustomize configuration
└── docker-compose.yml    # Docker development setup

Development Setup

This section provides instructions for setting up the project for local development.

1. Clone the repository:

git clone https://github.com/studocu/test-analytics-dashboard.git
cd test-analytics-dashboard

2. Install Dependencies and Setup (Single Command)

Use a single command to install all dependencies and set up the project:

# Install all dependencies (root, client, server, and API) and set up compatibility layer
npm run install:all

This command:

• Installs dependencies for the root project, client, server, and API
• Generates API types from the OpenAPI specification
• Sets up the compatibility layer for TypeScript types
• Creates necessary folders and files for development

3. API Types Compatibility Layer

The project uses a compatibility layer to ensure TypeScript types generated from the OpenAPI specification are available to both the client and server. This is automatically set up when you run npm run install:all.

If you make changes to the OpenAPI specification, you can regenerate the API types:

npm run setup-api

4. Environment Setup

Environment files are used to configure various aspects of the application, including API URLs for different environments.

Create Server Environment File (server/.env):

# Create server environment file
cat > server/.env << EOF
PORT=3002
NODE_ENV=development
AWS_REGION=eu-central-1
AWS_PLAYWRIGHT_BUCKET_NAME=playwright.reports
AWS_UNIT_TESTS_BUCKET_NAME=unit-tests.reports
AWS_ACCESS_KEY_ID=your-access-key-id
AWS_SECRET_ACCESS_KEY=your-secret-access-key
AWS_SESSION_TOKEN=your-session-token
SHORTCUT_API_TOKEN=your-token
SHORTCUT_API_URL=your-api-url
SONAR_URL=your-sonar-url
SONAR_TOKEN=your-sonar-token
GOOGLE_SPREADSHEET_ID=your-spreadsheet-id
EOF

Create Client Environment Files: The client uses different environment files for development and production:

# Create all environment files with one command
cat > client/.env.development << EOF
REACT_APP_API_URL=http://localhost:3002/api
EOF
cat > client/.env.production << EOF
REACT_APP_API_URL=http://server:3002/api
EOF
cat > client/.env << EOF
REACT_APP_API_URL=http://localhost:3002/api
EOF

Environment Files Purpose:

• client/.env.development - Used in local development (npm start)
• client/.env.production - Used in production build and Docker
• client/.env - Fallback environment file

AWS Credentials Setup: You can use the provided script to update AWS credentials from your AWS CLI configuration:

# Make the script executable
chmod +x server/update_env.sh

# Run the script to update AWS credentials
cd server && ./update_env.sh

5. Running the Application

For Development (Local Environment)

# Run both client and server with hot reloading
npm run dev

This single command:

• Starts the React client on http://localhost:3000
• Starts the Node.js server on http://localhost:3002
• Enables hot reloading for both components
• Uses the development environment settings

For Docker (Containerized Environment)

# Build and start Docker containers
docker-compose up --build

This single command:

• Builds the client and server Docker images
• Sets up the API compatibility layer
• Starts containers with the proper networking
• Makes the application available at the same ports

6. Accessing the Application

• Client (frontend): http://localhost:3000
• Server API: http://localhost:3002/api

Testing

Running All Tests

# Run all backend tests (from root directory)
npm test

# Run all frontend tests
cd client
npm test -- --watchAll=false

Running Specific Tests

# Run a specific test file (from root directory)
npm test "server/src/__tests__/services/bugService.test.ts"

# Run multiple test files (from root directory)
npm test "server/src/__tests__/controllers/bugController.test.ts" "server/src/__tests__/services/sonarQubeService.test.ts"

# Run tests with coverage report (from root directory)
npm test -- --coverage

# Run tests in watch mode (from root directory)
npm run test:server:watch

Test Structure

Backend Tests

• Controllers Tests: Tests for API controllers that handle HTTP requests

  • Example: server/src/__tests__/controllers/bugController.test.ts

• Services Tests: Tests for service modules that contain business logic

  • Examples:
    • server/src/__tests__/services/bugService.test.ts
    • server/src/__tests__/services/integrationTestService.test.ts
    • server/src/__tests__/services/unitTestService.test.ts
    • server/src/__tests__/services/sonarQubeService.test.ts

• Utils Tests: Tests for utility functions

  • Example: server/src/__tests__/utils/logger.test.ts

Frontend Tests

client/src/__tests__/
├── components/       # Tests for UI components
│   ├── bug-tracker/  # Bug tracking component tests
│   ├── test-analytics/ # Test analytics component tests
│   └── common/       # Shared component tests
├── hooks/           # Tests for custom React hooks
├── pages/           # Tests for page components
└── utils/           # Tests for utility functions

Feature Setup Guides

The application includes several features that require specific setup and configuration. Each feature has its own detailed setup guide:

Feature Ownership Mapping

Links test cases to teams and feature codes using Google Sheets integration.

AWS S3 Integration

Configure AWS S3 buckets for storing and retrieving test reports.

Bug Tracking Integration

Set up Shortcut integration for bug tracking and issue management.

SonarQube Integration

Configure SonarQube for code quality metrics and test coverage analysis.

Feature Flags

Configure the feature flags system for dynamic feature control.

Feature Ownership Mapping

The application includes a feature ownership mapping system that links test cases to teams and feature codes. This enables better test organization, team accountability, and insights into test coverage by team.

What is Feature Ownership?

Feature ownership mapping allows you to:

• Link tests to teams - Associate test cases with the teams responsible for the features
• Track feature codes - Map tests to specific feature codes (e.g., @FTR-123, @BUG-456)
• Analyze team metrics - Get insights into test performance and coverage by team
• Improve accountability - Identify which teams own which tests and features

How it Works

The system uses Google Sheets as the data source for feature ownership mappings. It automatically:

• Loads feature-team mappings from a Google Spreadsheet
• Caches the data in memory for fast access (30-minute cache)
• Refreshes data automatically when cache expires

Setup Requirements

To enable feature ownership mapping, you need:

1. Google Sheets Setup

   • A Google Spreadsheet with feature codes and team assignments
   • Proper sharing permissions for the service account
   • Correct spreadsheet ID and range configuration

2. Google Service Account

   • A Google Cloud service account with Sheets API access
   • Service account credentials JSON file
   • Proper API permissions

3. Environment Variables

   • GOOGLE_SPREADSHEET_ID - Your Google Spreadsheet ID
   • GOOGLE_SPREADSHEET_RANGE - Sheet range (default: Features(FTR)!A:E)
   • GOOGLE_APPLICATION_CREDENTIALS - Path to service account credentials

For detailed setup instructions, see the Feature Ownership Setup Guide.

AWS S3 Integration

AWS S3 Helper

The application includes a centralized AwsS3Helper utility that provides consistent AWS S3 client creation and command execution across all services:

// Example usage in services
import { AwsS3Helper } from '../utils/awsS3Helper';

// Create S3 client with proper credential handling
this.s3Client = AwsS3Helper.createS3Client({
  bucketName: process.env.AWS_BUCKET_NAME!,
  region: process.env.AWS_REGION
});

// Execute S3 commands with error handling
const response = await AwsS3Helper.executeS3CommandWithRegionHandling(
  this.s3Client,
  new ListObjectsV2Command({ Bucket: this.bucketName })
);

IAM Role Support

The application supports both IAM roles (for production) and explicit credentials (for development):

• Production: Uses IAM roles with proper cross-region support
• Development: Uses explicit AWS credentials from environment variables
• Automatic Detection: Automatically detects the appropriate credential method

S3 Buckets

The application integrates with multiple S3 buckets:

• Playwright Reports: E2E test reports
• Unit Tests: Unit test reports
• Integration Tests: Integration test reports

Deployment

The application is deployed on AWS EKS (Elastic Kubernetes Service) using Terraform for infrastructure management and Kubernetes for container orchestration.

For detailed deployment documentation, infrastructure setup, and troubleshooting guides, see the Deployment Guide in the deploy/ directory.

Secrets Management

The application requires several secrets to be configured in GitHub Actions for deployment.

Required Secrets

• AWS_ACCESS_KEY_ID - AWS access key for S3 access (e.g., AKIAIOSFODNN7EXAMPLE)
• AWS_SECRET_ACCESS_KEY - AWS secret key for S3 access (e.g., wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY)
• AWS_SESSION_TOKEN - AWS session token (if using temporary credentials) (e.g., FQoGZXIvYXdzE...)
• AWS_REGION - AWS region for S3 buckets (e.g., eu-central-1, us-east-1)
• AWS_USE_IAM_ROLE - Whether to use IAM role for AWS authentication (e.g., true, false)
• AWS_PLAYWRIGHT_BUCKET_NAME - S3 bucket name for Playwright test reports (e.g., playwright.reports, my-company-test-reports)
• AWS_UNIT_TESTS_BUCKET_NAME - S3 bucket name for unit test reports (e.g., unit-tests.reports, my-company-unit-tests)
• AWS_PHP_REPORT_BUCKET_NAME - S3 bucket name for PHP XML reports (e.g., php-coverage.reports, my-company-php-reports)
• SHORTCUT_API_TOKEN - Shortcut API token for bug tracking (e.g., sc_pt_1234567890abcdef...)
• SHORTCUT_API_URL - Shortcut API base URL (e.g., https://api.app.shortcut.com)
• SONAR_URL - SonarQube server URL (e.g., https://sonarqube.mycompany.com, https://sonarcloud.io)
• SONAR_TOKEN - SonarQube authentication token (e.g., squ_1234567890abcdef...)
• SONAR_PROJECT_KEY - SonarQube project key (e.g., mycompany:myproject, mycompany_myproject)
• GOOGLE_SPREADSHEET_ID - Google Sheets ID for feature ownership data (e.g., 1BxiMVs0XRA5nFMdKdjghfjghdfjgfdhkgdfhgkdfjgh)
• GOOGLE_SPREADSHEET_RANGE - Google Sheets range for feature data (e.g., Features(FTR)!A:E, Sheet1!A1:D100)
• GOOGLE_APPLICATION_CREDENTIALS - Path to Google service account credentials JSON file (e.g., /path/to/service-account-key.json)
• REACT_APP_API_URL - API URL for the React frontend (e.g., http://localhost:3002/api, https://api.mycompany.com)
• PORT - Server port (e.g., 3002, 8080)
• NODE_ENV - Node.js environment (e.g., development, production, test)
• ENABLE_CONSOLE_LOGS - Enable console logging (e.g., true, false)

Monitoring and Maintenance

Logs

The application logs are stored in:

• Server logs: server/test-processing.log

Regular Maintenance

1. Log Rotation

   • Configure log rotation for application logs
   • Monitor disk space usage
   • Archive old logs

2. Security Updates

   • Regularly update dependencies
   • Monitor security advisories
   • Apply security patches promptly

3. Performance Monitoring

   • Monitor server resources
   • Track API response times
   • Watch for memory leaks

4. Test Data Management

   • Monitor S3 bucket usage
   • Clean up old test reports
   • Optimize data retrieval patterns

Feature Flags

The application includes a feature flags system that allows you to enable or disable specific features dynamically.

Available Feature Flags

The feature flags are defined in client/src/config/featureFlags.ts and include:

Page-Level Flags

• enableBugTracker - Enable/disable the Bug Tracker page
• enableTestPyramid - Enable/disable the Test Pyramids page

Test Analytics Tab Flags

• enableBackendTab - Enable/disable the Backend tab in Test Analytics
• enableE2ETestsTab - Enable/disable the E2E Tests tab in Test Analytics
• enableIntegrationTestsTab - Enable/disable the Integration Tests tab in Test Analytics

Test Analytics Feature Flags

• enableDetailedResults - Enable/disable detailed test results display
• enableDurationDistribution - Enable/disable test duration distribution charts
• enableSlowestTests - Enable/disable slowest tests table

Using Feature Flags

Feature flags are stored in browser localStorage and can be managed programmatically:

import { useFeatureFlags, updateFeatureFlags } from './config/featureFlags';

// In a component
const { flags } = useFeatureFlags();

// Check if a feature is enabled
if (flags.enableBugTracker) {
  // Render bug tracker component
}

// Update feature flags
updateFeatureFlags({ enableBugTracker: false });

Console Management

You can also manage feature flags directly from the browser console using JavaScript:

// View current feature flags
console.log(JSON.parse(localStorage.getItem('featureFlags') || '{}'));

// Quick disable bug tracker (copy-paste ready)
localStorage.setItem('featureFlags', JSON.stringify({enableBugTracker: false}));


// Enable/disable specific features
const flags = JSON.parse(localStorage.getItem('featureFlags') || '{}');
flags.enableBugTracker = false;
flags.enableTestPyramid = true;
localStorage.setItem('featureFlags', JSON.stringify(flags));

// Reset all flags to default (all enabled)
localStorage.removeItem('featureFlags');

// Refresh the page to see changes
location.reload();

Default Configuration

All feature flags are enabled by default. You can modify the default values in the defaultFeatureFlags object in the feature flags configuration file.

API Response Examples

The API provides test analytics data through various endpoints. Here are examples of the main response structures:

Detailed Test Report Response

{
  "testSuites": [
    {
      "id": "suite-1",
      "name": "User Authentication Tests",
      "passRate": 95.5,
      "duration": 120.5,
      "tests": [
        {
          "id": "test-1",
          "name": "Login with valid credentials",
          "status": "passed",
          "duration": 2.3,
          "timestamp": "2024-01-15T10:30:00Z",
          "tags": ["@FTR-123", "authentication"],
          "skipped": false,
          "passRate": 100,
          "flaky": false
        }
      ],
      "skipped": false,
      "tags": ["@FTR-123", "authentication"],
      "stability": 98.2
    }
  ],
  "totalTests": 150,
  "passedTests": 143,
  "failedTests": 5,
  "flakyTests": 2,
  "skippedTests": 0,
  "averageDurationPerTest": 1.8,
  "overallExecutionTime": 270.5,
  "lastRunTime": "2024-01-15T10:30:00Z",
  "overallSuccessRate": 95.3,
  "overallStability": 97.1,
  "testDurations": [
    {
      "name": "Login with valid credentials",
      "duration": 2.3,
      "status": "PASSED",
      "title": "User Authentication",
      "file": "auth.spec.ts",
      "tags": ["@FTR-123", "authentication"]
    }
  ]
}

Unit Test Report Response

{
  "totalTests": 1250,
  "passedTests": 1200,
  "failedTests": 45,
  "skippedTests": 5,
  "totalDuration": 180.5,
  "successRate": 96.0,
  "overallStability": 94.2,
  "averageDurationPerTest": 0.14,
  "lastRunTime": "2024-01-15T10:30:00Z",
  "flakyTests": 8,
  "testSuites": [
    {
      "name": "UserService",
      "tests": [
        {
          "fullName": "UserService.createUser should create user successfully",
          "title": "createUser should create user successfully",
          "status": "passed",
          "duration": 0.05,
          "featureCode": "@FTR-123",
          "team": "Team A"
        }
      ],
      "passRate": 98.5,
      "duration": 15.2,
      "stability": 96.8,
      "isFlaky": false,
      "tags": ["@FTR-123", "user-service"],
      "teams": ["Team A"]
    }
  ],
  "testsByTeam": {
    "Team A": {
      "total": 450,
      "passed": 435,
      "failed": 12,
      "skipped": 3,
      "duration": 65.2,
      "tests": [...]
    }
  },
  "testsWithoutTeam": [...],
  "featureCodeMapping": {
    "@FTR-123": "Team A",
    "@FTR-456": "Team B"
  },
  "teams": ["Team A", "Team B", "Team C"]
}

PHP XML Test Report Response

{
  "summary": {
    "totalTests": 850,
    "passedTests": 820,
    "failedTests": 25,
    "skippedTests": 5,
    "warningTests": 0,
    "errors": 0,
    "warnings": 0,
    "executionTime": 45.2,
    "testSuites": [...],
    "suitesWithWarnings": [...]
  },
  "testsByLayer": {
    "unitTests": 600,
    "integrationTests": 200,
    "apiTests": 50
  }
}

Bug Data Response

[
  {
    "id": "bug-123",
    "title": "Login page crashes on mobile devices",
    "description": "Users report that the login page crashes when accessed from mobile devices",
    "status": "In Progress",
    "team": "Frontend Team",
    "customFields": {
      "severity": "High",
      "priority": "P1",
      "environment": "Production"
    },
    "createdAt": "2024-01-10T09:00:00Z",
    "updatedAt": "2024-01-15T14:30:00Z"
  }
]

Complete API Documentation

For the complete API specification, see the OpenAPI document at api/openapi.yaml. This file contains detailed information about all available endpoints, request/response schemas, and data models.

Troubleshooting

Common Issues

1. API Connection Issues:

   • Check console for errors with server hostname
   • Verify environment variables in .env.* files
   • The application automatically converts server hostname to localhost in browser

2. TypeScript Errors:

   • Run npm run setup-api to regenerate the API compatibility layer
   • Check that the API types are being correctly exported

3. Docker Issues:

   • Use docker-compose down && docker-compose up --build to ensure a clean rebuild
   • Verify that ports 3000 and 3002 are not in use by other applications

4. AWS Credential Issues:

   • Use the update_env.sh script to sync credentials from your AWS CLI configuration
   • Ensure AWS session tokens haven't expired
   • Check IAM role permissions in production

5. Test Failures:

   • Ensure environment variables are set in CI environment
   • Check that mocks are properly configured
   • Verify that test data matches expected formats

6. S3 Connection Issues:

   • Verify bucket names and regions are correct
   • Check IAM permissions for S3 access
   • Ensure proper credential configuration

7. Feature Flag Issues:

   • Clear browser localStorage if feature flags are not working as expected
   • Check the browser console for feature flag errors
   • Verify that the feature flags configuration file is properly imported

Performance Optimization

1. Caching: Implement caching for frequently accessed data
2. Pagination: Use pagination for large datasets
3. Lazy Loading: Load data on demand

Contributors

• Calvin Custodio - Main development and architecture - GitHub @Calvincac
• Enrique Garbi - Deployment and infrastructure tasks

Development Tools

• Cursor - AI-powered development environment that assisted in code generation and development tasks

License

This project is licensed under the GNU General Public License v3.0 - see the LICENSE file for details.

The GNU GPL v3 is a free, copyleft license that ensures the software remains free and open source. This means:

• You can use, modify, and distribute this software
• Any derivative works must also be licensed under the GPL v3
• Source code must be made available to users
• The software comes with no warranty

For more information about the GNU GPL v3, visit https://www.gnu.org/licenses/gpl-3.0.html.