Spider Impact AI POC

A proof-of-concept Spring Boot application that integrates with Anthropic's AI for chat functionality.

Quick Start

1. Check Java Version (requires JDK 17+)

   java -version

2. Run the Application

   gradle clean build installBootDist
   gradle bootRun

3. Open in Browser

   http://localhost:8080
   

4. Development Tools

   Creating Code Prompt Files:

   # Create a file with all project source code for AI prompts
   files-to-prompt src/ --ignore "bundle.js" --ignore "chart.css" --ignore "*.csv" > raw_files.txt

   Building Enhanced AI Prompts:

   # Combine your clipboard content with the project files for a comprehensive prompt
   # 1. Copy your instructions to clipboard
   # 2. Run this command
   # 3. Paste the result into the AI interface
   (pbpaste | sed '/^#/d' && cat files_prompt.txt && files-to-prompt src/ --ignore "bundle.js" --ignore "chart.css" --ignore "*.csv") | pbcopy

Setup Requirements

• JDK 17 or later
• Internet connection
• Anthropic API key set as an environment variable

Setting up API Key

The application requires an Anthropic API key to function. For security reasons, the API key should be set as an environment variable.

First, check if the variable is already set:

echo $ANTHROPIC_API_KEY

If not set or you need to update it:

# For Unix/Linux/macOS
export ANTHROPIC_API_KEY=your-api-key-here

To make it persistent, add the export line to your shell profile:

• Bash: ~/.bash_profile or ~/.bashrc
• Zsh: ~/.zshrc
• Fish: ~/.config/fish/config.fish

Example:

echo 'export ANTHROPIC_API_KEY="your-api-key-here"' >> ~/.bash_profile
source ~/.bash_profile

Features

• Web-based chat interface
• Real-time AI responses using Anthropic's API
• Built on Spring Boot's reactive web stack
• Comprehensive testing framework

Documentation

• Spring Boot Docs
• Spring AI Docs

Testing Framework

The project includes a comprehensive testing framework for both standard Java tests and AI-specific tests.

Test Hierarchy

Testing Framework
├── All Java Tests (gradle test)
│   ├── Unit Tests (gradle unitTest)
│   └── Integration Tests (gradle integrationTest)
└── All AI Tests (gradle aiTestAll)
    ├── Critical Path Tests (gradle aiTestCriticalPath) 
    ├── Adversarial Tests (gradle aiTestAdversarial)
    ├── Conversation Tests (gradle aiTestConversation)
    ├── Regression Tests (gradle aiTestRegression)
    └── Single AI Test (gradle aiTestRun -PtestId=<id>)

Running Java Tests

# Run all Java tests
gradle test

# Run unit tests only
gradle unitTest

# Run integration tests only 
gradle integrationTest

# Run a specific Java test class
gradle test --tests "com.spiderimpact.ai.TestClassName"

Running AI Tests

# Run all AI tests
gradle aiTestAll

# Run critical path tests (core functionality)
gradle aiTestCriticalPath

# Run adversarial tests (edge cases)
gradle aiTestAdversarial

# Run conversation tests (multi-turn)
gradle aiTestConversation

# Run regression tests
gradle aiTestRegression

# Run a single AI test
gradle aiTestRun -PtestId=bar-chart-basic-test

# List all available AI tests
gradle aiTestList

Analyzing Test Results

After running tests, you can view and analyze the results through various reports. Tests do not automatically generate reports - you need to generate them separately after test execution.

# View immediate test execution results
gradle aiTestResults -PexecutionId=<execution-id>

Report Generation Workflow

1. Run your tests to generate execution data
2. Note the execution ID from the test output
3. Generate the appropriate report using one of these commands:

# Generate detailed report for a specific test execution
# Formats available: html (default), json, csv
gradle aiTestReport -PexecutionId=<id> -Pformat=html

# Generate aggregated report for all runs of a specific test definition
# Useful for tracking performance over time
gradle aiTestReportBatch -PtestId=<id>

# Generate report for all tests in a date range
# Useful for weekly/monthly quality reviews
gradle aiTestReportDateRange -PstartDate=2025-03-01 -PendDate=2025-03-31

# Generate visual test coverage dashboard
# Provides a high-level overview of all test results
gradle aiTestDashboard

All reports are stored in ./ai-testing-results/reports/ organized by type:

• Execution reports: ./ai-testing-results/reports/execution/
• Batch reports: ./ai-testing-results/reports/batch/
• Date range reports: ./ai-testing-results/reports/date-range/
• Dashboard: ./ai-testing-results/dashboard/

Model Comparison

# Compare models on a specific test
gradle aiTestModelComparison -PtestId=conversation-test -Pmodels=gpt4,claude,llama3

Documentation

For more details on the testing framework, refer to:

• Testing Framework README - Overview and capabilities
• Test Definitions README - Test structure and format
• Reporting README - Report types and formats