feat: implement exponential backoff and rate limiting for AI API calls

Major improvements to API reliability and performance:

## Rate Limiting & Retry Logic
- Add RetryHandler module with exponential backoff algorithm
- Implement configurable retry parameters (max_retries, base_delay, max_delay)
- Add 10% jitter to prevent thundering herd problems
- Support Retry-After header extraction for rate limit compliance
- Handle different HTTP error codes appropriately (429, 5xx vs 4xx)

## Memory Management Improvements
- Replace magic numbers with named constants (LARGE_DIFF_THRESHOLD, etc.)
- Convert all diff processing from split('\n') to StringIO streaming
- Add proper resource cleanup with ensure blocks
- Prevent memory spikes on large diffs with consistent streaming approach

## API Provider Enhancements
- HTTPClient: Smart retry logic for rate limits and server errors
- DustProvider: Exponential backoff for conversation polling
- AnthropicProvider: Benefits from HTTPClient retry improvements
- Comprehensive error logging with retry attempt visibility

## Testing & Documentation
- Update test expectations for new retry behavior
- Mock sleep calls to maintain fast test execution (0.15s for 50 tests)
- Rename documentation file to AI_TEST_RUNNER.md for consistency
- Update all references to use proper 'AI Test Runner' naming

## Performance Benefits
- Graceful handling of temporary API failures
- Reduced failure rates through intelligent backoff
- Optimal retry timing (1s → 2s → 4s → 8s → 16s, capped at 30s)
- Fast recovery for transient issues

All 50 tests pass. No breaking changes to existing functionality.

Author: mickael-palma-wttj

.github/scripts/ai_test_runner.rb

@@ -51,6 +51,11 @@ def pr_mode?
 
 # Service to analyze git changes and extract relevant information
 class GitChangeAnalyzer
+  # Constants for diff processing limits
+  LARGE_DIFF_THRESHOLD = 10_000_000 # 10MB - threshold for switching to streaming mode
+  MEMORY_DIFF_THRESHOLD = 1_000_000 # 1MB - threshold for memory-efficient processing
+  MAX_STREAMING_LINES = 100 # Maximum lines to process in streaming mode to avoid memory bloat
+
   attr_reader :logger
 
   def initialize(logger)
@@ -117,8 +122,8 @@ def get_git_diff(base, head)
 
     # Log diff size for monitoring
     diff_size = stdout.bytesize
-    if diff_size > 10_000_000 # 10MB
-      logger.warn "Large diff detected: #{diff_size / 1_000_000}MB - using streaming mode"
+    if diff_size > LARGE_DIFF_THRESHOLD
+      logger.warn "Large diff detected: #{diff_size / MEMORY_DIFF_THRESHOLD}MB - using streaming mode"
     else
       logger.debug "Diff size: #{diff_size} bytes"
     end
@@ -128,7 +133,7 @@ def get_git_diff(base, head)
 
   def parse_diff_for_files(diff_output)
     # For very large diffs, use streaming to avoid loading everything into memory
-    if diff_output.length > 1_000_000 # 1MB threshold
+    if diff_output.length > MEMORY_DIFF_THRESHOLD
       parse_diff_streaming(diff_output)
     else
       parse_diff_in_memory(diff_output)
@@ -171,7 +176,12 @@ def parse_diff_in_memory(diff_output)
     changed_files = []
     current_file = nil
 
-    diff_output.split("\n").each do |line|
+    # Use StringIO for memory-efficient line processing instead of split("\n")
+    io = StringIO.new(diff_output)
+
+    io.each_line do |line|
+      line = line.chomp # Remove newline without loading full diff
+
       if line.start_with?('diff --git')
         # Extract filename from "diff --git a/path/to/file b/path/to/file"
         match = line.match(%r{diff --git a/(.*?) b/(.*)})
@@ -188,6 +198,8 @@ def parse_diff_in_memory(diff_output)
     end
 
     changed_files.uniq { |f| f[:path] }
+  ensure
+    io&.close
   end
 
   def determine_file_type(file_path)
@@ -209,13 +221,16 @@ def determine_file_type(file_path)
 
   def extract_changes_for_file(diff_output, file_path)
     # For large diffs, limit change extraction to avoid memory issues
-    return extract_changes_streaming(diff_output, file_path) if diff_output.length > 1_000_000 # 1MB threshold
+    return extract_changes_streaming(diff_output, file_path) if diff_output.length > MEMORY_DIFF_THRESHOLD
 
-    lines = diff_output.split("\n")
+    # Use StringIO for memory-efficient processing instead of split("\n")
+    io = StringIO.new(diff_output)
     file_diff_started = false
     changes = { added: [], removed: [], context: [] }
 
-    lines.each do |line|
+    io.each_line do |line|
+      line = line.chomp # Remove newline without loading full diff
+
       if line.include?("b/#{file_path}")
         file_diff_started = true
         next
@@ -235,6 +250,8 @@ def extract_changes_for_file(diff_output, file_path)
     end
 
     changes
+  ensure
+    io&.close
   end
 
   def extract_changes_streaming(diff_output, file_path)
@@ -244,7 +261,7 @@ def extract_changes_streaming(diff_output, file_path)
     file_diff_started = false
     changes = { added: [], removed: [], context: [] }
     line_count = 0
-    max_lines = 100 # Limit to first 100 lines of changes to avoid memory bloat
+    max_lines = MAX_STREAMING_LINES
 
     io.each_line do |line|
       line = line.chomp

.github/scripts/shared/ai_services.rb

@@ -4,8 +4,75 @@
 require 'json'
 require 'logger'
 
+# Retry handler with exponential backoff for API calls
+module RetryHandler
+  # Constants for retry configuration
+  DEFAULT_MAX_RETRIES = 3
+  DEFAULT_BASE_DELAY = 1.0  # Initial delay in seconds
+  DEFAULT_MAX_DELAY = 30.0  # Maximum delay in seconds
+  DEFAULT_BACKOFF_FACTOR = 2.0 # Exponential backoff multiplier
+
+  # Retry with exponential backoff
+  def retry_with_backoff(max_retries: DEFAULT_MAX_RETRIES, base_delay: DEFAULT_BASE_DELAY,
+                         max_delay: DEFAULT_MAX_DELAY, backoff_factor: DEFAULT_BACKOFF_FACTOR)
+    retries = 0
+
+    loop do
+      result = yield(retries)
+      return result
+    rescue StandardError => e
+      retries += 1
+
+      if retries >= max_retries
+        logger.error "❌ Max retries (#{max_retries}) exceeded. Last error: #{e.message}"
+        raise e
+      end
+
+      delay = calculate_delay(retries, base_delay, max_delay, backoff_factor)
+      logger.warn "⚠️ Retry #{retries}/#{max_retries} after #{delay}s. Error: #{e.message}"
+
+      sleep(delay)
+    end
+  end
+
+  # Handle rate limiting response with exponential backoff
+  def handle_rate_limit_error(response, retries, max_retries)
+    return false if retries >= max_retries - 1
+
+    # Extract rate limit information if available
+    retry_after = extract_retry_after(response)
+    delay = retry_after || calculate_delay(retries + 1, DEFAULT_BASE_DELAY, DEFAULT_MAX_DELAY, DEFAULT_BACKOFF_FACTOR)
+
+    logger.warn "🚫 Rate limited. Waiting #{delay}s before retry (attempt #{retries + 1}/#{max_retries})"
+    sleep(delay)
+    true
+  end
+
+  private
+
+  def calculate_delay(attempt, base_delay, max_delay, backoff_factor)
+    # Exponential backoff with jitter
+    delay = base_delay * (backoff_factor**(attempt - 1))
+    delay = [delay, max_delay].min # Cap at max_delay
+    delay += rand * 0.1 * delay # Add up to 10% jitter to avoid thundering herd
+    delay.round(2)
+  end
+
+  def extract_retry_after(response)
+    return nil unless response.respond_to?(:headers) || response.respond_to?(:header)
+
+    # Try to extract Retry-After header
+    retry_after = response.respond_to?(:headers) ? response.headers['Retry-After'] : response.header['Retry-After']
+    return nil unless retry_after
+
+    retry_after.to_i if retry_after.to_i.positive?
+  end
+end
+
 # Shared HTTP client helper
 class HTTPClient
+  include RetryHandler
+
   attr_reader :logger, :http_timeout, :read_timeout
 
   def initialize(logger, timeouts = {})
@@ -19,38 +86,60 @@ def post(uri, headers, body)
     headers.each { |key, value| request[key] = value }
     request.body = body
 
-    make_request(uri, request)
+    make_request_with_retry(uri, request)
   end
 
   def get(uri, headers)
     request = Net::HTTP::Get.new(uri)
     headers.each { |key, value| request[key] = value }
 
-    make_request(uri, request)
+    make_request_with_retry(uri, request)
   end
 
   private
 
-  def make_request(uri, request)
+  def make_request_with_retry(uri, request)
+    retry_with_backoff do |retries|
+      make_request(uri, request, retries)
+    end
+  end
+
+  def make_request(uri, request, retries = 0)
     response = Net::HTTP.start(uri.hostname, uri.port,
                                use_ssl: true,
                                open_timeout: @http_timeout,
                                read_timeout: @read_timeout) do |http|
       http.request(request)
     end
 
-    handle_response(response)
+    handle_response(response, retries)
   rescue Net::OpenTimeout, Net::ReadTimeout => e
     logger.error "HTTP request timed out: #{e.message}"
     raise StandardError, "HTTP request timed out after #{@read_timeout} seconds"
   end
 
-  def handle_response(response)
-    unless response.code == '200'
+  def handle_response(response, retries)
+    case response.code
+    when '200'
+      parse_response_body(response)
+    when '429' # Rate limited
+      raise StandardError, 'Rate limited - will retry' if handle_rate_limit_error(response, retries, DEFAULT_MAX_RETRIES)
+
+      raise StandardError, 'Rate limited - max retries exceeded'
+
+    when '500', '502', '503', '504' # Server errors - retry
+      error_msg = "Server error #{response.code}: #{response.body}"
+      logger.warn error_msg
+      raise StandardError, error_msg
+    else
       error_msg = "HTTP request failed with status #{response.code}: #{response.body}"
       logger.error error_msg
       raise StandardError, error_msg
     end
+  end
+
+  def parse_response_body(response)
+    return response.body if response.body.nil? || response.body.empty?
 
     JSON.parse(response.body)
   rescue JSON::ParserError => e
@@ -204,6 +293,7 @@ def extract_final_content(agent_messages)
 
 # Dust AI provider
 class DustProvider < AIProvider
+  include RetryHandler
   include DustResponseProcessor
   API_BASE_URL = 'https://dust.tt'
 
@@ -238,7 +328,7 @@ def make_request(prompt)
     logger.info "⏳ Waiting #{initial_wait} seconds for agent to process..."
     sleep(initial_wait)
 
-    get_response_with_retries(conversation_id)
+    get_response_with_exponential_backoff(conversation_id)
   end
 
   def provider_name
@@ -283,39 +373,29 @@ def get_response(conversation_id)
     extract_content(response)
   end
 
-  def get_response_with_retries(conversation_id, max_retries = 5)
-    retries = 0
+  def get_response_with_exponential_backoff(conversation_id)
+    logger.info "🔍 Fetching response for conversation: #{conversation_id}"
 
-    while retries < max_retries
-      response = attempt_fetch_response(conversation_id, retries, max_retries)
+    retry_with_backoff(max_retries: 5, base_delay: 2.0, max_delay: 30.0) do |retries|
+      logger.info "🔄 Attempting to fetch response (attempt #{retries + 1}/5) for conversation: #{conversation_id}"
+
+      response = get_response(conversation_id)
 
       if response_is_valid?(response)
         logger.info "✅ Response validated successfully for conversation: #{conversation_id}"
         return response
       end
 
       logger.info "⏳ Response not valid, will retry. Response: '#{response.to_s[0..100]}...'"
-      handle_retry_delay(retries, max_retries, conversation_id)
-      retries += 1
+      raise StandardError, 'Response not ready yet'
     end
-
-    logger.error "❌ Dust agent did not respond after #{max_retries} attempts (conversation: #{conversation_id})"
+  rescue StandardError => e
+    logger.error "❌ Failed to get response after maximum retries for conversation: #{conversation_id}. Error: #{e.message}"
     conversation_uri = "#{API_BASE_URL}/api/v1/w/#{workspace_id}/assistant/conversations/#{conversation_id}"
     logger.error "🔗 Check conversation status at: #{conversation_uri}"
     nil
   end
 
-  def attempt_fetch_response(conversation_id, retries, max_retries)
-    logger.info "🔄 Attempting to fetch response (attempt #{retries + 1}/#{max_retries}) for conversation: #{conversation_id}"
-    get_response(conversation_id)
-  rescue StandardError => e
-    logger.warn "⚠️ Error fetching response (attempt #{retries + 1}) for conversation #{conversation_id}: #{e.message}"
-    raise e if retries >= max_retries - 1
-
-    sleep(3)
-    'retry_needed'
-  end
-
   def response_is_valid?(response)
     return false if response.nil?
     return false if response == 'retry_needed'
@@ -324,14 +404,6 @@ def response_is_valid?(response)
     logger.debug "Response validated as valid: length=#{response.to_s.length}"
     true
   end
-
-  def handle_retry_delay(retries, max_retries, conversation_id)
-    return unless retries < max_retries - 1
-
-    wait_time = (retries + 1) * 5 # 5s, 10s, 15s, 20s
-    logger.info "⏳ Agent hasn't responded yet, waiting #{wait_time} seconds before retry (conversation: #{conversation_id})..."
-    sleep(wait_time)
-  end
 end
 
 # AI provider factory

README.md

@@ -25,7 +25,7 @@ This project includes an **AI-powered Test Runner** that intelligently selects a
 
 The AI test runner automatically triggers on all pushes and pull requests, analyzing your changes and running only the necessary tests.
 
-**📖 [Learn more about the AI Test Runner →](./doc/SMART_TEST_RUNNER.md)**
+**📖 [Learn more about the AI Test Runner →](./doc/AI_TEST_RUNNER.md)**
 
 ## Setup
 

doc/SMART_TEST_RUNNER.md renamed to doc/AI_TEST_RUNNER.md

@@ -1,11 +1,11 @@
-# 🤖 Smart Test Runner
+# 🤖 AI Test Runner
 
 An AI-powered GitHub Action that intelligently selects and runs only the tests relevant to your code changes, reducing CI time while maintaining comprehensive coverage.
 
 ## Features
 
 - **🧠 AI-Powered Analysis**: Uses Claude 3 Sonnet to analyze code changes and understand test dependencies
-- **🎯 Smart Test Selection**: Identifies both direct and indirect tests that may be affected by changes
+- **🎯 AI Test Selection**: Identifies both direct and indirect tests that may be affected by changes
 - **⚡ Performance Optimization**: Runs only relevant tests instead of the entire test suite
 - **📊 Detailed Reporting**: Provides comprehensive analysis of why tests were selected
 - **🔄 Fallback Safety**: Falls back to running all tests if AI analysis fails
@@ -42,7 +42,7 @@ Set repository variables:
 
 ### 3. The Workflow is Ready!
 
-The smart test runner is already configured in `.github/workflows/smart_tests.yml` and will automatically:
+The AI test runner is already configured in `.github/workflows/smart_tests.yml` and will automatically:
 
 - Trigger on pushes to `main` and `develop` branches
 - Trigger on pull requests to `main` and `develop` branches
@@ -51,7 +51,7 @@ The smart test runner is already configured in `.github/workflows/smart_tests.ym
 
 ## Manual Usage
 
-You can also run the smart test selector locally:
+You can also run the AI test selector locally:
 
 ```bash
 # Set required environment variables
@@ -112,7 +112,7 @@ The AI considers multiple factors when selecting tests:
 
 ## Output Files
 
-The smart test runner generates several output files:
+The AI test runner generates several output files:
 
 ### `tmp/selected_tests.txt`
 Simple list of selected test files (one per line) used by the GitHub workflow.
@@ -206,7 +206,7 @@ Enable debug logging by setting the log level:
 
 ```ruby
 logger = Logger.new($stdout, level: Logger::DEBUG)
-runner = SmartTestRunner.new(config, logger)
+runner = AITestRunner.new(config, logger)
 ```
 
 ## Contributing
@@ -250,13 +250,13 @@ The system automatically falls back to a built-in prompt if the external file is
 ## Architecture
 
 ```
-SmartTestRunner
-├── SmartTestConfig          # Configuration management
+AITestRunner
+├── AITestConfig             # Configuration management
 ├── GitChangeAnalyzer        # Git diff analysis and parsing
 ├── TestDiscoveryService     # Test file discovery and mapping
 ├── AITestSelector          # AI-powered test selection
 │   └── ai_test_selection_prompt.md  # External AI prompt template
-└── SmartTestRunner         # Main orchestrator
+└── AITestRunner            # Main orchestrator
 ```
 
 ## Performance Benefits