Add Millisecond or Decimal Second Options for API Request Timing

[neno-is-ooo]

Feature Request: Add Millisecond or Decimal Second Options for API Request Timing

Current Limitation

The UI currently only allows setting minimum time between API requests in whole seconds (1, 2, 3 seconds, etc.). This creates unnecessary restrictions when working with high-performance AI models like Claude 3.7 Sonnet that support much faster request rates.

Proposed Enhancement

Add support for:

• Millisecond input (e.g., 200ms, 400ms)
• Decimal second input (e.g., 0.2s, 0.5s)

Note: Models like Claude 3.7 Sonnet can support requests as frequent as every 60ms, but are currently limited by the UI's minimum 1-second interval.

Benefits

1. Improved Responsiveness: Models like Claude 3.7 Sonnet support 1,000 RPM (one request per 60ms), but the current UI forces a minimum 1-second delay, reducing potential throughput by ~94%.

2. Better Rate Limit Optimization: Different AI models have varying rate limits - enabling finer-grained control allows developers to precisely tune request timing to each model's capabilities.

3. Enhanced Developer Experience: Smoother, more responsive coding assistance without unnecessary waiting periods.

Use Cases

• Fast completions: Fast completion suggestions while coding with Claude (optimal at ~200-400ms)
• Model-specific timing: Different timing profiles for different models (Claude vs Gemini)
• Task optimization: Fine-tuning performance for specific tasks that require different response cadences

Implementation Suggestion

A simple input field that accepts decimal values (0.2, 0.3, etc.) or values with "ms" suffix, with appropriate validation to prevent values below model-specific minimums.

Thank you for considering this enhancement to improve the developer experience!