Terminal output limit - # Feature Request: Flexible terminal size limit Based on Model Selection

[scottnzuk]

Feature Request: Flexible Context Window Scaling Based on Model Selection

Summary

Add an intelligent, model-aware context window scaling mechanism that dynamically adjusts the maximum context size based on the selected model's capabilities.

Motivation

Currently, users must manually set context window sizes without clear guidance on optimal settings for different models. This feature would:

• Automatically recommend and limit context window sizes
• Prevent users from setting unrealistic context sizes
• Improve performance and resource management

Proposed Solution

Implement a model-specific context window scaling system with the following characteristics:

Proposed Implementation

1. Create a model capability database with:

   • Model name
   • Maximum recommended context window
   • Optimal performance range
   • Computational requirements

2. Dynamic UI/Configuration options:

   • Auto-select default based on model
   • Slider/input with model-specific max limits
   • Warning/tooltip for suboptimal settings

Example Model Mapping (Hypothetical)

Model	Default Context	Max Context	Recommended Range
GPT-3.5	4000	4000	2000-4000
Claude-2	100,000	200,000	50,000-150,000
Llama-2 7B	2000	4000	1000-3000
Custom Model	User-defined	User-defined	Validated range

User Experience Flow

1. Select model
2. Context window automatically scales
3. Optional manual adjustment with inline guidance
4. Performance recommendations displayed

Expected Benefits

• Improved user experience
• Reduced configuration errors
• Automatic optimization
• Clearer model capability understanding

Implementation Considerations

• Require ongoing model capability database updates
• Fallback to conservative defaults
• Provide override options for advanced users

Questions for Discussion

• How granular should model-specific scaling be?
• What validation mechanisms are needed?
• How to handle emerging/custom models?

Even if this was not automatic but gave text guidelines next to slider?