Feature Request: Improve context management #544
Replies: 12 comments 8 replies
-
|
https://github.com/GreatScottyMac/roo-code-memory-bank i found this and seem to be helpful to me |
Beta Was this translation helpful? Give feedback.
-
|
I'm wondering if anyone knows whether the Cline Memory Bank (remembering that Roo Code is a fork of Cline) is compatible with Roo Code. The Cline Memory Bank looks simpler or at least more straightforward compared to the memory bank available at https://github.com/GreatScottyMac/roo-code-memory-bank . This one might work better, but I haven't had the time to read through the prompts to fully understand them. |
Beta Was this translation helpful? Give feedback.
-
|
Hi everyone! Looking at this discussion, I see we're all facing similar challenges with context management. The current memory bank solutions seem focused on storing predefined context, but I think we need something more dynamic. What if, instead of just managing what we already have in context, the AI could proactively discover what context it needs? My proposal builds on these ideas but takes a different approach:
I've been experimenting with LangChain for implementing this kind of system, and it provides good tools for memory management that could address the exact issues @nissa-seru mentioned about garbage collection priorities and duplicates. This would go beyond what the current memory banks provide by being adaptive rather than static. Would this direction be interesting to explore? I'd be happy to elaborate more on the technical approach or even work on a proof of concept if there's interest! |
Beta Was this translation helpful? Give feedback.
-
🌐 Versión en Español📂 Solicitud: Mejoras en la Gestión de Archivos Duplicados y Contexto en RooCodeDescripción:
Beneficios:
🌐 English Version📂 Request: Improvements in Duplicate File Management and Context Handling in RooCodeDescription:
Benefits:
|
Beta Was this translation helpful? Give feedback.
-
|
I'm currently workshopping a design for a ContextManager that organizes messages as a DAG of content addressable messages. The idea is to allow the context history to be manipulated, supporting functions like updating older messages to have the most up to date content, like files, summarizing past messages and thinking (using a local or remote LLM), providing tool usage to the LLM for managing context itself, like collapsing and expanding messages (similar to a summary, but more concise and with more meta data for the LLM), and other uses I haven't thought of. While it's intended to support a wide variety of manipulations, the "demo" use case would be to provide a block of messages after the system block that acts as a file browser and manager. The context for each message in the block is real time up to date with each prompt, and only has open the details the LLM has requested, collapsing the rest. Here's the design overview so far: 100-199: Overview1. Introduction1.1 PurposeThe primary purpose of the ContextManagement system is to strategically manage and manipulate message histories within AI interactions to optimize both cost and performance. By intelligently structuring and operating on conversational context, the system aims to reduce context size, eliminate redundancy, and address staleness in chat histories. This will lead to more efficient and effective AI applications, enabling them to maintain coherent conversations while minimizing computational overhead and API costs. Furthermore, the system is designed to maintain a complete history of each prompt and its corresponding LLM response, ensuring traceability and the ability to selectively modify and restore specific points in the conversation flow. 1.2 ScopeThe ContextManagement system is designed to provide a flexible and powerful framework for managing conversational context, primarily focused on enabling efficient manipulation and optimization rather than inherent persistent storage. Key capabilities and scope considerations include:
1.3 Definitions, Acronyms, and Abbreviations
1.4 References
|
Beta Was this translation helpful? Give feedback.
-
|
Fundamental to any content management design I have floating around in my head is first a refactor that maintains functional parity with existing behavior. It's intent is to encapsulate the structures that maintain the message histories for the LLM and UI. Proposal: Encapsulate Conversation History in ClineContextManagerIntroductionThis document proposes the encapsulation of Analysis of Current Usage and InvariantsBased on the analysis of apiConversationHistory
clineMessages
Relationship Between ListsAnalysis of the codebase reveals that operations on
Invariants
Architectural ConsiderationsWhen designing the Approach 1: Combined Encapsulation (Selected Approach)This approach encapsulates both Justification:
Approach 2: Separate Encapsulation with Synchronizer (Alternative Considered)This approach would encapsulate each list in its own class, with a third component responsible for synchronization. Considerations: This approach would make sense if:
From the code analysis, none of these conditions appear to be true. The structures are tightly coupled conceptually, share important invariants, are frequently modified together, and require non-trivial synchronization logic. Decision: The combined encapsulation approach was selected because it better reflects the conceptual relationship between the structures and provides a more robust foundation for future development. Refined Plan for ClineContextManagerThe refactoring will be implemented in two phases: Phase 1: Basic Encapsulation
Phase 2: Enhanced SynchronizationIn the second phase, we'll focus on better encapsulating the invariants and synchronization logic within the
Implementation DetailsInterface Definition (Phase 1)interface IClineContextManager {
// Basic operations - Return readonly copies to prevent unintended modifications
getClineMessages(): ReadonlyArray<ClineMessage>;
getApiConversationHistory(): ReadonlyArray<Anthropic.MessageParam>;
// Message operations
addClineMessage(message: ClineMessage): Promise<void>;
overwriteClineMessages(newMessages: ClineMessage[]): Promise<void>;
updateClineMessage(partialMessage: ClineMessage): Promise<void>;
addToApiConversationHistory(message: Anthropic.MessageParam): Promise<void>;
overwriteApiConversationHistory(newHistory: Anthropic.MessageParam[]): Promise<void>;
// Persistence
saveContext(): Promise<void>;
loadContext(): Promise<void>;
// Complex operations
deleteMessage(timestamp: number, deleteSubsequent: boolean): Promise<void>;
forkFromMessage(timestamp: number): Promise<{
clineMessages: ReadonlyArray<ClineMessage>;
apiConversationHistory: ReadonlyArray<Anthropic.MessageParam>;
}>;
// Validation
validateConsistency(): boolean;
// Event handling
on(event: 'messageCreated' | 'messageUpdated', listener: (message: ClineMessage) => void): void;
off(event: 'messageCreated' | 'messageUpdated', listener: (message: ClineMessage) => void): void;
}Enhanced Interface (Phase 2)interface IClineContextManager {
// Phase 1 methods...
// Higher-level operations (Phase 2)
addUserMessage(text: string, images?: string[]): Promise<void>;
addAssistantMessage(content: string | Anthropic.Messages.ContentBlockParam[]): Promise<void>;
addSystemMessage(content: string): Promise<void>;
// Transformation methods
convertToApiMessage(clineMessage: ClineMessage): Anthropic.MessageParam;
convertToClineMessage(apiMessage: Anthropic.MessageParam): ClineMessage;
// Consistency management
ensureConsistency(): Promise<void>;
validateMessagePair(clineIndex: number, apiIndex: number): boolean;
// Enhanced complex operations
truncateConversation(options: TruncateOptions): Promise<void>;
}Data Encapsulation and ImmutabilityTo ensure proper encapsulation and prevent unintended modifications to internal state:
// Implementation
private apiConversationHistory: Anthropic.MessageParam[] = [];
private clineMessages: ClineMessage[] = [];
// Getter methods
getClineMessages(): ReadonlyArray<ClineMessage> {
return this.clineMessages;
}
getApiConversationHistory(): ReadonlyArray<Anthropic.MessageParam> {
return this.apiConversationHistory;
}
async forkFromMessage(timestamp: number): Promise<{
clineMessages: ReadonlyArray<ClineMessage>;
apiConversationHistory: ReadonlyArray<Anthropic.MessageParam>;
}> {
// Perform forking logic...
// Return deep copies as readonly arrays
return {
clineMessages: [...forkedClineMessages] as ReadonlyArray<ClineMessage>,
apiConversationHistory: [...forkedApiHistory] as ReadonlyArray<Anthropic.MessageParam>
};
}
async overwriteClineMessages(newMessages: ClineMessage[]): Promise<void> {
// Create a defensive copy
this.clineMessages = [...newMessages];
await this.saveClineMessages();
}Synchronization Logic (Phase 2)The key to successful synchronization is understanding the mapping between message types. Here's how the transformation logic would work: // Converting ClineMessage to Anthropic.MessageParam
convertToApiMessage(clineMessage: ClineMessage): Anthropic.MessageParam {
// Determine role based on message type
const role = clineMessage.type === "say" ? "assistant" : "user";
// Convert content based on message type and format
let content: string | Anthropic.Messages.ContentBlockParam[];
if (clineMessage.images && clineMessage.images.length > 0) {
// Handle messages with images
content = [
{ type: "text", text: clineMessage.text || "" },
...clineMessage.images.map(img => ({ type: "image", source: { type: "base64", data: img } }))
];
} else {
// Text-only messages
content = clineMessage.text || "";
}
return { role, content };
}
// Converting Anthropic.MessageParam to ClineMessage
convertToClineMessage(apiMessage: Anthropic.MessageParam): ClineMessage {
const ts = Date.now();
const type = apiMessage.role === "assistant" ? "say" : "ask";
// Extract text and images from content
let text: string | undefined;
let images: string[] | undefined;
if (typeof apiMessage.content === "string") {
text = apiMessage.content;
} else if (Array.isArray(apiMessage.content)) {
// Extract text blocks
const textBlocks = apiMessage.content.filter(block => block.type === "text");
text = textBlocks.map(block => (block as any).text).join("\n");
// Extract image blocks
const imageBlocks = apiMessage.content.filter(block => block.type === "image");
if (imageBlocks.length > 0) {
images = imageBlocks.map(block => (block as any).source.data);
}
}
return {
ts,
type,
say: type === "say" ? "text" : undefined,
ask: type === "ask" ? "followup" : undefined,
text,
images,
};
}Error HandlingThe implementation will maintain the current error handling approach for file operations: async saveApiConversationHistory(): Promise<void> {
try {
const filePath = path.join(await this.ensureTaskDirectoryExists(), GlobalFileNames.apiConversationHistory);
await fs.writeFile(filePath, JSON.stringify(this.apiConversationHistory));
} catch (error) {
// Log error but don't stop task execution
console.error("Failed to save API conversation history:", error);
}
}Transaction SafetyFor operations that modify both lists, implement transaction-like patterns: async deleteMessage(timestamp: number, deleteSubsequent: boolean): Promise<void> {
// Create copies of the current state for rollback if needed
const originalClineMessages = [...this.clineMessages];
const originalApiHistory = [...this.apiConversationHistory];
try {
// Perform the deletion operation on both lists
// ...
// Save both lists
await this.saveContext();
} catch (error) {
// Rollback to original state
this.clineMessages = originalClineMessages;
this.apiConversationHistory = originalApiHistory;
throw error;
}
}Backward CompatibilityEnsure the new implementation can read task data saved by the old implementation: async loadContext(): Promise<void> {
// Load clineMessages
this.clineMessages = await this.getSavedClineMessages();
// Load apiConversationHistory
this.apiConversationHistory = await this.getSavedApiConversationHistory();
// Check for old format and convert if necessary
if (this.needsFormatConversion()) {
await this.convertFromOldFormat();
}
}Migration StrategyThe migration will be implemented in two phases to reduce risk:
Implications and Considerations
Next Steps
|
Beta Was this translation helpful? Give feedback.
-
|
The current gemini's official context window is too big, is it possible to set it manually, it affects model response.Of course I know it's possible to set it with openai compatible format but it's too much of a hassle. |
Beta Was this translation helpful? Give feedback.
-
|
could using AST as intermediate be useful ? like in http://x.com/itechnologynet/status/1908450412119597326 thanks |
Beta Was this translation helpful? Give feedback.
-
|
i think long context is the future . recently meta-corp's LLM released with 10M context window |
Beta Was this translation helpful? Give feedback.
-
To run 10m context they need 10xh100, which is not efficient for 0.2 per million, and llama maverick is low quality as fuck. |
Beta Was this translation helpful? Give feedback.
-
|
if we talk about efficiency , why are we even using floats ,instead of fixed-point math. on integers ,which are natural for binary-compluters ,instead of bruteforcing decimal numbers on binary computers . isnt it ? |
Beta Was this translation helpful? Give feedback.
-
|
Thanks for all the insightful ideas in this thread! I’d like to propose an extension that goes beyond deduplication and prioritization, aiming for more cost-effective and intelligent context management, especially for users with a limited budget who cannot afford frequent, large-context API calls. Proposal: Active Summarization & Embedding-Based Context CompressionCore Goal: Key Points
Why This Matters
How This Complements Existing Ideas
Would love to hear your thoughts! If there’s interest, I’m happy to help elaborate on the technical approach. |
Beta Was this translation helpful? Give feedback.
-
|
Also check on Task Master and "memory bank" and their graph equivalents, maybe context can be more modular, while not losing relational context? |
Beta Was this translation helpful? Give feedback.
-
|
Sub-task scheduling would also be benificial to this feature as I discussed in #1574 (comment) |
Beta Was this translation helpful? Give feedback.
-
|
Interesting idea of prompt compression/optimization to reduce tokens used. |
Beta Was this translation helpful? Give feedback.
-
|
This project looks amazing！ |
Beta Was this translation helpful? Give feedback.
-
|
But it is strange that this project has no issue |
Beta Was this translation helpful? Give feedback.
-
|
whats "amazing" about it ? its just structured-text . any sensible technical person would do this . |
Beta Was this translation helpful? Give feedback.
-
|
Anything that eliminates redundancy and increases clarity is good |
Beta Was this translation helpful? Give feedback.
-
|
SynthLang looks incredible. I'd love to have this implemented in Roo Code! This would save so many tokens with each request 🤤 |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
-
IIUC, the current logic garbage-collects the first-half of the conversation (except for the original task content) upon hitting ~80% model context length. A few optimizations to consider:
I would recommend doing one or the other as a "free" optimization, especially as the original task + context can be quite bulky in terms of tokens.
Having multiple versions of the same file in context:
When context is being garbage-collected, I would recommend:
Beta Was this translation helpful? Give feedback.
All reactions