Feature Request: Improve management of mode transitions #545
nissa-seru
started this conversation in
Feature Requests
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
IIUC, current functionality is that when the user switches the mode of a model mid-task (or approves a model's request to switch modes), the model's system prompt changes accordingly but the entire current context is retained.
I strongly recommend garbage-collecting context when this occurs, as cache is broken. See #544 for recommended augmentations to the current garbage-collection functionality, but simply triggering the current functionality would be a medium win by itself. Tailored logic for this case may be incrementally valuable as well - ie it may be superior for the model to auto-generate a summary of the implementation plan and then start a new thread or more aggressively garbage-collect (ie more than half the message history) to avoid garbage-collection occurring relatively briefly after starting implementation.
There is a second issue here, though, that is likely more critical. When a task is the first message in a thread, it is protected from the context-garbage-collection logic. However, when a mode switch occurs mid-thread, that typically indicates that the user+model have crafted an implementation plan, and then the user is switching the model's mode for execution. In this case, the implementation plan is not protected from garbage-collection (!!). Hence, once the implementation plan trickles upwards into the first-half of message pairs (which is garbage-collected), it is presently removed, and the model is left to continue implementation without the benefit of the plan. IME, this results in the model getting stuck in a very narrow debug loop.
Beta Was this translation helpful? Give feedback.
All reactions