Add support for storing, maintaining user memories in the backend #1168
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
the user memory is comprised of the user to whom it belongs, the generated embeddings, the raw text field, and the search model used to create the embeddings remove irrelevant datastore object
adapters provide various methods for creating, updating, deleting the user memory objects
add logic to automatically create new memories and delete existing memories when saving conversation to log. the logic prevents the agent from editing existing memories, as a create + delete can suffice for this case and reduces the complexity.
add backend APIs exposing basic CRUD capabilities on memories
add a section to the settings page that allows you to browse, view, edit existing memories the backend has about you
There was a problem hiding this comment.
Pull Request Overview
This PR introduces support for storing and maintaining user memories in the backend as part of Khoj’s user adaptation improvements. Key changes include:
- Adding a new UserMemory model (including a migration to delete the old DataStore model), along with corresponding admin registration and adapter methods.
- Creating new API endpoints to retrieve, update, and delete memories.
- Integrating memory support into conversation processing and updating the settings page UI to let users view and manage their memories.
Reviewed Changes
Copilot reviewed 12 out of 12 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| src/khoj/routers/helpers.py | Added functions for extracting facts and updating memories, with proper user validation. |
| src/khoj/routers/api_memories.py | New API endpoints for fetching, updating, and deleting user memories. |
| src/khoj/routers/api_chat.py | Integrated memory retrieval and de-duplication into chat telemetry generation. |
| src/khoj/processor/conversation/utils.py | Updated conversation log saving to include memory updates. |
| src/khoj/database/models/init.py | Introduced the new UserMemory model and removed obsolete DataStore model. |
| src/khoj/database/adapters/init.py | Added a UserMemoryAdapters class with functions for pulling, saving, searching, and deleting memories. |
| src/configure.py | Included the new api_memories router in the application configuration. |
| src/interface/web/app/settings/page.tsx | Enhanced the settings page with UI for fetching, updating, and deleting memories. |
| src/interface/web/app/components/userMemory/userMemory.tsx | Added a new UI component to display and manage individual user memories. |
Comments suppressed due to low confidence (2)
src/khoj/routers/api_chat.py:1486
- The code creates a set from UserMemory objects via 'set(recent_memories + long_term_memories)' which may raise issues if these model instances lack a defined hash function. Consider using a unique identifier (such as memory.id) or converting the objects appropriately before de-duplication.
matching_memories: List[UserMemory] = [],
src/khoj/database/adapters/init.py:2057
- The parameter 'memory_id' is declared as a string, but the primary key for UserMemory is likely an integer. Consider updating the type to int or ensuring consistency with the model definition.
async def delete_memory(user: KhojUser, memory_id: str) -> bool:
wire up the relevant_memories data to helper functions as needed. e.g., image generation, online search, extract webpage content, run code, reserach mode
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Memory
A major component of useful AI systems is adaptation to the user context. This is a major reason why we've prioritized uploading of knowledge bases. The next steps in this direction is dynamically updating state of the user as more conversations take place.
We introduce a new paradigm for maintaining medium and long term memories in Khoj. The scope of a conversation can be thought of as short term memory.
Medium term memory extends to the past week.
Long term memory extends to anytime in the past, where a search query results in a match.
Add support for viewing and managing the memories with user action in the settings page as well.
Fully integrate the memory object into all downstream usage, from image generation, notes extraction, online search, etc.