Skip to content

Content sync enhancement #1189

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 110 commits into
base: master
Choose a base branch
from

Conversation

hjamet
Copy link
Contributor

@hjamet hjamet commented May 31, 2025

This pull request fixes a major synchronization bug in the Obsidian plugin with the Khoj backend.
Previously, if more than 1000 files were synchronized simultaneously, it would cause an error that interrupted the process, making it impossible to synchronize large vaults from scratch.

This pull request also adds console logging to track the progress of synchronization with the backend, which can sometimes be considerably slow and might worry the user.

debanjum and others added 29 commits May 25, 2025 03:27
Upgrading standard dompurify package breaks the khoj obsidian plugin.
Isomorphic dompurify wraps the latest dompurify 3.2.6, works with
the khoj obsidian plugin and is recommended on the dompurify npm
package as well. So migrate to that
Ask khoj to return file edits in the SEARCH/REPLACE form.
This is a more standard form, also used by aider.

Remove json5 package as we now rely on regex to pars edit blocks from
khoj's response

Remove unused functions in chat_view.
Can re-nable if invalid edit blocks higher than tolerable and clean-up
chat history clutter after edit block fixup retries
These should be used sparingly and in UX where icons do not work like
error messages and notices.
- Show user profile picture or email initial if signed in
- Show connection status as status dot
- Add link to get khoj api key from khoj url
- Add interact, connect section to improve setting groups
- Switch between similar and chat view in the same leaf node when
  opened via the header pane nav links
- Only show new conversation when in chat view, not similar view
- Put new conversation container after header row. Previously it would
  flex around in the header row which felt untidy.
- Introduced a progress bar for better user feedback during file uploads.
- Improved logging for file synchronization, including success and error messages.
- Added error handling for network issues and server responses, with retry logic for failed uploads.
- Optimized batch processing to respect server limits and handle file deletions more effectively.
- Updated console messages for clarity and user engagement.
Copy link

gitguardian bot commented May 31, 2025

⚠️ GitGuardian has uncovered 1 secret following the scan of your pull request.

Please consider investigating the findings and remediating the incidents. Failure to do so may lead to compromising the associated services or software components.

Since your pull request originates from a forked repository, GitGuardian is not able to associate the secrets uncovered with secret incidents on your GitGuardian dashboard.
Skipping this check run and merging your pull request will create secret incidents on your GitGuardian dashboard.

🔎 Detected hardcoded secret in your pull request
GitGuardian id GitGuardian status Secret Commit Filename
15894534 Triggered Generic High Entropy Secret 6f21860 docker-compose.yml View secret
🛠 Guidelines to remediate hardcoded secrets
  1. Understand the implications of revoking this secret by investigating where it is used in your code.
  2. Replace and store your secret safely. Learn here the best practices.
  3. Revoke and rotate this secret.
  4. If possible, rewrite git history. Rewriting git history is not a trivial act. You might completely break other contributing developers' workflow and you risk accidentally deleting legitimate data.

To avoid such incidents in the future consider


🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants