Skip to content

Navigation Menu

Appearance settings

Explore
By company size
By use case
By industry
View all solutions
Topics
- AI
- DevOps
- Security
- Software Development
- View all
Explore
- GitHub Sponsors
  Fund open source developers
- The ReadME Project
  GitHub community articles
Repositories
- Enterprise platform
  AI-powered developer platform
Available add-ons
Pricing

Search code, repositories, users, issues, pull requests...

Search

Clear

Search syntax tips

Provide feedback

We read every piece of feedback, and take your input very seriously.

Include my email address so I can be contacted

Saved searches

Use saved searches to filter your results more quickly

Name

Query

To see all available qualifiers, see our documentation.

Appearance settings

You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session.

Dismiss alert

webrecorder / browsertrix Public

Notifications You must be signed in to change notification settings
Fork 52
Star 289

Code
Issues 212
Pull requests 10
Actions
Projects 1
Security
Insights

Additional navigation options

Code
Issues
Pull requests
Actions
Projects
Security
Insights

Implement custom storage for orgs#2093

Open

tw4l wants to merge 69 commits intomainwebrecorder/browsertrix:mainfrom

issue-578-custom-storagewebrecorder/browsertrix:issue-578-custom-storageCopy head branch name to clipboard

Conversation Commits69 (69)Checks Files changed

Open

Implement custom storage for orgs#2093
tw4l wants to merge 69 commits intomainfrom
issue-578-custom-storage

Commits

Commits on Apr 29, 2025

Add back custom storage endpoints
tw4l
committed
Flush out tests for setting custom storage
tw4l
committed
Fix test issue with bucket not existing for now
tw4l
committed
Add additional tests
tw4l
committed
Fix custom storage so it works as expected
tw4l
committed
Actually unset custom replica storage before deleting
tw4l
committed
Add TODO where custom storage deletion is failing
tw4l
committed
Fix check for whether storage label is in use
tw4l
committed
Remove todo on endpoint that's fine
tw4l
committed
Add todos re: tasks necessary to change storage
tw4l
committed
Check that no crawls are running before updating storage
tw4l
committed
Start adding post-storage update logic
tw4l
committed
WIP: Add background job to copy old s3 bucket to new
tw4l
committed
WIP: Start adding logic to handle replica location updates
tw4l
committed
Add additional note
tw4l
committed
Fix argument
tw4l
committed
Fix another argument
tw4l
committed
Fixups
tw4l
committed
Fix linting
tw4l
committed
More linting fixes
tw4l
committed
Refactor, seperate storage and replicas updates
tw4l
committed
More refactoring
tw4l
committed
Make post-update task methods private
tw4l
committed
Check if any bg jobs running before changing storage
tw4l
committed
Check bg job finished as well
tw4l
committed
Fixups
tw4l
committed
Storage update improvements
tw4l
committed
Fixup
tw4l
committed
Remove another todo
tw4l
committed
More fixups
tw4l
committed
Add provider to s3storage for rclone
tw4l
committed
Fix typo
tw4l
committed
Make API endpoints that change storage superuser-only for now
tw4l
committed
Add typing for init_storages_api, import Callable
tw4l
committed
Fix StorageOps in operator main
tw4l
committed
Always use oid prefix in s3 storage
tw4l
committed
Post-rebase fixups and remove create bucket fallback
tw4l
committed
Create extra test buckets in CI
tw4l
committed
Add test for non-verified custom storage
tw4l
committed
Refactor to move updates to FastAPI background tasks
tw4l
committed
Include default replicas in /storage response if no org replicas
tw4l
committed
Fix unsetting of presigned URLs
tw4l
committed
Add --progress flag to rclone copy command
tw4l
committed
Increase ttl seconds after finished for testing on dev
tw4l
committed
Ensure there are no double slashes between bucket name and oid
tw4l
committed
Increase memory limit/request for copy job to 500Mi
tw4l
committed
Reduce copy job ttlSecondsAfterFinished to 60
tw4l
committed
Add storage tag to API endpoints
tw4l
committed
Add flags to rclone to reduce memory usage, set limit to 350Mi
tw4l
committed
Fix positional operator in storage ref update
tw4l
committed
One more positional operator fix
tw4l
committed
Update docstrings and comments
tw4l
committed
Make all-storages response valid JSON with response model
tw4l
committed
Add admin docs for storage
tw4l
committed
Fix API endpoint path in docs example
tw4l
committed
Docs typo fix
tw4l
committed
Add provider field note
tw4l
committed
Docs language cleanup
tw4l
committed
Check /all-storages in backend tests
tw4l
committed
Add API endpoint for background job progress
tw4l
committed
Fix linting
tw4l
committed
Format post-rebase with Black
tw4l
committed
Format with Black
tw4l
committed
Fix linting
tw4l
committed
Fix linking
tw4l
committed
Cast job to CopyBucketJob
tw4l
committed
Fix type errors from rebase
tw4l
committed
Reformat
tw4l
committed
Fix linting
tw4l
committed

Footer

© 2025 GitHub, Inc.

Footer navigation

Terms
Privacy
Security
Status
Docs
Contact

You can’t perform that action at this time.