ci: merge main to release #8994
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* fix: paginate search results correctly * style: magic number -> named constant
* test: Add test for api.ietf_utils * test: Use ietf.utils.test_utils instead of django.test * test: Add test case for request without a token * test: Test for misconfigured endpoint * test: Improve tests * test: Add test for a API call without X_API_KEY header
* refactor: use new meeting.Registration model for OidcExtraScopeClaims * refactor: use new meeting.Registration model for nomcom eligibility calculations * refactor: use new meeting.Registration model for various meeting views * fix: revert to on_delete=PROTECT * refactor: update new APIs and corresponding tests * fix: tweaks for clarity * fix: fix assert syntax * fix: fix minor issue with MeetingRegistrtion migration * fix: add missing meeting.tasks.fetch_meeting_attendance_task (#8987) --------- Co-authored-by: Robert Sparks <rjsparks@nostrum.com>
* feat: blobdb app * feat: blob model * fix: app name * feat: admin for blob model * feat: database router * feat: BlobdbStorage storage class * refactor: storage metadata via File subclass * refactor: eliminate in_flight_custom_metadata * chore: raise blobstore exceptions in development mode * feat: StagedBlobStore + StorageObjectStorageMixin * feat: StoredObject.committed timestamp * refactor: some refactoring (wip) * refactor: better separation of StorageObject and Storage Partial revert of 5d6bd49 * feat: basic 2-blobstore commit * fix: StorageObject should have been StoredObject * fix: propagate metadata to final_storage * feat: easier config of StagedBlobStorage * refactor: save-specific commit * chore(dev): add dev database for blob storage * chore(dev): migrate blobdb in app-init.sh * chore(dev): use staged blob storages for dev env * feat: deletion for StagedBlobStorage * refactor: single-query update of committed field * fix: better commit behavior for celery Make async commit optional so that celery tasks don't spawn other celery tasks. This is a definite fix. Reverses order of StoredObject.create() and storage.save(). This avoids a race condition between an async commit and the creation of a StoredObject, but need to review whether this introduces other problems. * style: black + copyright stmts * fix: dont test for v1 apis into the blobdb app models * chore: fix storage configs (esp for tests) (#8664) * chore: remove empty blobdb.views * feat: mtime/content-type for blobdb * test: test BlobdbStorage * refactor: split storage utils from Storage class This is a rather consequential refactor. Moves the utility methods (store_file(), remove_from_storage(), etc) out of the Storage subclass. They are now only in storage_utils.py. Drops the StoredObjectStorageMixin, instead moving the Storage-like parts directly into StoredObjectStagedBlobStorage. The CustomS3Storage class is dropped, at least for now. * fix: circular import * chore: CRLFs in ISSUE_TEMPLATE/config.yml * chore: add resources.py * refactor: extract metadata from BlobFile * refactor: reconcile dueling MetadataFiles * refactor: plumb metadata and reorg code * fix: handle content type properly * chore: remove blobdb/resources.py * test: ietf.blobdb->OMITTED_APPS; drop ad hoc ignore * chore: copyright * chore: Remove CR * refactor: storage_backends.py -> storage.py Matches standard naming convention * refactor: remove StagedBlobStorage * chore: fix mypy lint * chore: remove StagedBlobStorage tasks * refactor: committed -> replicated * feat: admin improvements * chore: configure storages for dev * configure separate replica storage * enable retries * switch to "standard" mode instead of legacy * feat: copy blobs to "replica" storage when stored * feat: propagate blob deletion, too * fix: reset replicated flag on delete * chore: remove replication * refactor: unify artifact storage configs All environments now use the same storage config, so set it up directly in settings.py instead of scattered throughout. * fix: mypy lint * fix: more mypy lint... * fix: overwrite, don't rename, in BlobdbStorage * fix: more lint * feat: hooks for Blobdb watcher notifications * feat: pybob the blob replicator Simulation of an external replicator task. * chore: fix quoting in celery docker-init.sh * fix: adjust replicator/storage to work together * chore: concurrency=1 worker for blobdb queue * feat: blobdb replication module + settings * feat: en/disable replication per settings * feat: per-bucket replication en/disable * fix: use correct type for EXCLUDE_BUCKETS * style: reorder methods * feat: logging/exceptions in replication.py * feat: retry replication * fix: valid storage pattern setting * chore: consistent naming * feat: detect missing replica storage configs * fix: ensure mtime is set for the replica Use blob modified timestamp as mtime in replicated metadata. This is the intended behavior and matches BlobdbStorage. * fix: log message grammar * feat: blobdb + k8s deployment (#8946) * chore(dev): remove assert I like the idea, but settings are sometimes loaded more than once so this approach doesn't work * chore: blobdb migrations in k8s * chore: fix typo in commet * chore: add replicator pod for k8s deploy * chore: drop unnecessary ports from celery/beat * chore: fix tyop * chore: settings for blobdb / replication * chore: put quotes back around $@ * ci: typo in k8s/settings_local.py * chore: additional blobdb replicator logging (#8950) * refactor: minor refactor + ruff style * chore: additional logging * fix: db transactions for blobdb save/delete (#8951) * fix: use correct DB for transaction (#8952) * fix: use correct DB for transaction * chore: stale comment * fix: on_commit needs "using" also (#8953) * chore: unused imports in settings_locals * refactor: typing-friendly blob handling in replication.py * chore: more lint * chore: fix lint affecting GHA tests --------- Co-authored-by: Robert Sparks <rjsparks@nostrum.com>
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.