Backend work for public collections: thumbnails, url list, upload pages, and so on#2198

Merged

tw4l merged 57 commits intopublic-collections-featurefrom

issue-2182-thumbnail-backend

Dec 23, 2024

Commits on Dec 17, 2024

Add list endpoint to get sorted list of URLs in collection

tw4l
authored and
SuaYoo
committed
Add endpoint to set or update collection home url

tw4l
authored and
SuaYoo
committed
Fixups

tw4l
authored and
SuaYoo
committed
Use updated response for /home-urls endpoint

tw4l
authored and
SuaYoo
committed
Add tests

tw4l
authored and
SuaYoo
committed
WIP: Add collection thumbnail upload

tw4l
authored and
SuaYoo
committed
Add collection thumbnail upload test

tw4l
authored and
SuaYoo
committed
WIP: Only share subset of coll and image data for public endpoint

tw4l
authored and
SuaYoo
committed
Fix test that was checking wrong collection

tw4l
authored and
SuaYoo
committed
Change path for thumbnail endpoint

tw4l
authored and
SuaYoo
committed
Add endpoint to delete collection thumbnail

tw4l
authored and
SuaYoo
committed
Add caption field to collections

tw4l
authored and
SuaYoo
committed
Calculate and store earliest and latest dates in collection

tw4l
authored and
SuaYoo
committed
Add comment to get CI to run

tw4l
authored and
SuaYoo
committed
Fix test

tw4l
authored and
SuaYoo
committed
Add pages to database for uploads

tw4l
authored and
SuaYoo
committed
WIP: Add all-crawls/ and uploads/ versions of GET pages endpoints

tw4l
authored and
SuaYoo
committed
Add fallbacks if pages have no id or a non-UUID id

tw4l
authored and
SuaYoo
committed
Move pages test later to give it more time

tw4l
authored and
SuaYoo
committed
Delete upload pages when deleted or replaced

tw4l
authored and
SuaYoo
committed
Remove asserts for upload pages for optional fields

tw4l
authored and
SuaYoo
committed
Remove outdated comment

tw4l
authored and
SuaYoo
committed
Filter (re-)add all pages endpoint by crawl type in path

tw4l
authored and
SuaYoo
committed
Move re-adding all pages in org to background job

tw4l
authored and
SuaYoo
committed
Add migration to add upload files for all orgs in bg jobs

tw4l
authored and
SuaYoo
committed
Add API endpoint to GET single public collection

tw4l
authored and
SuaYoo
committed
Recalculate collection dates after adding upload pages

tw4l
authored and
SuaYoo
committed
Rename type_filter to crawl_type

tw4l
authored and
SuaYoo
committed
Recalculate collection stats after adding upload pages

tw4l
authored and
SuaYoo
committed
Reduce per-page print logging

tw4l
authored and
SuaYoo
committed
Include upload pages in collection pageCount

tw4l
authored and
SuaYoo
committed
Return 404 for GET public collection if org isn't public

tw4l
authored and
SuaYoo
committed
Add oid to PublicCollOut model

tw4l
authored and
SuaYoo
committed
Modify public collections endpoint paths

tw4l
authored and
SuaYoo
committed
Fix endpoint path in test

tw4l
authored and
SuaYoo
committed
Only fetch org once, not per-crawl

tw4l
authored and
SuaYoo
committed
Add counts and size to PublicCollOut

tw4l
authored and
SuaYoo
committed
Enforce max thumbnail file size of 2MB

tw4l
authored and
SuaYoo
committed
Use proper thumbnail as test data

tw4l
authored and
SuaYoo
committed
Allow unlisted collections in single public collection GET

tw4l
authored and
SuaYoo
committed
Add access field to PublicCollOut

tw4l
authored and
SuaYoo
committed
Don't expect thumbnail for unlisted collection

tw4l
authored and
SuaYoo
committed
Fix thumbnail-related 500 error with private collections list

tw4l
authored and
SuaYoo
committed
Add defaultThumbnailName field to collections

tw4l
authored and
SuaYoo
committed
Add fallback values for access in coll out models

tw4l
authored and
SuaYoo
committed
Add ability to unset home url via /home-url endpoint

tw4l
authored and
SuaYoo
committed
Add public collection download endpoint

tw4l
authored and
SuaYoo
committed
Implement and enforce Collection.allowPublicDownload

tw4l
authored and
SuaYoo
committed
Fix linting

tw4l
authored and
SuaYoo
committed
Make sure coll out models return allowPublicDownload as bool

tw4l
authored and
SuaYoo
committed
Make migration idempotent - don't readd existing upload pages
tw4l
committed
Reformat migration
tw4l
committed
Fix typing
tw4l
committed