add support for hidden collections parameter for applying auth (#874)

* add support for hidden collections parameter for applying auth

* add ENABLE_COLLECTIONS_AUTHX

Author: philvarner

CHANGELOG.md

@@ -18,6 +18,12 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
 
 - Use Node 22 by default.
 
+## Added
+
+- To all endpoints that depend on collections, add support for a query parameter (GET)
+  or body field (POST) `_collections` that will filter to only those collections, but
+  will not reveal that in link contents. This is controlled by the "ENABLE_COLLECTIONS_AUTHX"
+
 ## [3.11.0] - 2025-03-27
 
 ### Added

README.md

@@ -12,6 +12,7 @@
     - [4.0.0](#400)
       - [Context Extension disabled by default](#context-extension-disabled-by-default)
       - [Node 22 update](#node-22-update)
+      - [Hidden collections filter](#hidden-collections-filter)
     - [3.10.0](#3100)
       - [Node 20 update](#node-20-update)
     - [3.1.0](#310)
@@ -50,6 +51,7 @@
     - [Filter Extension](#filter-extension)
     - [Query Extension](#query-extension)
   - [Aggregation](#aggregation)
+  - [Hidden collections filter for authorization](#hidden-collections-filter-for-authorization)
   - [Ingesting Data](#ingesting-data)
     - [Ingesting large items](#ingesting-large-items)
     - [Subscribing to SNS Topics](#subscribing-to-sns-topics)
@@ -183,6 +185,13 @@ The default Lambda deployment environment is now Node 22.
 To update the deployment to use Node 22, modify the serverless config file value
 `provider.runtime` to be `nodejs22.x` and the application re-deployed.
 
+#### Hidden collections filter
+
+To all endpoints that depend on collections, there is now support for a query parameter
+(GET) or body field (POST) `_collections` that will filter to only those collections, but
+will not reveal that in link contents. This is useful for the application of permissions
+to only certain collections.
+
 ### 3.10.0
 
 #### Node 20 update
@@ -574,7 +583,7 @@ There are some settings that should be reviewed and updated as needeed in the se
 | REQUEST_LOGGING_FORMAT           | Express request logging format to use. Any of the [Morgan predefined formats](https://github.com/expressjs/morgan#predefined-formats).                                                                                                                                          | tiny                                                                                 |
 | STAC_API_URL                     | The root endpoint of this API                                                                                                                                                                                                                                                   | Inferred from request                                                                |
 | ENABLE_TRANSACTIONS_EXTENSION    | Boolean specifying if the [Transaction Extension](https://github.com/radiantearth/stac-api-spec/tree/master/ogcapi-features/extensions/transaction) should be activated                                                                                                         | false                                                                                |
-| ENABLE_CONTEXT_EXTENSION    | Boolean specifying if the [Context Extension](https://github.com/stac-api-extensions/context) should be activated                                                                                                         | false                                                                                |
+| ENABLE_CONTEXT_EXTENSION         | Boolean specifying if the [Context Extension](https://github.com/stac-api-extensions/context) should be activated                                                                                                                                                               | false                                                                                |
 | STAC_API_ROOTPATH                | The path to append to URLs if this is not deployed at the server root. For example, if the server is deployed without a custom domain name, it will have the stage name (e.g., dev) in the path.                                                                                | ""                                                                                   |
 | PRE_HOOK                         | The name of a Lambda function to be called as the pre-hook.                                                                                                                                                                                                                     | none                                                                                 |
 | POST_HOOK                        | The name of a Lambda function to be called as the post-hook.                                                                                                                                                                                                                    | none                                                                                 |
@@ -589,6 +598,7 @@ There are some settings that should be reviewed and updated as needeed in the se
 | CORS_CREDENTIALS                 | Configure whether or not to send the `Access-Control-Allow-Credentials` CORS header. Header will be sent if set to `true`.                                                                                                                                                      | none                                                                                 |
 | CORS_METHODS                     | Configure whether or not to send the `Access-Control-Allow-Methods` CORS header. Expects a comma-delimited string, e.g., `GET,PUT,POST`.                                                                                                                                        | `GET,HEAD,PUT,PATCH,POST,DELETE`                                                     |
 | CORS_HEADERS                     | Configure whether or not to send the `Access-Control-Allow-Headers` CORS header. Expects a comma-delimited string, e.g., `Content-Type,Authorization`. If not specified, defaults to reflecting the headers specified in the request’s `Access-Control-Request-Headers` header. | none                                                                                 |
+| ENABLE_COLLECTIONS_AUTHX         | Enables support for hidden `_collections` query parameter / field when set to `true`.                                                                                                                                                                                                              | none                                                                                 |
 
 Additionally, the credential for OpenSearch must be configured, as decribed in the
 section [Populating and accessing credentials](#populating-and-accessing-credentials).
@@ -1093,6 +1103,32 @@ Available aggregations are:
 - geometry_geohash_grid_frequency ([geohash grid](https://opensearch.org/docs/latest/aggregations/bucket/geohash-grid/) on Item.geometry)
 - geometry_geotile_grid_frequency ([geotile grid](https://opensearch.org/docs/latest/aggregations/bucket/geotile-grid/) on Item.geometry)
 
+## Hidden collections filter for authorization
+
+All endpoints that involve the use of Collections support the use of a "hidden" query
+parameter named  (for GET requests) or body JSON field (for POST requests) named
+`_collections` that can be used by an authorization proxy (e.g., a pre-hook Lambda)
+to filter the collections a user has access to. This parameter/field will be excluded
+from pagination links, so it does not need to be removed on egress.
+
+This feature must be enabled with the `ENABLE_COLLECTIONS_AUTHX` configuration.
+
+The endpoints this applies to are:
+
+- /collections
+- /collections/:collectionId
+- /collections/:collectionId/queryables
+- /collections/:collectionId/aggregations
+- /collections/:collectionId/aggregate
+- /collections/:collectionId/items
+- /collections/:collectionId/items/:itemId
+- /collections/:collectionId/items/:itemId/thumbnail
+- /search
+- /aggregate
+
+The five endpoints of the Transaction Extension do not use this parameter, as there are
+other authorization considerations for these, that are left as future work.
+
 ## Ingesting Data
 
 STAC Collections and Items are ingested by the `ingest` Lambda function, however this Lambda is not invoked directly by a user, it consumes records from the `stac-server-<stage>-queue` SQS. To add STAC Items or Collections to the queue, publish them to the SNS Topic `stac-server-<stage>-ingest`.

package.json

@@ -25,7 +25,7 @@
     "deploy": "sls deploy",
     "sls-remove": "sls remove",
     "package": "sls package",
-    "serve": "REQUEST_LOGGING_FORMAT=dev LOG_LEVEL=debug STAC_API_URL=http://localhost:3000 ENABLE_TRANSACTIONS_EXTENSION=true nodemon --esm ./src/lambdas/api/local.ts",
+    "serve": "REQUEST_LOGGING_FORMAT=dev LOG_LEVEL=debug STAC_API_URL=http://localhost:3000 ENABLE_TRANSACTIONS_EXTENSION=true nodemon --exec node --loader ts-node/esm ./src/lambdas/api/local.ts",
     "build-api-docs": "npx @redocly/cli build-docs src/lambdas/api/openapi.yaml -o ./docs/index.html",
     "prepare": "husky"
   },

src/lambdas/api/app.js

@@ -6,7 +6,7 @@ import path from 'path'
 import { fileURLToPath } from 'url'
 import database from '../../lib/database.js'
 import api from '../../lib/api.js'
-import { ValidationError } from '../../lib/errors.js'
+import { NotFoundError, ValidationError } from '../../lib/errors.js'
 import { readFile } from '../../lib/fs.js'
 import addEndpoint from './middleware/add-endpoint.js'
 import logger from '../../lib/logger.js'
@@ -153,7 +153,7 @@ app.get('/aggregations', async (req, res, next) => {
 
 app.get('/collections', async (req, res, next) => {
   try {
-    const response = await api.getCollections(database, req.endpoint)
+    const response = await api.getCollections(database, req.endpoint, req.query)
     if (response instanceof Error) next(createError(500, response.message))
     else res.json(response)
   } catch (error) {
@@ -185,7 +185,7 @@ app.post('/collections', async (req, res, next) => {
 app.get('/collections/:collectionId', async (req, res, next) => {
   const { collectionId } = req.params
   try {
-    const response = await api.getCollection(collectionId, database, req.endpoint)
+    const response = await api.getCollection(collectionId, database, req.endpoint, req.query)
     if (response instanceof Error) next(createError(404))
     else res.json(response)
   } catch (error) {
@@ -196,7 +196,9 @@ app.get('/collections/:collectionId', async (req, res, next) => {
 app.get('/collections/:collectionId/queryables', async (req, res, next) => {
   const { collectionId } = req.params
   try {
-    const queryables = await api.getCollectionQueryables(collectionId, database, req.endpoint)
+    const queryables = await api.getCollectionQueryables(
+      collectionId, database, req.endpoint, req.query
+    )
 
     if (queryables instanceof Error) next(createError(404))
     else {
@@ -215,7 +217,9 @@ app.get('/collections/:collectionId/queryables', async (req, res, next) => {
 app.get('/collections/:collectionId/aggregations', async (req, res, next) => {
   const { collectionId } = req.params
   try {
-    const aggs = await api.getCollectionAggregations(collectionId, database, req.endpoint)
+    const aggs = await api.getCollectionAggregations(
+      collectionId, database, req.endpoint, req.query
+    )
     if (aggs instanceof Error) next(createError(404))
     else res.json(aggs)
   } catch (error) {
@@ -231,7 +235,9 @@ app.get('/collections/:collectionId/aggregate',
   async (req, res, next) => {
     const { collectionId } = req.params
     try {
-      const response = await api.getCollection(collectionId, database, req.endpoint)
+      const response = await api.getCollection(
+        collectionId, database, req.endpoint, req.query
+      )
 
       if (response instanceof Error) next(createError(404))
       else {
@@ -249,20 +255,22 @@ app.get('/collections/:collectionId/aggregate',
 app.get('/collections/:collectionId/items', async (req, res, next) => {
   const { collectionId } = req.params
   try {
-    const response = await api.getCollection(collectionId, database, req.endpoint)
+    if ((await api.getCollection(
+      collectionId, database, req.endpoint, req.query
+    )) instanceof Error) {
+      next(createError(404))
+    }
 
-    if (response instanceof Error) next(createError(404))
-    else {
-      const items = await api.searchItems(
+    res.type('application/geo+json')
+    res.json(
+      await api.searchItems(
         collectionId,
         req.query,
         database,
         req.endpoint,
         'GET'
       )
-      res.type('application/geo+json')
-      res.json(items)
-    }
+    )
   } catch (error) {
     if (error instanceof ValidationError) {
       next(createError(400, error.message))
@@ -280,7 +288,7 @@ app.post('/collections/:collectionId/items', async (req, res, next) => {
     if (req.body.collection && req.body.collection !== collectionId) {
       next(createError(400, 'Collection resource URI must match collection in body'))
     } else {
-      const collectionRes = await api.getCollection(collectionId, database, req.endpoint)
+      const collectionRes = await api.getCollection(collectionId, database, req.endpoint, req.query)
       if (collectionRes instanceof Error) next(createError(404))
       else {
         try {
@@ -312,15 +320,14 @@ app.get('/collections/:collectionId/items/:itemId', async (req, res, next) => {
       collectionId,
       itemId,
       database,
-      req.endpoint
+      req.endpoint,
+      req.query
     )
 
-    if (response instanceof Error) {
-      if (response.message === 'Item not found') {
-        next(createError(404))
-      } else {
-        next(createError(500))
-      }
+    if (response instanceof NotFoundError) {
+      next(createError(404))
+    } else if (response instanceof Error) {
+      next(createError(500))
     } else {
       res.type('application/geo+json')
       res.json(response)
@@ -339,7 +346,7 @@ app.put('/collections/:collectionId/items/:itemId', async (req, res, next) => {
     } else if (req.body.id && req.body.id !== itemId) {
       next(createError(400, 'Item ID in resource URI must match id in body'))
     } else {
-      const itemRes = await api.getItem(collectionId, itemId, database, req.endpoint)
+      const itemRes = await api.getItem(collectionId, itemId, database, req.endpoint, req.query)
       if (itemRes instanceof Error) next(createError(404))
       else {
         req.body.collection = collectionId
@@ -372,7 +379,7 @@ app.patch('/collections/:collectionId/items/:itemId', async (req, res, next) =>
     } else if (req.body.id && req.body.id !== itemId) {
       next(createError(400, 'Item ID in resource URI must match id in body'))
     } else {
-      const itemRes = await api.getItem(collectionId, itemId, database, req.endpoint)
+      const itemRes = await api.getItem(collectionId, itemId, database, req.endpoint, req.query)
       if (itemRes instanceof Error) next(createError(404))
       else {
         try {
@@ -418,15 +425,13 @@ app.get('/collections/:collectionId/items/:itemId/thumbnail', async (req, res, n
       collectionId,
       itemId,
       database,
+      req.query
     )
 
-    if (response instanceof Error) {
-      if (response.message === 'Item not found'
-          || response.message === 'Thumbnail not found') {
-        next(createError(404))
-      } else {
-        next(createError(500))
-      }
+    if (response instanceof NotFoundError) {
+      next(createError(404))
+    } else if (response instanceof Error) {
+      next(createError(500))
     } else {
       res.redirect(response.location)
     }