Proxy asset HREFs

CHANGELOG.md

@@ -5,6 +5,22 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/)
 and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.html).
 
+## [Unreleased]
+
+### Added
+
+- Asset proxying for generating pre-signed S3 URLs through proxy endpoints `GET
+  /collections/{collectionId}/items/{itemId}/assets/{assetKey}` and `GET
+  /collections/{collectionId}/assets/{assetKey}`.
+- Environment variables `ASSET_PROXY_BUCKET_OPTION`, `ASSET_PROXY_BUCKET_LIST`, and
+  `ASSET_PROXY_URL_EXPIRY` to configure asset proxying.
+
+### Changed
+
+- When asset proxying is enabled, when a STAC Item or Collection is served, asset S3 hrefs
+  are replaced with proxy endpoint URLs and the original S3 URLs are preserved in
+  `alternate.s3.href` using the Alternate Assets Extension.
+
 ## [4.4.0] - 2025-09-10
 
 ## Changed
@@ -579,8 +595,7 @@ Initial release, forked from [sat-api](https://github.com/sat-utils/sat-api/tree
 
 Compliant with STAC 0.9.0
 
-<!-- [unreleased]: https://github.com/stac-utils/stac-api/compare/v3.6.0...main -->
-
+[unreleased]: https://github.com/stac-utils/stac-server/compare/v4.4.0...main
 [4.4.0]: https://github.com/stac-utils/stac-api/compare/v4.3.0...v4.4.0
 [4.3.0]: https://github.com/stac-utils/stac-api/compare/v4.2.0...v4.3.0
 [4.2.0]: https://github.com/stac-utils/stac-api/compare/v4.1.0...v4.2.0

README.md

@@ -53,6 +53,7 @@
     - [Filter Extension](#filter-extension)
     - [Query Extension](#query-extension)
   - [Aggregation](#aggregation)
+  - [Asset Proxy](#asset-proxy)
   - [Collections and filter parameters for authorization](#collections-and-filter-parameters-for-authorization)
     - [Collections](#collections)
     - [CQL2 Filter](#cql2-filter)
@@ -617,6 +618,9 @@ There are some settings that should be reviewed and updated as needeed in the se
 | ENABLE_INGEST_ACTION_TRUNCATE    | Enables support for ingest action "truncate".                                                                                                                                                                                                                                   | none (not enabled)                                                                   |
 | ENABLE_RESPONSE_COMPRESSION      | Enables response compression. Set to 'false' to disable.                                                                                                                                                                                                                        | enabled                                                                              |
 | ITEMS_MAX_LIMIT                  | The maximum limit for the number of items returned from the /search and /collections/{collection_id}/items endpoints. It is recommended that this be set to 100. There is an absolute max limit of 10000 for this.                                                              | 10000                                                                                |
+| ASSET_PROXY_BUCKET_OPTION        | Control which S3 buckets are proxied through the API. Options: `NONE` (disabled), `ALL` (all S3 assets), `ALL_BUCKETS_IN_ACCOUNT` (all buckets in AWS account), `LIST` (specific buckets only).                                                                                 | NONE                                                                                 |
+| ASSET_PROXY_BUCKET_LIST          | Comma-separated list of S3 bucket names to proxy. Required when `ASSET_PROXY_BUCKET_OPTION` is `LIST`.                                                                                                                                                                          |                                                                                      |
+| ASSET_PROXY_URL_EXPIRY           | Pre-signed URL expiry time in seconds for proxied assets.                                                                                                                                                                                                                        | 300                                                                                  |
 
 Additionally, the credential for OpenSearch must be configured, as decribed in the
 section [Populating and accessing credentials](#populating-and-accessing-credentials).
@@ -1124,6 +1128,138 @@ Available aggregations are:
 - geometry_geohash_grid_frequency ([geohash grid](https://opensearch.org/docs/latest/aggregations/bucket/geohash-grid/) on Item.geometry)
 - geometry_geotile_grid_frequency ([geotile grid](https://opensearch.org/docs/latest/aggregations/bucket/geotile-grid/) on Item.geometry)
 
+## Asset Proxy
+
+The Asset Proxy feature enables stac-server to proxy access to S3 assets through the STAC
+API by generating pre-signed URLs. Only assets with S3 URIs (`s3://` prefix) are proxied;
+other URL schemes are ignored. When the Asset Proxy feature is enabled, asset `href`
+values pointing to S3 are replaced with proxy endpoint URLs when an Item or Collection is
+served, while the original S3 URLs are preserved in the `alternate.s3.href` field using
+the [Alternate Assets Extension](https://github.com/stac-extensions/alternate-assets).
+Subsequent GET requests to the proxy endpoint URLs are redirected to pre-signed S3 URLS
+for download. Note that the AWS account that stac-server is running under must have
+permission to access the S3 buckets containing the assets and that the stac-server AWS
+account will be charged for the S3 egress, regardless of whether the bucket is a
+"Requester Pays" bucket or not (the stac-server AWS account is the requester when
+generating the pre-signed URL).
+
+### Configuration
+
+Asset proxying uses three environment variables:
+
+- **`ASSET_PROXY_BUCKET_OPTION` -** Specifies one of four modes to control which S3 buckets are proxied.
+
+  - **NONE** (default): Asset proxy is disabled. All asset hrefs are returned unchanged.
+  - **ALL**: Proxy all S3 assets regardless of which bucket they are in.
+  - **ALL_BUCKETS_IN_ACCOUNT**: Proxy assets from any S3 bucket accessible to the AWS account credentials. The list of buckets is fetched at Lambda startup.
+  - **LIST**: Only proxy assets from specific buckets listed in `ASSET_PROXY_BUCKET_LIST`.
+
+- **`ASSET_PROXY_BUCKET_LIST`** — Comma-separated list of bucket names (required only when the `ASSET_PROXY_BUCKET_OPTION` environment variable is set to `LIST`)
+
+  ```yaml
+  ASSET_PROXY_BUCKET_OPTION: "LIST"
+  ASSET_PROXY_BUCKET_LIST: "my-bucket-1,my-bucket-2,my-bucket-3"
+  ```
+
+- **`ASSET_PROXY_URL_EXPIRY`** — Pre-signed URL expiry in seconds (default: `300`)
+
+### Endpoints
+
+When asset proxying is enabled, two endpoints are available for accessing proxied assets:
+
+- `GET /collections/{collectionId}/items/{itemId}/assets/{assetKey}` - Redirects (HTTP 302) to a pre-signed S3 URL for an item asset
+- `GET /collections/{collectionId}/assets/{assetKey}` - Redirects (HTTP 302) to a pre-signed S3 URL for a collection asset
+
+### IAM Permissions
+
+For the Asset Proxy feature to generate pre-signed URLs, the API and ingest Lambdas must be assigned permissions for the S3 buckets containing the assets. Add the following to the IAM role statements in your `serverless.yml` file, adjusting the resources as needed:
+
+For the `LIST` mode, you can specify the buckets listed in `ASSET_PROXY_BUCKET_LIST`:
+
+```yaml
+- Effect: Allow
+  Action:
+    - s3:GetObject
+  Resource:
+    - "arn:aws:s3:::my-bucket-1/*"
+    - "arn:aws:s3:::my-bucket-2/*"
+- Effect: Allow
+  Action:
+    - s3:HeadBucket
+  Resource:
+    - "arn:aws:s3:::my-bucket-1"
+    - "arn:aws:s3:::my-bucket-2"
+```
+
+For the `ALL` mode, use wildcards:
+
+```yaml
+- Effect: Allow
+  Action:
+    - s3:GetObject
+  Resource: "arn:aws:s3:::*/*"
+- Effect: Allow
+  Action:
+    - s3:HeadBucket
+  Resource: "arn:aws:s3:::*"
+```
+
+When using `ALL_BUCKETS_IN_ACCOUNT` mode, the Lambda also needs permission to list buckets:
+
+```yaml
+- Effect: Allow
+  Action:
+    - s3:GetObject
+  Resource: "arn:aws:s3:::*/*"
+- Effect: Allow
+  Action:
+    - s3:HeadBucket
+  Resource: "arn:aws:s3:::*"
+- Effect: Allow
+  Action:
+    - s3:ListAllMyBuckets
+  Resource: "*"
+```
+
+### Asset Transformation
+
+When asset proxying is enabled and an asset's `href` points to an S3 URL, the asset is transformed as follows:
+
+**Original asset:**
+```json
+{
+  "thumbnail": {
+    "href": "s3://my-bucket/path/to/thumbnail.png",
+    "type": "image/png",
+    "roles": ["thumbnail"]
+  }
+}
+```
+
+**Transformed asset:**
+```json
+{
+  "thumbnail": {
+    "href": "https://api.example.com/collections/my-collection/items/my-item/assets/thumbnail",
+    "type": "image/png",
+    "roles": ["thumbnail"],
+    "alternate": {
+      "s3": {
+        "href": "s3://my-bucket/path/to/thumbnail.png"
+      }
+    }
+  }
+}
+```
+
+The item or collection will also have the Alternate Assets Extension added to its `stac_extensions` array:
+
+```json
+"stac_extensions": [
+  "https://stac-extensions.github.io/alternate-assets/v1.2.0/schema.json"
+]
+```
+
 ## Collections and filter parameters for authorization
 
 One key concern in stac-server is how to restrict user's access to items.  These
@@ -1160,6 +1296,8 @@ The endpoints this applies to are:
 - /collections/:collectionId/items
 - /collections/:collectionId/items/:itemId
 - /collections/:collectionId/items/:itemId/thumbnail
+- /collections/:collectionId/items/:itemId/assets/:assetKey
+- /collections/:collectionId/assets/:assetKey
 - /search
 - /aggregate
 
@@ -1187,6 +1325,8 @@ The endpoints this applies to are:
 - /collections/:collectionId/items
 - /collections/:collectionId/items/:itemId
 - /collections/:collectionId/items/:itemId/thumbnail
+- /collections/:collectionId/items/:itemId/assets/:assetKey
+- /collections/:collectionId/assets/:assetKey
 - /search
 - /aggregate
 

serverless.example.yml

@@ -34,6 +34,10 @@ provider:
     STAC_API_URL: "https://some-stac-server.example.com"
     CORS_ORIGIN: "https://ui.example.com"
     CORS_CREDENTIALS: true
+    # Asset Proxy Environment Variables
+    # ASSET_PROXY_BUCKET_OPTION: "NONE"  # Options: NONE, ALL, ALL_BUCKETS_IN_ACCOUNT, LIST
+    # ASSET_PROXY_BUCKET_LIST: "bucket1,bucket2,bucket3"  # Required only when ASSET_PROXY_BUCKET_OPTION is LIST
+    # ASSET_PROXY_URL_EXPIRY: 300  # Pre-signed URL expiry in seconds (default: 300)
   iam:
     role:
       statements: