Create shared sfeos helpers module

.github/workflows/cicd.yml

@@ -96,6 +96,10 @@ jobs:
         run: |
           pip install ./stac_fastapi/core
 
+      - name: Install helpers library stac-fastapi
+        run: |
+          pip install ./stac_fastapi/sfeos_helpers
+
       - name: Install elasticsearch stac-fastapi
         run: |
           pip install ./stac_fastapi/elasticsearch[dev,server]

.github/workflows/publish.yml

@@ -35,6 +35,18 @@ jobs:
           # Publish to PyPI
           twine upload dist/*
 
+      - name: Build and publish sfeos_helpers
+        working-directory: stac_fastapi/sfeos_helpers
+        env:
+          TWINE_USERNAME: "__token__"
+          TWINE_PASSWORD: ${{ secrets.PYPI_API_TOKEN }}
+        run: |
+          # Build package
+          python setup.py sdist bdist_wheel
+
+          # Publish to PyPI
+          twine upload dist/*
+
       - name: Build and publish stac-fastapi-elasticsearch
         working-directory: stac_fastapi/elasticsearch
         env:

CHANGELOG.md

@@ -5,22 +5,30 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/)
 and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.html).
 
+
 ## [Unreleased]
 
 ### Added
 
+- Created new `sfeos_helpers` package to improve code organization and maintainability [#376](https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/pull/376)
+
 ### Changed
 
+- Refactored utility functions into dedicated modules within `sfeos_helpers` [#376](https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/pull/376):
+  - Created `database` package with specialized modules for index, document, and utility operations
+  - Created `aggregation` package for Elasticsearch/OpenSearch-specific aggregation functionality
+  - Moved shared logic from core module to helper functions for better code reuse
+  - Separated utility functions from constant mappings for clearer code organization
+- Updated documentation to reflect recent code refactoring  [#376](https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/pull/376)
 - Improved README documentation with consistent formatting and enhanced sections [#381](https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/pull/381):
   - Added sfeos logo and banner
   - Added a comprehensive Quick Start guide
   - Reorganized sections for better navigation
   - Reformatted content with bullet points for improved readability
   - Added more detailed examples for API interaction
-
+  
 ### Fixed
 
-
 ## [v4.2.0] - 2025-05-15
 
 ### Added

Makefile

@@ -95,7 +95,8 @@ pybase-install:
 	pip install -e ./stac_fastapi/api[dev] && \
 	pip install -e ./stac_fastapi/types[dev] && \
 	pip install -e ./stac_fastapi/extensions[dev] && \
-	pip install -e ./stac_fastapi/core
+	pip install -e ./stac_fastapi/core && \
+	pip install -e ./stac_fastapi/sfeos_helpers
 
 .PHONY: install-es
 install-es: pybase-install

README.md

@@ -3,7 +3,6 @@
 <!-- markdownlint-disable MD033 MD041 -->
 
 
-
 <p align="left">
   <img src="assets/sfeos.png" width=1200>
 </p>
@@ -45,6 +44,7 @@ This project is built on the following technologies: STAC, stac-fastapi, FastAPI
 ## Table of Contents
 
 - [Documentation & Resources](#documentation--resources)
+- [Package Structure](#package-structure)
 - [Examples](#examples)
 - [Performance](#performance)
 - [Quick Start](#quick-start)
@@ -72,6 +72,21 @@ This project is built on the following technologies: STAC, stac-fastapi, FastAPI
   - [Gitter Chat](https://app.gitter.im/#/room/#stac-fastapi-elasticsearch_community:gitter.im) - For real-time discussions
   - [GitHub Discussions](https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/discussions) - For longer-form questions and answers
 
+## Package Structure
+
+This project is organized into several packages, each with a specific purpose:
+
+- **stac_fastapi_core**: Core functionality that's database-agnostic, including API models, extensions, and shared utilities. This package provides the foundation for building STAC API implementations with any database backend. See [stac-fastapi-mongo](https://github.com/Healy-Hyperspatial/stac-fastapi-mongo) for a working example.
+
+- **sfeos_helpers**: Shared helper functions and utilities used by both the Elasticsearch and OpenSearch backends. This package includes:
+  - `database`: Specialized modules for index, document, and database utility operations
+  - `aggregation`: Elasticsearch/OpenSearch-specific aggregation functionality
+  - Shared logic and utilities that improve code reuse between backends
+
+- **stac_fastapi_elasticsearch**: Complete implementation of the STAC API using Elasticsearch as the backend database. This package depends on both `stac_fastapi_core` and `sfeos_helpers`.
+- 
+- **stac_fastapi_opensearch**: Complete implementation of the STAC API using OpenSearch as the backend database. This package depends on both `stac_fastapi_core` and `sfeos_helpers`.
+
 ## Examples
 
 The `/examples` directory contains several useful examples and reference implementations:
@@ -158,6 +173,7 @@ There are two main ways to run the API locally:
 - **Compatibility**: The most recent Elasticsearch 7.x versions should also work. See the [opensearch-py docs](https://github.com/opensearch-project/opensearch-py/blob/main/COMPATIBILITY.md) for compatibility information.
 
 
+
 ## Configuration Reference
 
 You can customize additional settings in your `.env` file:
@@ -495,5 +511,6 @@ You can customize additional settings in your `.env` file:
   - Limits each client to a specified number of requests per time period (e.g., 500 requests per minute)
   - Helps prevent API abuse and maintains system stability
   - Ensures fair resource allocation among all clients
-
+  
 - **Examples**: Implementation examples are available in the [examples/rate_limit](examples/rate_limit) directory.
+

dockerfiles/Dockerfile.ci.es

@@ -12,6 +12,7 @@ RUN apt-get update && \
 COPY . /app/
 
 RUN pip3 install --no-cache-dir -e ./stac_fastapi/core && \
+    pip3 install --no-cache-dir -e ./stac_fastapi/sfeos_helpers && \
     pip3 install --no-cache-dir ./stac_fastapi/elasticsearch[server]
 
 USER root

dockerfiles/Dockerfile.ci.os

@@ -12,6 +12,7 @@ RUN apt-get update && \
 COPY . /app/
 
 RUN pip3 install --no-cache-dir -e ./stac_fastapi/core && \
+    pip3 install --no-cache-dir -e ./stac_fastapi/sfeos_helpers && \
     pip3 install --no-cache-dir ./stac_fastapi/opensearch[server]
 
 USER root

dockerfiles/Dockerfile.deploy.es

@@ -13,6 +13,7 @@ WORKDIR /app
 COPY . /app
 
 RUN pip install --no-cache-dir -e ./stac_fastapi/core
+RUN pip install --no-cache-dir -e ./stac_fastapi/sfeos_helpers
 RUN pip install --no-cache-dir ./stac_fastapi/elasticsearch[server]
 
 EXPOSE 8080

dockerfiles/Dockerfile.deploy.os

@@ -13,6 +13,7 @@ WORKDIR /app
 COPY . /app
 
 RUN pip install --no-cache-dir -e ./stac_fastapi/core
+RUN pip install --no-cache-dir -e ./stac_fastapi/sfeos_helpers
 RUN pip install --no-cache-dir ./stac_fastapi/opensearch[server]
 
 EXPOSE 8080

dockerfiles/Dockerfile.dev.es

@@ -16,4 +16,5 @@ WORKDIR /app
 COPY . /app
 
 RUN pip install --no-cache-dir -e ./stac_fastapi/core
+RUN pip install --no-cache-dir -e ./stac_fastapi/sfeos_helpers
 RUN pip install --no-cache-dir -e ./stac_fastapi/elasticsearch[dev,server]

dockerfiles/Dockerfile.dev.os

@@ -16,4 +16,5 @@ WORKDIR /app
 COPY . /app
 
 RUN pip install --no-cache-dir -e ./stac_fastapi/core
+RUN pip install --no-cache-dir -e ./stac_fastapi/sfeos_helpers
 RUN pip install --no-cache-dir -e ./stac_fastapi/opensearch[dev,server]

dockerfiles/Dockerfile.docs

@@ -12,6 +12,7 @@ WORKDIR /opt/src
 
 RUN python -m pip install \
     stac_fastapi/core \
+    stac_fastapi/sfeos_helpers \
     stac_fastapi/elasticsearch \
     stac_fastapi/opensearch