Skip to content

add support for feature/record domain queries #1988

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Apr 2, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
56 changes: 40 additions & 16 deletions docs/source/data-publishing/ogcapi-features.rst
Original file line number Diff line number Diff line change
Expand Up @@ -16,24 +16,24 @@ parameters.


.. csv-table::
:header: Provider, property filters/display, resulttype, bbox, datetime, sortby, skipGeometry, CQL, transactions, crs
:header: Provider, property filters/display, resulttype, bbox, datetime, sortby, skipGeometry, domains, CQL, transactions, crs
:align: left

`CSV`_,✅/✅,results/hits,❌,❌,❌,✅,❌,❌,✅
`Elasticsearch`_,✅/✅,results/hits,✅,✅,✅,✅,✅,✅,✅
`ERDDAP Tabledap Service`_,❌/❌,results/hits,✅,✅,❌,❌,❌,❌,✅
`ESRI Feature Service`_,✅/✅,results/hits,✅,✅,✅,✅,❌,❌,✅
`GeoJSON`_,✅/✅,results/hits,❌,❌,❌,✅,❌,❌,✅
`MongoDB`_,✅/❌,results,✅,✅,✅,✅,❌,❌,✅
`OGR`_,✅/❌,results/hits,✅,❌,❌,✅,❌,❌,✅
`OpenSearch`_,✅/✅,results/hits,✅,✅,✅,✅,✅,✅,✅
`Oracle`_,✅/✅,results/hits,✅,❌,✅,✅,❌,❌,✅
`Parquet`_,✅/✅,results/hits,✅,✅,❌,✅,❌,❌,✅
`PostgreSQL`_,✅/✅,results/hits,✅,✅,✅,✅,✅,✅,✅
`SQLiteGPKG`_,✅/❌,results/hits,✅,❌,❌,✅,❌,❌,✅
`SensorThings API`_,✅/✅,results/hits,✅,✅,✅,✅,❌,✅,✅
`Socrata`_,✅/✅,results/hits,✅,✅,✅,✅,❌,❌,✅
`TinyDB`_,✅/✅,results/hits,✅,✅,✅,✅,❌,✅,✅
`CSV`_,✅/✅,results/hits,❌,❌,❌,✅,❌,❌,❌,
`Elasticsearch`_,✅/✅,results/hits,✅,✅,✅,✅,✅,✅,✅,✅
`ERDDAP Tabledap Service`_,❌/❌,results/hits,✅,✅,❌,❌,❌,❌,❌,
`ESRI Feature Service`_,✅/✅,results/hits,✅,✅,✅,✅,❌,❌,❌,
`GeoJSON`_,✅/✅,results/hits,❌,❌,❌,✅,❌,❌,❌,
`MongoDB`_,✅/❌,results,✅,✅,✅,✅,❌,❌,❌,
`OGR`_,✅/❌,results/hits,✅,❌,❌,✅,❌,❌,❌,
`OpenSearch`_,✅/✅,results/hits,✅,✅,✅,✅,❌,✅,✅,✅
`Oracle`_,✅/✅,results/hits,✅,❌,✅,✅,❌,❌,❌,
`Parquet`_,✅/✅,results/hits,✅,✅,❌,✅,❌,❌,❌,
`PostgreSQL`_,✅/✅,results/hits,✅,✅,✅,✅,❌,✅,✅,✅
`SQLiteGPKG`_,✅/❌,results/hits,✅,❌,❌,✅,❌,❌,❌,
`SensorThings API`_,✅/✅,results/hits,✅,✅,✅,✅,❌,❌,✅,✅
`Socrata`_,✅/✅,results/hits,✅,✅,✅,✅,❌,❌,❌,
`TinyDB`_,✅/✅,results/hits,✅,✅,✅,✅,✅,❌,✅,✅

.. note::

Expand Down Expand Up @@ -755,18 +755,35 @@ Data access examples
* list all collections

* http://localhost:5000/collections

* overview of dataset

* http://localhost:5000/collections/foo

* queryables

* http://localhost:5000/collections/foo/queryables

* queryables on specific properties

* http://localhost:5000/collections/foo/queryables?properties=title,type

* queryables with current domain values

* http://localhost:5000/collections/foo/queryables?profile=actual-domain

* queryables on specific properties with current domain values

* http://localhost:5000/collections/foo/queryables?profile=actual-domain&properties=title,type

* browse features

* http://localhost:5000/collections/foo/items

* paging

* http://localhost:5000/collections/foo/items?offset=10&limit=10

* CSV outputs

* http://localhost:5000/collections/foo/items?f=csv
Expand All @@ -779,24 +796,31 @@ Data access examples
* query features (attribute)

* http://localhost:5000/collections/foo/items?propertyname=foo

* query features (temporal)

* http://localhost:5000/collections/foo/items?datetime=2020-04-10T14:11:00Z

* query features (temporal) and sort ascending by a property (if no +/- indicated, + is assumed)

* http://localhost:5000/collections/foo/items?datetime=2020-04-10T14:11:00Z&sortby=+datetime

* query features (temporal) and sort descending by a property

* http://localhost:5000/collections/foo/items?datetime=2020-04-10T14:11:00Z&sortby=-datetime

* query features in a given (and supported) CRS

* http://localhost:5000/collections/foo/items?crs=http%3A%2F%2Fwww.opengis.net%2Fdef%2Fcrs%2FEPSG%2F0%2F32633

* query features in a given bounding BBOX and return in given CRS

* http://localhost:5000/collections/foo/items?bbox=120000,450000,130000,460000&bbox-crs=http%3A%2F%2Fwww.opengis.net%2Fdef%2Fcrs%2FEPSG%2F0%2F28992&crs=http%3A%2F%2Fwww.opengis.net%2Fdef%2Fcrs%2FEPSG%2F0%2F32633

* fetch a specific feature

* http://localhost:5000/collections/foo/items/123

* fetch a specific feature in a given (and supported) CRS

* http://localhost:5000/collections/foo/items/123?crs=http%3A%2F%2Fwww.opengis.net%2Fdef%2Fcrs%2FEPSG%2F0%2F32633
Expand Down
18 changes: 15 additions & 3 deletions docs/source/data-publishing/ogcapi-records.rst
Original file line number Diff line number Diff line change
Expand Up @@ -15,12 +15,12 @@ pygeoapi core record providers are listed below, along with a matrix of supporte
parameters.

.. csv-table::
:header: Provider, properties (filters), resulttype, q, bbox, datetime, sortby, properties (display), CQL, transactions
:header: Provider, properties (filters), resulttype, q, bbox, datetime, sortby, properties (display), domains, CQL, transactions
:align: left

`ElasticsearchCatalogue`_,✅,results/hits,✅,✅,✅,✅,✅,✅,✅
`TinyDBCatalogue`_,✅,results/hits,✅,✅,✅,✅,❌,✅,✅
`CSWFacade`_,✅,results/hits,✅,✅,✅,❌,❌,❌,❌
`TinyDBCatalogue`_,✅,results/hits,✅,✅,✅,✅,❌,✅,✅,✅,✅
`CSWFacade`_,✅,results/hits,✅,✅,✅,❌,❌,✅,❌,❌


Below are specific connection examples based on supported providers.
Expand Down Expand Up @@ -107,6 +107,18 @@ Metadata search examples

* http://localhost:5000/collections/foo/queryables

* queryables on specific properties

* http://localhost:5000/collections/foo/queryables?properties=title,type

* queryables with current domain values

* http://localhost:5000/collections/foo/queryables?profile=actual-domain

* queryables on specific properties with current domain values

* http://localhost:5000/collections/foo/queryables?profile=actual-domain&properties=title,type

* browse records

* http://localhost:5000/collections/foo/items
Expand Down
54 changes: 52 additions & 2 deletions pygeoapi/api/itemtypes.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
# Colin Blackburn <colb@bgs.ac.uk>
# Ricardo Garcia Silva <ricardo.garcia.silva@geobeyond.it>
#
# Copyright (c) 2024 Tom Kralidis
# Copyright (c) 2025 Tom Kralidis
# Copyright (c) 2025 Francesco Bartoli
# Copyright (c) 2022 John A Stevenson and Colin Blackburn
# Copyright (c) 2023 Ricardo Garcia Silva
Expand Down Expand Up @@ -111,6 +111,7 @@ def get_collection_queryables(api: API, request: Union[APIRequest, Any],
:returns: tuple of headers, status code, content
"""

domains = {}
headers = request.get_response_headers(**api.api_headers)

if any([dataset is None,
Expand All @@ -135,16 +136,39 @@ def get_collection_queryables(api: API, request: Union[APIRequest, Any],
if p is None:
msg = 'queryables not available for this collection'
return api.get_exception(

HTTPStatus.BAD_REQUEST, headers, request.format,
'NoApplicableError', msg)

LOGGER.debug('Processing profile')
profile = request.params.get('profile', '')

LOGGER.debug('Processing properties')
val = request.params.get('properties')
if val is not None:
properties = [x for x in val.split(',') if x]
properties_to_check = set(p.properties) | set(p.fields.keys())

if len(list(set(properties) - set(properties_to_check))) > 0:
msg = 'unknown properties specified'
return api.get_exception(
HTTPStatus.BAD_REQUEST, headers, request.format,
'InvalidParameterValue', msg)
else:
properties = []

queryables_id = f'{api.get_collections_url()}/{dataset}/queryables'

if request.params:
queryables_id += '?' + urllib.parse.urlencode(request.params)

queryables = {
'type': 'object',
'title': l10n.translate(
api.config['resources'][dataset]['title'], request.locale),
'properties': {},
'$schema': 'http://json-schema.org/draft/2019-09/schema',
'$id': f'{api.get_collections_url()}/{dataset}/queryables'
'$id': queryables_id
}

if p.fields:
Expand All @@ -153,8 +177,17 @@ def get_collection_queryables(api: API, request: Union[APIRequest, Any],
'x-ogc-role': 'primary-geometry'
}

if profile == 'actual-domain':
try:
domains, _ = p.get_domains(properties)
except NotImplementedError:
LOGGER.debug('Domains are not suported by this provider')
domains = {}

for k, v in p.fields.items():
show_field = False
if properties and k not in properties:
continue
if p.properties:
if k in p.properties:
show_field = True
Expand All @@ -175,6 +208,8 @@ def get_collection_queryables(api: API, request: Union[APIRequest, Any],
queryables['properties'][k]['x-ogc-role'] = 'id'
if k == p.time_field:
queryables['properties'][k]['x-ogc-role'] = 'primary-instant' # noqa
if domains.get(k):
queryables['properties'][k]['enum'] = domains[k]

if request.format == F_HTML: # render
tpl_config = api.get_dataset_templates(dataset)
Expand Down Expand Up @@ -1057,6 +1092,19 @@ def get_oas_30(cfg: dict, locale: str) -> tuple[list[dict[str, str]], dict[str,
}
}

profile = {
'name': 'profile',
'in': 'query',
'description': 'The profile to be applied to a given request',
'required': False,
'style': 'form',
'explode': False,
'schema': {
'type': 'string',
'enum': ['actual-domain', 'valid-domain']
}
}

LOGGER.debug('setting up collection endpoints')
paths = {}

Expand Down Expand Up @@ -1190,7 +1238,9 @@ def get_oas_30(cfg: dict, locale: str) -> tuple[list[dict[str, str]], dict[str,
'tags': [k],
'operationId': f'get{k.capitalize()}Queryables',
'parameters': [
coll_properties,
{'$ref': '#/components/parameters/f'},
profile,
{'$ref': '#/components/parameters/lang'}
],
'responses': {
Expand Down
2 changes: 1 addition & 1 deletion pygeoapi/django_/urls.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
# Copyright (c) 2025 Francesco Bartoli
# Copyright (c) 2022 Luca Delucchi
# Copyright (c) 2022 Krishna Lodha
# Copyright (c) 2024 Tom Kralidis
# Copyright (c) 2025 Tom Kralidis
#
# Permission is hereby granted, free of charge, to any person
# obtaining a copy of this software and associated documentation
Expand Down
2 changes: 1 addition & 1 deletion pygeoapi/flask_app.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
# Authors: Tom Kralidis <tomkralidis@gmail.com>
# Norman Barker <norman.barker@gmail.com>
#
# Copyright (c) 2024 Tom Kralidis
# Copyright (c) 2025 Tom Kralidis
#
# Permission is hereby granted, free of charge, to any person
# obtaining a copy of this software and associated documentation
Expand Down
16 changes: 15 additions & 1 deletion pygeoapi/provider/base.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
#
# Authors: Tom Kralidis <tomkralidis@gmail.com>
#
# Copyright (c) 2022 Tom Kralidis
# Copyright (c) 2025 Tom Kralidis
#
# Permission is hereby granted, free of charge, to any person
# obtaining a copy of this software and associated documentation
Expand Down Expand Up @@ -145,6 +145,20 @@ def get_metadata(self):

raise NotImplementedError()

def get_domains(self, properties=[], current=False):
"""
Get domains from dataset

:param properties: `list` of property names
:param current: `bool` of whether to provide list of live
values (default `False`)

:returns: `tuple` of domains and whether they are based on the
current/live dataset
"""

raise NotImplementedError()

def query(self):
"""
query the provider
Expand Down
31 changes: 30 additions & 1 deletion pygeoapi/provider/csw_facade.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
#
# Authors: Tom Kralidis <tomkralidis@gmail.com>
#
# Copyright (c) 2023 Tom Kralidis
# Copyright (c) 2025 Tom Kralidis
#
# Permission is hereby granted, free of charge, to any person
# obtaining a copy of this software and associated documentation
Expand Down Expand Up @@ -91,6 +91,35 @@ def get_fields(self):

return self._fields

def get_domains(self, properties=[], current=False) -> tuple:
"""
Get domains from dataset

:param properties: `list` of property names
:param current: `bool` of whether to provide list of live
values (default `False`)

:returns: `tuple` of domains and whether they are based on the
current/live dataset
"""

LOGGER.debug(f'Querying CSW: {self.data}')
records = self.query()
domains = {}

if properties:
keys = properties
else:
keys = records['features'][0]['properties'].keys()

csw = self._get_csw()

for key in keys:
csw.getdomain(key, dtype='property')
domains[key] = csw.results['values']

return domains, True

@crs_transform
def query(self, offset=0, limit=10, resulttype='results',
bbox=[], datetime_=None, properties=[], sortby=[],
Expand Down
Loading