API Thoughts & Styles

[jacob-a-brown]

Thoughts on adopting the following API styles? After debate and discussion and consensus I'll write up an ADR:

Goals

Maintain an API that is easy to upkeep, understand, and use.

How

Style

• use snake-case for all paths
  • e.g. /observation/groundwater-level instead of /observation/groundwater_level or /observation/groundwaterlevel
• use kebab_case for all query and path parameters
  • e.g. /thing/{thing_id} or /thing?thing_id=...
• if the same data can be returned in multiple formats provide a ?format= query parameter to let the user specify which format they desire.
  • e.g. ?format=shapefile, ?format=csv, ?format=geojson
• enable users to obtain data for specific objects through either a path parameter or a query parameter that share the same name.
  • e.g. /thing/{thing_id} and /thing?thing_id=...
• enable comma-separated lists to be passed to query parameters for filtering data
  • e.g. /thing?thing_id=1,2,3,4,5

Data Validation

The same custom data validations should be performed for both POST and PATCH requests. This is easily achieved by creating a base Validation Pydantic schema from which Create and Update schemas inherit. Since some fields are required for Create and optional for Update schemas, each field validator should set check_fields=False.

e.g.

from pydantic import BaseModel, field_validator

class LocationValidator(BaseModel):

    @field_validator("point", check_fields=False)
    def validate_point_near_nm(cls, point):
        if point is not None:    # <--- since this is optional in the Update schema check if it exists 
            <code to validate point is within 25 km buffer of NM (or whichever buffer we choose)>
        return point

class CreateLocation(LocationValidator):
    ...

class UpdateLocation(LocationValidator):
    ...

class LocationResponse(BaseModel):
    ...

Database errors, such as foreign key constraints and uniqueness, should be handled by the database, not Pydantic. This minimizes the number of calls made to the database because if Pydantic were used for these data validations extra calls would be made to the database (for every request n more database calls would be made where n are the number of validations performed). try/except clauses should be used to handle these errors and return appropriate error messages.

Errors & Empty Data Sets

• data validation errors for POST and PATCH requests should be done through Pydantic. These errors are automatically formatted by Pydantic. The error message should be human-readable/friendly so that the frontend can display it to the user.
• data validation errors for POST and PATCH requests return a status code of 422
• getting an object by its ID via a path parameter, like GET /thing/1 should return a 404 error if there is no object with that ID
• getting an object by its ID via a query parameter, like GET/thing?thing_id=1 should return an empty set if there is no object with that id
  • similarly, getting a list of objects by their IDs via a query parameter should just omit the ones that are not found, such as GET /thing?thing_id=1,2,3,4,5 should return all objects with the valid IDs. No errors should be raised for invalid IDs here
• errors handled by the database, such as foreign key constraints and uniqueness, should return a status code of 409
• FastAPI should raise HTTPExceptions for errors not handled by Pydantic as

raise HTTPException(
    status_code=...,
    detail=...
)

Documentation

• The docs should contain
  • an email address for getting in touch
  • a link to the API's public GitHub repo (if it exists)
  • descriptions of endpoints
    • documentation on how the endpoint works if necessary and any idiosyncrasies
    • if some data are hidden from non-authenticated users, that should be documented here
  • descriptions of schemas
  • recipes for how to use the API, or links to recipes
  • information on authorization/validation

[jacob-a-brown]

Should all error messages follow Pydantic's detail format? Doing so would make it easy for users and the frontend to quickly identify where in the payload such an error occurred. This format was utilized in the original DataManager/AMPAPI to dynamically display those messages in appropriate fields and locations.

The format is

"detail": [
    {
        "type": error type like value_error,
        "loc": where the error occurred in payload, like [body, sample_top],
        "msg": error message,
        "input": input value that caused the error
    }
]