Some links are duplicating the `root_path`

[ividito]

We're deploying stac-fastapi.pgstac using a root_path in the app declaration (not through uvicorn, which has already been fixed in #221). We're finding some links where the root_path is duplicated. Most links are fine - it's only certain links that are duplicating paths.

Examples:

https://dev.openveda.cloud/api/stac/collections - links.self is affected

"links": [
    {
      "rel": "root",
      "type": "application/json",
      "href": "https://dev.openveda.cloud/api/stac/"
    },
    {
      "rel": "self",
      "type": "application/json",
      "href": "https://dev.openveda.cloud/api/stac/api/stac/collections"
    }
  ],

https://dev.openveda.cloud/api/stac/search?collections=no2-monthly - links.next is affected

"links": [
    {
      "rel": "root",
      "type": "application/json",
      "href": "https://dev.openveda.cloud/api/stac/"
    },
    {
      "rel": "self",
      "type": "application/geo+json",
      "href": "https://dev.openveda.cloud/api/stac/search"
    },
    {
      "rel": "next",
      "type": "application/geo+json",
      "method": "GET",
      "href": "https://dev.openveda.cloud/api/stac/api/stac/search?collections=no2-monthly&.language=en&token=next:no2-monthly:OMI_trno2_0.10x0.10_202303_Col3_V4.nc"
    }
  ],

I think #250 might be addressing the same issue. Should that PR be re-opened, or is there another solution we're overlooking?

[alexismanin]

Same problem seen on search api, when deploying the app using a ROOT_PATH environment variable.

Initially, we were using version 5.0.0, and the problem was present on both self and next links in search response (extract from a local test server):

    {
      "rel": "next",
      "type": "application/geo+json",
      "method": "GET",
      "href": "http://localhost:8080/catalog/catalog/search?limit=5&token=..."
    },
    {
      "rel": "root",
      "type": "application/json",
      "href": "http://localhost:8080/catalog/"
    },
    {
      "rel": "self",
      "type": "application/json",
      "href": "http://localhost:8080/catalog/catalog/search?limit=5"
    }

After updating to 5.0.2, the problem disappeared on self links, but remain present on next links:

    {
      "rel": "root",
      "type": "application/json",
      "href": "http://localhost:8080/catalog/"
    },
    {
      "rel": "self",
      "type": "application/geo+json",
      "href": "http://localhost:8080/catalog/search"
    },
    {
      "rel": "next",
      "type": "application/geo+json",
      "method": "GET",
      "href": "http://localhost:8080/catalog/catalog/search?limit=5&token=..."
    }

The deployment used to produce the samples above is the following docker compose :

services:
  stac_db:
    image: ghcr.io/stac-utils/pgstac:v0.9.5
    environment:
      POSTGRES_DB: stac
      POSTGRES_USER: stac
      POSTGRES_PASSWORD: stac
      PGUSER: stac
      PGPASSWORD: stac
      PGDATABASE: stac
      PG_DATA: /var/lib/postgresql/data/pgdata
    ports:
      - "5432:5432"
    volumes:
      - "pgdata:/var/lib/postgresql/data"
    healthcheck:
      test: [ "CMD", "pg_isready", "-U", "stac", "-d", "stac" ]
      retries: 10
      interval: "2s"

  stac_api:
    image: ghcr.io/stac-utils/stac-fastapi-pgstac:5.0.0
    ports:
      - "8080:8080"
    environment:
      # Database configuration
      POSTGRES_USER: stac
      POSTGRES_PASS: stac
      POSTGRES_DBNAME: stac
      POSTGRES_HOST_READER: stac_db
      POSTGRES_HOST_WRITER: stac_db
      POSTGRES_PORT: 5432

      # App configuration
      APP_HOST: 0.0.0.0
      APP_PORT: 8080
      ROOT_PATH: /catalog
    depends_on:
      stac_db:
        condition: service_healthy

volumes:
  pgdata:

Is there any other configuration to set that could get rid of the problem ?

[alexismanin]

I think #250 might be addressing the same issue. Should that PR be re-opened, or is there another solution we're overlooking?

I just tested PR #250, and it does fix my problem on the search endpoint.

After having built a docker image for the PR branch and updated my compose configuration to use it, the service links looks correct :

    {
      "rel": "root",
      "type": "application/json",
      "href": "http://localhost:8080/catalog/"
    },
    {
      "rel": "self",
      "type": "application/geo+json",
      "href": "http://localhost:8080/catalog/search"
    },
    {
      "rel": "next",
      "type": "application/geo+json",
      "method": "GET",
      "href": "http://localhost:8080/catalog/search?limit=5&token=..."
    }

It think it might be worth a second look.

[fabito]

I'm facing the same issue with the pagination links (next).

[fabito]

I think I manage to reproduce and isolate the issue in a unit test:

• Reproduce duplicated root-path issue in unit test #265

[vincentsarago]

I think the issue @ividito pointed is specific to their deployment (AWS Lambda + API Gateway + CloudFront)

The problem is that root-path is option is designed for proxies. I think users can solve this by settings correct headers in the request (ref: https://josef.codes/using-the-x-forwarded-prefix-header-to-prefix-your-hateoas-links/)

[alexismanin]

I think the issue @ividito pointed is specific to their deployment (AWS Lambda + API Gateway + CloudFront)

If that was the issue, I would expect all links in the response to get the same problem. But we all have the same phenomenon: in a single JSON response, some links are correct, and others are not. It means that prefix configuration is not handled in the same way by all displayed links.

If the problem was coming from a bad X-Forwarded-Prefix header, it would mean that the X-Forwarded-Prefix header itself would duplicate the path. But then, if STAC API was trusting the X-Forwarded-Prefix header, all displayed links would duplicate the root path.

And again, I also see the same problem in a docker compose environment without any proxy between the client (my browser) and the the STAC API server.

I admit that I am really confused by FastAPI: Behind a Proxy documentation. I have the feeling that root_path is specifically designed for proxies that perform path rewriting (whose path prefix stripping is the most common use-case).

But one common usage of proxies is to only redirect domain/port, without any path rewrite involved (Example: Kubernetes Ingress in mode pathType: Prefix). In such cases, we want to be able to startup proxied application on a custom path, so the application just "knows" it is started on the given context/root path, and users will access it on this exact path, independently of the fact that it is proxied or not.

I wonder how such case should be handled, and I also wonder if that's how @ividito and @fabito use/expose their STAC API servers.

[vincentsarago]

root-path is for proxies not for application prefix.

to use prefix you need to use custom Router in your application: https://github.com/stac-utils/stac-fastapi-pgstac/blob/main/tests/conftest.py#L193

[alexismanin]

to use prefix you need to use custom Router in your application: https://github.com/stac-utils/stac-fastapi-pgstac/blob/main/tests/conftest.py#L193

Is there an environment variable for this (Ex: PREFIX_PATH=/catalog), or are we forced to define our own FastAPI app in Python to access this feature ? (and therefore having to setup a python project, copy / paste a large amount of code/configuration and hoping we didn't miss something, and finally create and publish our own Docker image)

[vincentsarago]

Is there an environment variable for this (Ex: PREFIX_PATH=/catalog)

PR are welcome 🙏

[fabito]

What worked for me (API Gateway -> Lambda -> Mangum -> FastAPI) is to set Mangum's aws_api_gateway_root_path argument to the root path.

handler = Mangum(app, api_gateway_base_path=settings.root_path)

fastapi/fastapi#9018 (comment)

[alukach]

I also see the same problem in a docker compose environment without any proxy between the client (my browser) and the the STAC API server.

@alexismanin Are you able to share a simplified, reproducible example of this docker compose environment example? I'd like to dig into this a bit more.

[alexismanin]

@alexismanin Are you able to share a simplified, reproducible example of this docker compose environment example? I'd like to dig into this a bit more.

@alukach : You will find the ready-to-run compose configuration in my first comment on this issue (at the end of the comment).

Also, note that removing the root_path and using the router prefix as suggested by @vincentsarago fixes the issue for me (I do not really need path rewrite at all, I was just needed a prefix path for my k8s ingress to redirect requests properly).

[alukach]

You will find the ready-to-run compose configuration in my first comment on this issue (at the end of the comment).

@alexismanin ah, right you are, sorry to have missed that!

If that was the issue, I would expect all links in the response to get the same problem. But we all have the same phenomenon: in a single JSON response, some links are correct, and others are not. It means that prefix configuration is not handled in the same way by all displayed links.

Glad to hear that you've gotten around the issue, however, I agree with your above comment that something is fishy.

[vincentsarago]

FYI we already have a docker-compose with nginx proxy set up

stac-fastapi-pgstac/docker-compose.yml

Lines 80 to 93 in d33b0f3

	nginx:
	image: nginx
	ports:
	- ${STAC_FASTAPI_NGINX_PORT:-8080}:80
	volumes:
	- ./nginx.conf:/etc/nginx/nginx.conf
	depends_on:
	- app-nginx
	command: [ "nginx-debug", "-g", "daemon off;" ]
	app-nginx:
	extends:
	service: app
	command: bash -c "./scripts/wait-for-it.sh database:5432 && uvicorn stac_fastapi.pgstac.app:app --host 0.0.0.0 --port 8082 --proxy-headers --forwarded-allow-ips=* --root-path=/api/v1/pgstac"