Marquez deployment on (GCP) GKE - pod failure

[HARIKA126]

Hi Everyone, I am deploying Marquez API (lineage-marquez-api) and Cloud SQL Proxy (lineage-cloudsql-proxy) on GKE using ArgoCD, but my api pods are failing with “Doesn’t have minimum availability” status. Any help would be appreciated.

Setup Details:
• GKE Cluster with Workload Identity enabled
• Cloud SQL (PostgreSQL) instance in private IP mode
• Marquez API connected to PostgreSQL via Cloud SQL Proxy
• ArgoCD syncs separate Git repos for each component.
• lineage-cloudsql-proxy
• lineage-marquez-api
• lineage-marquez-web (Only this is working)
• lineage-app (Flask app to read metadata from big query and push events)

Problems Encountered:

1. Marquez API Not Starting
   • passing MARQUEZ_CONFIG variable /app/config/marquez_env.yml, fails with file not found. Tried using the service name to access the config in lineage-app.
2. When the code was tested locally, All Job facets are not being displayed in job details page but being displayed under events when we expand a particular job.

Questions for the Marquez Team:
• Are there specific Marquez configurations required for connecting through Cloud SQL Proxy?
• What additional debug steps would you recommend for diagnosing startup failures?