This repository was archived by the owner on Jul 24, 2024. It is now read-only.
This repository was archived by the owner on Jul 24, 2024. It is now read-only.
Where to reference/install dbt packages with MWAA? #73
Description
Where should I be installing packages (or referencing if deployed during a CI step) dbt packages while using MWAA? I'm currently installing to: /tmp/dbt/packages but am experiencing about a 50/50 success rate when running the test operator. Here is what the graph view of my dags look like:
This is the error log for when the runs fail:
The deps operator works great, then dbt run, but the test operator fails about 50% of the time. Should I be deploying package files to a folder within the MWAA S3 folder and then updating profiles.yml to reference that location?
Environment context:
MWAA v2.0.2
dbt-core>=1.0
dbt-postgres>=1.0
airflow-dbt==0.4.0