Should we indicate DAG dependencies #4413
benhammondmusic
started this conversation in
HET Ideas (Features, Requests, New Data, etc)
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
Some of our data sources read in other pop./topic data from our bigquery before writing that topic to bigquery. We should indicate within each new dag workflow these prerequisites; easiest would be comments but even more helpful would be some type of specific error message if a topic pipeline fails because the upstream dependencies bigquery don't exist (indicating the dev must run those first)
Here is a flowchart indicating which needs which:
https://healthequitytracker.notion.site/DAG-dependencies-1ba852c1676e80eface1f3c0155b810d?pvs=4
Beta Was this translation helpful? Give feedback.
All reactions