-
Notifications
You must be signed in to change notification settings - Fork 240
Description
Description
Currently, all dbt tasks generated by Cosmos may inherit the same retries
setting through the default_args
parameter. However, unlike dbt run
tasks, it seems unreasonable for dbt test
tasks to be retried upon failure.
If a data test fails, rerunning it will likely result in the same failure, making retries unnecessary and incurring additional time and resource costs.
To optimize execution, how about introducing a parameter like no_retries_on_test_failure: bool = True
? When enabled, this would ensure that dbt test
tasks raise an AirflowFailException
instead of an AirflowException
, preventing them from being retried while keeping the existing retry behavior for other dbt tasks.
Use case/motivation
To optimize execution, dbt test tasks should raise an AirflowFailException
instead of an AirflowException
when they fail. This will prevent them from being retried while keeping other dbt tasks, such as dbt run
, unaffected.
Related issues
No response
Are you willing to submit a PR?
- Yes, I am willing to submit a PR!