-
Notifications
You must be signed in to change notification settings - Fork 72
Description
Is there an existing issue for this?
- I have searched the existing issues
Describe the issue
We are seeing some errors in the country and region reports when no upstream country/region data is available.
So, we start the disabling process by setting include_list: []
and appending packages to it based on their region/country variables here.
But if all the platforms' variables are False, we end up passing an include_list
that's just []
to the get_enabled_packages
macro here.
Currently, get_enabled_packages
kinda assumes that if include_list=[]
, all packages are fair game to include. It is not able to differentiate between the case of something like a account_report, in which we don't use include_list
because all packages have account reports, and a country/region report where we just don't have any data.
Thus, we need to somehow differentiate these, so that if we truly don't find any data, we don't go through this section of the macro.
Relevant error log or model output
Error 1
Object 'GOOGLE_ADS' does not exist or not authorized.
----
Error 2
Model 'model.ad_reporting.int_ad_reporting__monthly_campaign_region_report' (models/intermediate/int_ad_reporting__monthly_campaign_region_report.sql) depends on a node named 'facebook_ads__region_report' which is disabled
Expected behavior
The country/region reports should not run.
Possible solution
We may want to add a new argument to convey that packages need to be explicitly whitelisted to be included. So if include_list=[]
in these cases, get_enabled_packages()
returns []
. Alternatively, we could change the default value of include_list
to be a list of all packages, so if it's []
, it truly is empty no matter what. This would require some reworking of the macro conditionals though.
Another option (that wouldn't require changing the macro) could be adjusting these sorts of sections to conversely append packages to the exclude_list
if their respective variable is disabled. Currently, we just focus on adding to the include_list
.
^ This way would allow us to easily add Google (and any other platforms without country + region reports, like Amazon) to the exclude_list
argument of get_enabled_packages
so it never gets brought in.
Now that I'm reading this back, the exclude_list
route is probably best
dbt Project configurations
NA
Package versions
latest
What database are you using dbt with?
databricks, snowflake
How are you running this dbt package?
Fivetran Quickstart Data Model
dbt Version
NA
Additional Context
No response
Are you willing to open a PR to help address this issue?
- Yes.
- Yes, but I will need assistance.
- No.