Skip to content

[Bug] Handle disabling the country and region reports when all upstream models are missing #155

@fivetran-jamie

Description

@fivetran-jamie

Is there an existing issue for this?

  • I have searched the existing issues

Describe the issue

We are seeing some errors in the country and region reports when no upstream country/region data is available.

So, we start the disabling process by setting include_list: [] and appending packages to it based on their region/country variables here.

But if all the platforms' variables are False, we end up passing an include_list that's just [] to the get_enabled_packages macro here.

Currently, get_enabled_packages kinda assumes that if include_list=[], all packages are fair game to include. It is not able to differentiate between the case of something like a account_report, in which we don't use include_list because all packages have account reports, and a country/region report where we just don't have any data.

Thus, we need to somehow differentiate these, so that if we truly don't find any data, we don't go through this section of the macro.

Relevant error log or model output

Error 1

Object 'GOOGLE_ADS' does not exist or not authorized.

---- 
Error 2

Model 'model.ad_reporting.int_ad_reporting__monthly_campaign_region_report' (models/intermediate/int_ad_reporting__monthly_campaign_region_report.sql) depends on a node named 'facebook_ads__region_report' which is disabled

Expected behavior

The country/region reports should not run.

Possible solution

We may want to add a new argument to convey that packages need to be explicitly whitelisted to be included. So if include_list=[] in these cases, get_enabled_packages() returns []. Alternatively, we could change the default value of include_list to be a list of all packages, so if it's [], it truly is empty no matter what. This would require some reworking of the macro conditionals though.

Another option (that wouldn't require changing the macro) could be adjusting these sorts of sections to conversely append packages to the exclude_list if their respective variable is disabled. Currently, we just focus on adding to the include_list.

^ This way would allow us to easily add Google (and any other platforms without country + region reports, like Amazon) to the exclude_list argument of get_enabled_packages so it never gets brought in.

Now that I'm reading this back, the exclude_list route is probably best

dbt Project configurations

NA

Package versions

latest

What database are you using dbt with?

databricks, snowflake

How are you running this dbt package?

Fivetran Quickstart Data Model

dbt Version

NA

Additional Context

No response

Are you willing to open a PR to help address this issue?

  • Yes.
  • Yes, but I will need assistance.
  • No.

Metadata

Metadata

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions