dagster-dbt is not respecting--indirect-selection cautious test selection

[bolinzzz]

Let's say I have two dbt models: model_a and model_b. Both models are referenced by a test called test_model_a_and_b.

Based on dbt documentation, indirect selection:

These are the modes to configure the behavior when performing indirect selection (with eager as the default):

- eager (default) - include ANY test that references the selected nodes
- cautious - restrict to tests that ONLY refer to selected nodes
- buildable - restrict to tests that ONLY refer to selected nodes (or their ancestors)
- empty - restrict to tests that are only for the selected node and ignore all tests from the attached nodes

When doing dbt build with dbt-core: dbt build --select model_a --indirect-selection cautious, test test_model_a_and_b is not run. It matches the behaviour described above, cautious, restricts to tests that ONLY refer to selected nodes.

When doing dbt build with dbt-core: dbt build --select model_a model_b --indirect-selection cautious, test test_model_a_and_b is run.

However on Dagster, when model_a were to auto-materialise on its own, test_model_a_and_b is run. Same is true when model_b were to auto-materialise on its own.

Is this behaviour expected? Seems like Dagster is not respecting dbt indirect test selection.

Below is how I am loading dbt models as Dagster assets:

def base_dbt_models() -> List:
    translator = CustomDagsterDbtTranslator(
        settings=DagsterDbtTranslatorSettings(enable_asset_checks=True)
    )
    dbt_assets_list = []
    # Only load assets from dbt project in Dagster SG instance.
    if is_region_SG:
        # hot reload dbt assets for local development
        if is_dev:
            dbt_parse_invocation = dbt_resource.cli(["parse", "--write-json"]).wait()
            dbt_manifest_path = dbt_parse_invocation.target_path.joinpath("manifest.json")
        else:
            dbt_manifest_path = file_relative_path(__file__, "target/manifest.json")

        @dbt_assets(manifest=dbt_manifest_path, dagster_dbt_translator=translator)
        def base_dbt_models_dbt_assets(
            context: AssetExecutionContext,
            dbt: DbtCliResource,
            config: DbtConfig
        ):
            dbt_build_args = ["build", "--indirect-selection", "cautious"]
            if config.full_refresh:
                dbt_build_args += ["--full-refresh"]
            yield from dbt.cli(dbt_build_args, context=context).stream()

        dbt_assets_list.append(base_dbt_models_dbt_assets)
    return dbt_assets_list

[johannkm]

Hi @bolinzzz, when Dagster asset checks are enabled, we override the dbt indirect selection setting to empty, then pass the exact set of tests we want to run.

It's possible with sensors/schedules to customize which tests run. However currently with Auto materialize, every check on the asset will run with the materialization. We have a tracking issue to customize asset checks + auto materializing here: #16719.

Another relevant issue is to be able to exclude certain tests from being ingested with @dbt_assets: #18124

Currently to stick with auto materializing and to not execute this test, you'll have to disable dbt asset checks until we close one of these issues.

[bolinzzz]

Thank you @johannkm! This is very clear

[fredonia88]

Hey @johannkm is there a timeline for this? I like loading my dbt tests as asset checks, but I am finding some of my dbt tests are being run in jobs because they're referenced by another model (i.e. model_a is materialized by job_a, and model_b is materialized by job_b, but since model_b has a generic test with a ref to model_a, the model_b generic test is being run in job_a).