Skip to content

Adding custom outputFiles to dbtCLI task causes dbt artifacts to no longer be rendered #157

@anna-geller

Description

@anna-geller

Issue description

This issue is not a blocker but sth worth investigating long term.

As long as you add outputFiles property to the dbtCLI task, the models are no longer rendered. We should investigate why

Reproducer:

setup flow:

id: upload_dbt_project
namespace: company.datateam.dbt

tasks:
  - id: wdir
    type: io.kestra.plugin.core.flow.WorkingDirectory
    tasks:
      - id: git_clone
        type: io.kestra.plugin.git.Clone
        url: https://github.com/kestra-io/dbt-example
        branch: master

      - id: upload
        type: io.kestra.plugin.core.namespace.UploadFiles
        namespace: "{{ flow.namespace }}"
        files:
          - "glob:**/dbt/**"

then:

id: dbt_build
namespace: company.datateam.dbt

tasks:
  - id: dbt
    type: io.kestra.plugin.dbt.cli.DbtCLI
    namespaceFiles:
      enabled: true
    containerImage: ghcr.io/kestra-io/dbt-duckdb:latest
    projectDir: dbt
    commands:
      - dbt build --project-dir dbt --profiles-dir dbt
    loadManifest:
      key: manifest.json
      namespace: "{{ flow.namespace }}"
    storeManifest:
      key: manifest.json
      namespace: "{{ flow.namespace }}"     
    taskRunner:
      type: io.kestra.plugin.scripts.runner.docker.Docker

the above flow will work fine, but then try adding:

    outputFiles:
      - "**/*.json"

dbt outputs are no longer rendered

Metadata

Metadata

Assignees

No one assigned

    Labels

    area/pluginPlugin-related issue or feature requestbugSomething isn't working

    Type

    Projects

    Status

    Backlog

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions