Skip to content

[CT-3521] [Feature] Remove REFRESH from snowflake_refresh_snowpipe #250

@clementmg-getaround

Description

@clementmg-getaround

Remove REFRESH from here since the Snowflake docs say that:

The REFRESH functionality is intended for short term use to resolve specific issues when Snowpipe fails to load a subset of files and is not intended for regular use

The original feature request transferred from dbt-core follows below:

Is this your first time submitting a feature request?

  • I have read the expectations for open source contributors
  • I have searched the existing issues, and I could not find an existing issue for this feature
  • I am requesting a straightforward extension of existing dbt functionality, rather than a Big Idea better suited to a discussion

Describe the feature

The idea is to create the famous sleep/wait command, that is present across multiple frameworks, and that simply allows the worker to wait a defined amount of time: dbt sleep 3

Describe alternatives you've considered

Alternatives were hacky really: launching some impactless commands to mimic a sleep behavior.
We also thought of splitting the dbt job in two, schedule their execution through an external tool, which supports sleep command (Airflow). We believe dbt should possess that power.

Who will this benefit?

This known command can be very useful in multiple situations.
In our case, a dbt job scheduled two distinct tasks.
And because of external shenanigans (related to snowflake pipes), the first job is marked as finished by dbt when in fact it is still running. Therefore, the second job starts too early.
A simple dbt sleep with a custom time would prevent that unwanted behavior.

I am positive this feature could help solve similar issues across various platforms.

Are you interested in contributing this feature?

Sure!

Anything else?

No response

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions