Object Tags Support

[jfiore-im]

@littleK0i
I see here that Object Tags are not currently supported.

Is there any update on if/when they will be supported?

Has anyone in the community implemented support for that object type that can be shared informally?

I understand that the view and function have a lag.
Wouldn't that simply cause SnowDDL to repeatedly re-apply the Tags (on each apply run) until that information is updated?
Would that be a reasonable tradeoff to accept in order to have that support?

I suppose that there could be improperly tagged items if the data is out of date.
But if the DDL to apply them is just ALTER <object> SET TAG <tagName> = '<tagValue>' then I would think that just always applying them if they aren't returned from the View/Function would be safe to do.

[littleK0i]

Unless Snowflake fixed the problem with delays, this feature is most likely useless and should be avoided. Without consistency is has no value.

If you set tags for objects, you are probably planning to rely on these tags for something. But reading tags will be as inconsistent as writing, so your scripts may not be able to read the most recent state and fail.

Having a basic config file with tags and reading it directly from your Python code is more reliable. Even using basic object COMMENT is more reliable.

[littleK0i]

In theory, we may consider the following option:

1. Introduce universal parameter extra, which is going to be allowed in all config files. This parameter can hold any additional meta-information attached to objects and not used by SnowDDL directly.
2. Write a Python script to read config files, find extra and do something on top of it.

Probably the same thing you've planned to do with tags, but consistent, free of charge, without any limitations.

Sample config for table:

columns:
  id: NUMBER(38,0)
  name: VARCHAR(255)

extra:
  description: "This is my fancy table for tests"
  tags:
    - foo
    - bar

[jfiore-im]

@littleK0i Thanks for the prompt response.

When you talk about adding a universal parameter extra are you proposing that that would be done in SnowDDL proper? or as a potential custom extension point that I would develop?

And if I understand you correctly, the extra parameter would not actually be used by SnowDDL to do anything in Snowflake, but rather when I want to calculate my costs for different parts of my organization (the primary objective of wanting to use Object Tags in my case), I would read those config files, create an in-memory mapping and then use that to post-process the cost data (or to generate a query against the cost data that has those mappings in CASE statements or something similar).

[littleK0i]

Correct, extra is for building custom stuff on top of existing SnowDDL config.

But for costs allocation object tags are not really needed. Consider the following option:

1. Report storage costs based on database / schema. If objects in the same schema belong to multiple teams, it usually means you should split it into multiple schemas for better clarity.
2. Report compute costs based on query_tag and warehouse_name. Aggregate similar queries by query_hash. Allocate idle time, since you still pay money for it, even if no queries are running.

[littleK0i]

In query_tag you may put any custom information.

For example:

{"team": "Marketing", "name": "John Doe", "source": "Metabase"}

It makes cost allocation easy.