Dataflow ID design choice

[Mivik]

Is there a specific rationale for using UUIDv7 as the dataflow ID? It could introduce difficulties for users to distinguish between different dataflows, potentially increasing the risk of errors.

For comparison, Docker containers are identified by a randomly generated 32-bytes hex string, typically relying on a short, readable prefix for daily use. We could mimic this behavior if we were to adopt other UUID variants like UUIDv4.

[phil-opp]

We used to use UUIDv4, but switched to Uuidv7 because of the sorting advantage. This way, you can sort the dataflow IDs and the last one is always the newest.

We also support adding a human readable name to dataflows, in addition to the UUID. If the name is unique, you can use it to identify the dataflow.

I'm also open to using something shorter than UUIDs, as long as it's sortable and guaranteed to be unique.

We could mimic this behavior if we were to adopt other UUID variants like UUIDv4.

Could you clarify how using v4 instead of v7 helps with that?

[Mivik]

Could you clarify how using v4 instead of v7 helps with that?

Because UUIDv7s are time-prefixed, they naturally tend to share the same starting characters. This means the prefix length needed to uniquely identify a v7 ID is generally larger than the few (6-12) characters typically sufficient for the high randomness of a UUIDv4.

[phil-opp]

Thanks for clarifying! So you want to use a prefix of the UUID for dataflow identification, am I understanding that right?

I'm not sure if UUIDv4 is the best way to solve this. How about one of the following options:

• identify the dataflow by a suffix of the UUID instead of a prefix: this way we can keep UUIDv7 and its the sortability, but still have short substring IDs
• We already assign a human-readable name to each dataflow run. If we use this in more places the user would not need to use the UUID as much
• Our current design uses a central coordinator to assign the dataflow IDs. So we don't really need to use UUIDs in the first place, as we can just use internal state for ensuring that the dataflow IDs are unique. So we could just use something that is considerably shorter than a UUID (even a simple counter would be enough). If we do that, there would be no need to use prefixes or suffixes at all.

What do you think?