Request Tag Cardinality Limit with tags exclusion option OR Best way to handle unexpected high cardinality tag #22438
Replies: 3 comments 1 reply
-
Hi @techministrator, if I understand your use correctly you can use two |
Beta Was this translation helpful? Give feedback.
-
Hi @pront, I hope you have the time. May I know how can I do that with the current config?
Let's say every metric event coming in the vector would have these same set of tags like the above. And
With this vector config, the So from your reply, does it mean that I can split the same metric event into 2 streams (1 with the important tags and 1 with the rest of the tags), check for |
Beta Was this translation helpful? Give feedback.
-
Got it, thanks! Let me check that out. Btw, I think vector is generally a stateless component. I'm planning to deploy it as aggregator where there are multiple vector instances are running behind a load balancer. So I guess this tag_cardinality_limit feature might not be fit for such case since it's a shared-nothing architecture. |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
-
Hi Vector maintainers and contributors,
Impressed of the solution so far as I'm exploring to use Vector in our system mostly for processing datadog agents metrics coming in. I'm finding a best way to solve the problem of unexpectedly high cardinality tag happened with our apps custom metrics.
I found that the Tag Cardinality Limit feature is exactly what we would want or get inspired to implement in our system. Though checking with you whether is there any plan to have "Tag Exclusion" option for this feature? The reason I ask is because there are some of the required tags that we don't want this limit feature to apply, like with dd-agents, the
host
tag, and some other tags that are important in our system. So I'm sitting in between whether accepting raising higher limits to avoid important tag values being dropped and also letting unexpected high volume tag a chance to be increased. If having an extra option to exclude a certain tags to be in this limit, it would be super helpful. May I know if this feature is being considered or any plan about this so far?If there's no plan then do you think there's any best workaround that I can check out based on the current available options?
Thank you so much for taking the time.
Beta Was this translation helpful? Give feedback.
All reactions