-
Notifications
You must be signed in to change notification settings - Fork 497
Open
Labels
enhancementNew feature or requestNew feature or request
Description
Who is this feature for?
dev/ops
What problem are they facing today?
I want a global broker configuration for all topics.
idea in my case is to autoconfigure the table topic feature depending the incoming data
idea is to say: if no specific topic config use these rules: "if message has id+timestamp columns use it as id, else if it has id use it etc..."
this kind of rule should be available for most configuration (id columns, partition by are the immediate needs there)
Why is solving this impactful?
would make the broker auto managed, no more need to hack at deployment time or anticipate it connecting the app to iceberg directly (note that the app only sees the broker so connecting it to the catalog would be a downside)
Proposed solution
Enrich the configuration with a wildcard fake topic + rules (prioritized list of default configuration)
Additional notes
No response
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestNew feature or request