Denormalized table with partitions to improve marquez performance #3061
swar00pduthks
started this conversation in
Ideas
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
Hi All,
we have seen marquez performance gets degraded as the volume of the lineage metadata increases. would like to propose a solution with denormalized table with partitioned with date so that we can improve performance and also purge the data from denormalized table if we dont need the data any longer
Beta Was this translation helpful? Give feedback.
All reactions