[Flink] question about RowKind in results

[ryyyyyy1]

In flink, the RowData and results will be marked by RowKind (+I,+U,-U). But currently in velox, it seems that there is no corresponding implementation to mark this in RowVector. Any ideas or plans on how we will support this feature to make it equal to Flink?

[ryyyyyy1]

@shuai-xu @zhztheplayer shall we wrap the RowVector with RowKind in a new class to make it similar to Flink? or is there any other plans or ideas on this? Thanks for your reply!

[shuai-xu]

We will support it soon by adding a hidden column to RowVector.

[ryyyyyy1]

Thanks for the clarification.
I have a follow-up question about the “hidden column”:

What does hidden mean in this context? Does it mean the column won’t appear in the outputType of the execution plan, but will still be physically carried in the RowVector?

If so, how can we ensure this hidden column is propagated correctly between operators, since normally operators only pass along the columns defined in the plan’s schema?

Could you elaborate a bit on how this hidden column mechanism will be designed/implemented? Also, do you have any rough timeline or plan for when this feature might be available?