Substrait Spec: Cross-Language Serialization for Relational Algebra

[BohuTANG]

Summary

https://substrait.io/

Substrait is a cross platform way to express data transformation, relational algebra, standardized record expression and plans.

[ZhiHanZ]

interesting demo: https://github.com/andygrove/substrait-rs

[BohuTANG]

Maybe substrait will help us to standardize our new planner and data type, FYI
https://github.com/facebookincubator/velox/tree/main/velox/substrait/proto/substrait
cc @leiysky @sundy-li

[leiysky]

https://github.com/pingcap/tipb

TiDB has a similar protocol, which made us suffer a lot. It's really hard to design a cross-language/platform intermidiate binary protocol.

I'd like to ask what is this serialization protocol for?

[BohuTANG]

Well.

All query engines which follow the substrait sepc can talk with each other on the logic plan level.
Just a reference information, no other special needs here.