Plan for supporting data destinations #62
wangxiaoying
started this conversation in
Ideas
Replies: 1 comment
-
I think that using arrow as data destination gives you a lot of future destinations for free. I believe arrow has the fastest algorithm to ingest data into pandas without triggering block manager re-allocations. This would give you zero copy transfer to products build around apache arrow like Polars and Dremio. And probably fast ingestion to other products. I could help on the arrow interop. |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
Other than Pandas.DataFrame, we plan to support the following destinations:
Reply here if you have other ideas!
Beta Was this translation helpful? Give feedback.
All reactions