-
Notifications
You must be signed in to change notification settings - Fork 1.6k
Labels
enhancementNew feature or requestNew feature or request
Description
Is your feature request related to a problem or challenge?
As @goldmedal started trying to move the DynamicFileProvider so others could use it in #10745 I think it is clear that there is not a good way to add additional catalog support in the core without everything being intertwined.
Thus I think we should try and extract the different catalog providers out of datafusion core so it it easier
Describe the solution you'd like
I suggest the following final layout:
- traits like
CatalogProvider
,SchemaProvider
, etc in a new cratedatafusion-catalog
(since these traits rely on table provider, etc I think this can't be indatafusion-common
ordatafusion-expr
) - The built in
Memory*
providers are indatafusion-catalog
- The bult in
InformationSchema
providers are indatafusion-catalog
- The newly proposed
DynamicFileCatalog
indatafusion-catalog
- (eventually) the LIstingTableProvider (which is by far the most complicated) moved to its own crate
datafusion-catalog-listing
Describe alternatives you've considered
No response
Additional context
No response
andygrove
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestNew feature or request