Skip to content

Datafusion Table Provider ties up memory in the shared ctx #10639

@jleibs

Description

@jleibs

Every query that returns a TableProvider registers it with the shared session context that's part of the CatalogClient .

These TableProviders are still fairly heavy, internally holding a ChunkStore for each queried partition.

This is particularly pathological fore notebooks where we may iterate and run the same query multiple times.

Metadata

Metadata

Assignees

Labels

dataplatformRerun Data Platform integration

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions