-
Notifications
You must be signed in to change notification settings - Fork 143
Open
Labels
bugSomething isn't workingSomething isn't workingdocumentationImprovements or additions to documentationImprovements or additions to documentation
Milestone
Description
Currently read-batch-size at pod level (at a vertex) is computed as
partition-read-batch = read-batch-size
pod-read-batch = read-batch-size * partition count
i.e., if we set 300 as batch-size and if we have 3 partitions, the pod cummulative read batch size is 900.
The above is wrong because vertex setting for a pod is inadvertently overridden by changing partition count.
Fix
Reach Batch Size for a pod at partition level should be inversely proportional to ISB partition count
partition-read-batch = read-batch-size / partition count
pod-read-batch = read-batch-size
Metadata
Metadata
Assignees
Labels
bugSomething isn't workingSomething isn't workingdocumentationImprovements or additions to documentationImprovements or additions to documentation