-
Notifications
You must be signed in to change notification settings - Fork 16
Open
Description
We currently apply the predicates both in BQ and Hive. Let's figure out if we can optimize this by skipping the Hive predicates that have already been applied in BQ (i.e. predicate pushdowns). There might also be some Hive predicate functions/UDFs that don't have equivalents in BQ and so we might have to exclude those from the predicate pushdowns.
See relevant code here:
Lines 82 to 90 in af82dcc
@Override | |
public DecomposedPredicate decomposePredicate( | |
JobConf jobConf, Deserializer deserializer, ExprNodeDesc exprNodeDesc) { | |
// TODO: See if we can dissociate the pushed predicates from the residual ones | |
DecomposedPredicate predicate = new DecomposedPredicate(); | |
predicate.residualPredicate = (ExprNodeGenericFuncDesc) exprNodeDesc; | |
predicate.pushedPredicate = (ExprNodeGenericFuncDesc) exprNodeDesc; | |
return predicate; | |
} |
Metadata
Metadata
Assignees
Labels
No labels