Skip to content

Dissociate BQ and Hive predicates? #37

@jphalip

Description

@jphalip

We currently apply the predicates both in BQ and Hive. Let's figure out if we can optimize this by skipping the Hive predicates that have already been applied in BQ (i.e. predicate pushdowns). There might also be some Hive predicate functions/UDFs that don't have equivalents in BQ and so we might have to exclude those from the predicate pushdowns.

See relevant code here:

@Override
public DecomposedPredicate decomposePredicate(
JobConf jobConf, Deserializer deserializer, ExprNodeDesc exprNodeDesc) {
// TODO: See if we can dissociate the pushed predicates from the residual ones
DecomposedPredicate predicate = new DecomposedPredicate();
predicate.residualPredicate = (ExprNodeGenericFuncDesc) exprNodeDesc;
predicate.pushedPredicate = (ExprNodeGenericFuncDesc) exprNodeDesc;
return predicate;
}

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions