Skip to content

Commit b5cac92

Browse files
authored
Python: Fix bad join in getOuterVariable
Much sadness: ``` Tuple counts for ImportTime::ImportTimeScope::getOuterVariable#dispred#f0820431#fff/3@64d04d33 after 7.6s: 19624 ~1% {1} r1 = SCAN py_Classes OUTPUT In.0 'this' 19531 ~1% {1} r2 = JOIN r1 WITH ImportTime::ImportTimeScope#class#7851b601#f ON FIRST 1 OUTPUT Lhs.0 'this' 19531 ~0% {2} r3 = JOIN r2 WITH Scope::Scope::getEnclosingModule#dispred#f0820431#ff ON FIRST 1 OUTPUT Lhs.0 'this', Rhs.1 296389 ~0% {3} r4 = JOIN r3 WITH Variables::Variable::getScope#dispred#f0820431#ff_10#join_rhs ON FIRST 1 OUTPUT Rhs.1 'var', Lhs.0 'this', Lhs.1 296389 ~0% {3} r5 = JOIN r4 WITH Variables::LocalVariable#3aa06bbf#f ON FIRST 1 OUTPUT Lhs.0 'var', Lhs.1 'this', Lhs.2 296389 ~1% {4} r6 = JOIN r5 WITH Variables::Variable::getId#dispred#f0820431#ff ON FIRST 1 OUTPUT Lhs.2, Lhs.1 'this', Lhs.0 'var', Rhs.1 62294919 ~0% {4} r7 = JOIN r6 WITH Variables::Variable::getScope#dispred#f0820431#ff_10#join_rhs ON FIRST 1 OUTPUT Rhs.1 'var', Lhs.1 'this', Lhs.2 'var', Lhs.3 62294919 ~0% {4} r8 = JOIN r7 WITH Variables::GlobalVariable#class#3aa06bbf#f ON FIRST 1 OUTPUT Lhs.0 'result', Lhs.3, Lhs.1 'this', Lhs.2 'var' 639 ~0% {3} r9 = JOIN r8 WITH Variables::Variable::getId#dispred#f0820431#ff ON FIRST 2 OUTPUT Lhs.2 'this', Lhs.3 'var', Lhs.0 'result' return r9 ``` Clearly we _shouldn't_ be joining on `getId` as the last thing, as this means we're building tuples of completely unrelated variables (not even with the same name!) which obviously blows up. A standard way of fixing this is to correlate as much information about these variables as possible in a `nomagic`ked helper predicate. This is what we do here, grouping together the variable with its scope and name (both of which are uniquely determined by the variable). This results in a much nicer join order: ``` Tuple counts for ImportTime::ImportTimeScope::getOuterVariable#dispred#f0820431#fff/3@82866b6p after 42ms: 23867 ~4% {2} r1 = JOIN Scope::Scope::getEnclosingModule#dispred#f0820431#ff WITH ImportTime::ImportTimeScope#class#7851b601#f ON FIRST 1 OUTPUT Lhs.0 'this', Lhs.1 296389 ~0% {4} r2 = JOIN r1 WITH ImportTime::class_var_scope#7851b601#fff ON FIRST 1 OUTPUT Rhs.1, Lhs.1, Lhs.0 'this', Rhs.2 'var' 639 ~0% {3} r3 = JOIN r2 WITH ImportTime::global_var_scope#7851b601#fff ON FIRST 2 OUTPUT Lhs.2 'this', Lhs.3 'var', Rhs.2 'result' return r3 ``` ``` Tuple counts for ImportTime::class_var_scope#7851b601#fff/3@366258vr after 47ms: 19624 ~1% {1} r1 = SCAN py_Classes OUTPUT In.0 'scope' 296743 ~0% {2} r2 = JOIN r1 WITH Variables::Variable::getScope#dispred#f0820431#ff_10#join_rhs ON FIRST 1 OUTPUT Rhs.1 'var', Lhs.0 'scope' 296743 ~0% {2} r3 = JOIN r2 WITH Variables::LocalVariable#3aa06bbf#f ON FIRST 1 OUTPUT Lhs.0 'var', Lhs.1 'scope' 296743 ~2% {3} r4 = JOIN r3 WITH Variables::Variable::getId#dispred#f0820431#ff ON FIRST 1 OUTPUT Lhs.1 'scope', Rhs.1 'name', Lhs.0 'var' return r4 ``` ``` Tuple counts for ImportTime::global_var_scope#7851b601#fff/3@718e4bpm after 18ms: 108173 ~0% {2} r1 = JOIN Variables::GlobalVariable#class#3aa06bbf#f WITH Variables::Variable::getId#dispred#f0820431#ff ON FIRST 1 OUTPUT Lhs.0 'var', Rhs.1 'name' 108173 ~0% {3} r2 = JOIN r1 WITH Variables::Variable::getScope#dispred#f0820431#ff ON FIRST 1 OUTPUT Lhs.1 'name', Rhs.1 'scope', Lhs.0 'var' return r2 ``` (You may be wondering what's up with the order of arguments for the two helper predicates. By ordering the arguments this way, there's no need to reorder the resulting relations when used in `getOuterVariable.)
1 parent 4101676 commit b5cac92

File tree

1 file changed

+17
-4
lines changed

1 file changed

+17
-4
lines changed

python/ql/lib/semmle/python/types/ImportTime.qll

Lines changed: 17 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -26,9 +26,22 @@ class ImportTimeScope extends Scope {
2626

2727
/** Gets the global variable that is used during lookup, should `var` be undefined. */
2828
GlobalVariable getOuterVariable(LocalVariable var) {
29-
this instanceof Class and
30-
var.getScope() = this and
31-
result.getScope() = this.getEnclosingModule() and
32-
var.getId() = result.getId()
29+
exists(string name |
30+
class_var_scope(this, name, var) and
31+
global_var_scope(name, this.getEnclosingModule(), result)
32+
)
3333
}
3434
}
35+
36+
pragma[nomagic]
37+
private predicate global_var_scope(string name, Scope scope, GlobalVariable var) {
38+
var.getScope() = scope and
39+
var.getId() = name
40+
}
41+
42+
pragma[nomagic]
43+
private predicate class_var_scope(Scope scope, string name, LocalVariable var) {
44+
var.getScope() = scope and
45+
scope instanceof Class and
46+
var.getId() = name
47+
}

0 commit comments

Comments
 (0)