Skip to content

Commit 99fb305

Browse files
timholytimholy
committed
snoopr ("snoop recompilations") to detect method invalidations
Also provide infrastructure analyze and explain these invalidations. This organizes the call chain as a tree and adds analysis and printing to facilitate understand of the most consequential invalidations.
1 parent cc80f2d commit 99fb305

File tree

9 files changed

+691
-2
lines changed

9 files changed

+691
-2
lines changed

‎Project.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
name = "SnoopCompile"
22
uuid = "aa65fe97-06da-5843-b5b1-d5d13cad87d2"
33
author = ["Tim Holy <tim.holy@gmail.com>"]
4-
version = "1.2.4"
4+
version = "1.3.0"
55

66
[deps]
77
OrderedCollections = "bac558e1-5e72-5ebc-8fee-abe8a469f55d"

‎docs/make.jl

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ makedocs(
77
prettyurls = get(ENV, "CI", nothing) == "true"
88
),
99
modules = [SnoopCompile],
10-
pages = ["index.md", "snoopi.md", "snoopc.md", "userimg.md", "bot.md", "reference.md"]
10+
pages = ["index.md", "snoopi.md", "snoopc.md", "userimg.md", "bot.md", "snoopr.md", "reference.md"]
1111
)
1212

1313
deploydocs(

‎docs/src/index.md

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,13 @@ functions and argument types it's compiling. From these lists of methods,
55
you can generate lists of `precompile` directives that may reduce the latency between
66
loading packages and using them to do "real work."
77

8+
SnoopCompile can also detect and analyze *method cache invalidations*,
9+
which occur when new method definitions alter dispatch in a way that forces Julia to discard previously-compiled code.
10+
Any later usage of invalidated methods requires recompilation.
11+
Invalidation can trigger a domino effect, in which all users of invalidated code also become invalidated, propagating all the way back to the top-level call.
12+
When a source of invalidation can be identified and either eliminated or mitigated,
13+
you can reduce the amount of work that the compiler needs to repeat and take better advantage of precompilation.
14+
815
## Background
916

1017
Julia uses

‎docs/src/snoopr.md

Lines changed: 217 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,217 @@
1+
# Snooping on invalidations: `@snoopr`
2+
3+
## Recording invalidations
4+
5+
```@meta
6+
DocTestFilters = r"(REPL\[\d+\]|none):\d+"
7+
DocTestSetup = quote
8+
using SnoopCompile
9+
end
10+
```
11+
12+
Invalidations occur when there is a danger that new methods would supersede older methods in previously-compiled code.
13+
We can illustrate this process with the following example:
14+
15+
```jldoctest invalidations
16+
julia> f(::Real) = 1;
17+
18+
julia> callf(container) = f(container[1]);
19+
20+
julia> call2f(container) = callf(container);
21+
```
22+
23+
Let's run this with different container types:
24+
```jldoctest invalidations
25+
julia> c64 = [1.0]; c32 = [1.0f0]; cabs = AbstractFloat[1.0];
26+
27+
julia> call2f(c64)
28+
1
29+
30+
julia> call2f(c32)
31+
1
32+
33+
julia> call2f(cabs)
34+
1
35+
```
36+
37+
It's important that you actually execute these methods: code doesn't get compiled until it gets run, and invalidations only affect compiled code.
38+
39+
Now we'll define a new `f` method, one specialized for `Float64`.
40+
So we can see the consequences for the compiled code, we'll make this definition while snooping on the compiler with `@snoopr`:
41+
42+
```jldoctest invalidations
43+
julia> trees = invalidation_trees(@snoopr f(::Float64) = 2)
44+
1-element Array{SnoopCompile.MethodInvalidations,1}:
45+
insert f(::Float64) in Main at REPL[9]:1 invalidated:
46+
backedges: 1: superseding f(::Real) in Main at REPL[2]:1 with MethodInstance for f(::Float64) (2 children) more specific
47+
2: superseding f(::Real) in Main at REPL[2]:1 with MethodInstance for f(::AbstractFloat) (2 children) more specific
48+
```
49+
50+
The list of `MethodInvalidations` indicates that some previously-compiled code got invalidated.
51+
In this case, "`insert f(::Float64)`" means that a new method, for `f(::Float64)`, was added.
52+
There were two proximal triggers for the invalidation, both of which superseded the method `f(::Real)`.
53+
One of these had been compiled specifically for `Float64`, due to our `call2f(c64)`.
54+
The other had been compiled specifically for `AbstractFloat`, due to our `call2f(cabs)`.
55+
56+
You can look at these invalidation trees in greater detail:
57+
58+
```jldoctest invalidations
59+
julia> tree = trees[1];
60+
61+
julia> root = tree.backedges[1]
62+
MethodInstance for f(::Float64) at depth 0 with 2 children
63+
64+
julia> show(root)
65+
MethodInstance for f(::Float64) (2 children)
66+
MethodInstance for callf(::Array{Float64,1}) (1 children)
67+
⋮
68+
69+
julia> show(root; minchildren=0)
70+
MethodInstance for f(::Float64) (2 children)
71+
MethodInstance for callf(::Array{Float64,1}) (1 children)
72+
MethodInstance for call2f(::Array{Float64,1}) (0 children)
73+
```
74+
75+
You can see that the sequence of invalidations proceeded all the way up to `call2f`.
76+
Examining `root2 = tree.backedges[2]` yields similar results, but for `Array{AbstractFloat,1}`.
77+
78+
The structure of these trees can be considerably more complicated. For example, if `callf`
79+
also got called by some other method, and that method had also been executed (forcing it to be compiled),
80+
then `callf` would have multiple children.
81+
This is often seen with more complex, real-world tests:
82+
83+
```julia
84+
julia> trees = invalidation_trees(@snoopr using SIMD)
85+
4-element Array{SnoopCompile.MethodInvalidations,1}:
86+
insert convert(::Type{Tuple{Vararg{R,N}}}, v::Vec{N,T}) where {N, R, T} in SIMD at /home/tim/.julia/packages/SIMD/Am38N/src/SIMD.jl:182 invalidated:
87+
mt_backedges: 1: signature Tuple{typeof(convert),Type{Tuple{DataType,DataType,DataType}},Any} triggered MethodInstance for Pair{DataType,Tuple{DataType,DataType,DataType}}(::Any, ::Any) (0 children) ambiguous
88+
2: signature Tuple{typeof(convert),Type{NTuple{8,DataType}},Any} triggered MethodInstance for Pair{DataType,NTuple{8,DataType}}(::Any, ::Any) (0 children) ambiguous
89+
3: signature Tuple{typeof(convert),Type{NTuple{7,DataType}},Any} triggered MethodInstance for Pair{DataType,NTuple{7,DataType}}(::Any, ::Any) (0 children) ambiguous
90+
91+
insert convert(::Type{Tuple}, v::Vec{N,T}) where {N, T} in SIMD at /home/tim/.julia/packages/SIMD/Am38N/src/SIMD.jl:188 invalidated:
92+
mt_backedges: 1: signature Tuple{typeof(convert),Type{Tuple},Any} triggered MethodInstance for Distributed.RemoteDoMsg(::Any, ::Any, ::Any) (1 children) more specific
93+
2: signature Tuple{typeof(convert),Type{Tuple},Any} triggered MethodInstance for Distributed.CallMsg{:call}(::Any, ::Any, ::Any) (1 children) more specific
94+
3: signature Tuple{typeof(convert),Type{Tuple},Any} triggered MethodInstance for Distributed.CallMsg{:call_fetch}(::Any, ::Any, ::Any) (1 children) more specific
95+
4: signature Tuple{typeof(convert),Type{Tuple},Any} triggered MethodInstance for Distributed.CallWaitMsg(::Any, ::Any, ::Any) (4 children) more specific
96+
12 mt_cache
97+
98+
insert <<(x1::T, v2::Vec{N,T}) where {N, T<:Union{Bool, Int128, Int16, Int32, Int64, Int8, UInt128, UInt16, UInt32, UInt64, UInt8}} in SIMD at /home/tim/.julia/packages/SIMD/Am38N/src/SIMD.jl:1061 invalidated:
99+
mt_backedges: 1: signature Tuple{typeof(<<),UInt64,Any} triggered MethodInstance for <<(::UInt64, ::Integer) (0 children) ambiguous
100+
2: signature Tuple{typeof(<<),UInt64,Any} triggered MethodInstance for copy_chunks_rtol!(::Array{UInt64,1}, ::Integer, ::Integer, ::Integer) (0 children) ambiguous
101+
3: signature Tuple{typeof(<<),UInt64,Any} triggered MethodInstance for copy_chunks_rtol!(::Array{UInt64,1}, ::Int64, ::Int64, ::Integer) (0 children) ambiguous
102+
4: signature Tuple{typeof(<<),UInt64,Any} triggered MethodInstance for copy_chunks_rtol!(::Array{UInt64,1}, ::Integer, ::Int64, ::Integer) (0 children) ambiguous
103+
5: signature Tuple{typeof(<<),UInt64,Any} triggered MethodInstance for <<(::UInt64, ::Unsigned) (16 children) ambiguous
104+
20 mt_cache
105+
106+
insert +(s1::Union{Bool, Float16, Float32, Float64, Int128, Int16, Int32, Int64, Int8, UInt128, UInt16, UInt32, UInt64, UInt8, Ptr}, v2::Vec{N,T}) where {N, T<:Union{Float16, Float32, Float64}} in SIMD at /home/tim/.julia/packages/SIMD/Am38N/src/SIMD.jl:1165 invalidated:
107+
mt_backedges: 1: signature Tuple{typeof(+),Ptr{UInt8},Any} triggered MethodInstance for handle_err(::JuliaInterpreter.Compiled, ::JuliaInterpreter.Frame, ::Any) (0 children) ambiguous
108+
2: signature Tuple{typeof(+),Ptr{UInt8},Any} triggered MethodInstance for #methoddef!#5(::Bool, ::typeof(LoweredCodeUtils.methoddef!), ::Any, ::Set{Any}, ::JuliaInterpreter.Frame) (0 children) ambiguous
109+
3: signature Tuple{typeof(+),Ptr{UInt8},Any} triggered MethodInstance for #get_def#94(::Set{Tuple{Revise.PkgData,String}}, ::typeof(Revise.get_def), ::Method) (0 children) ambiguous
110+
4: signature Tuple{typeof(+),Ptr{Nothing},Any} triggered MethodInstance for filter_valid_cachefiles(::String, ::Array{String,1}) (0 children) ambiguous
111+
5: signature Tuple{typeof(+),Ptr{Union{Int64, Symbol}},Any} triggered MethodInstance for pointer(::Array{Union{Int64, Symbol},N} where N, ::Int64) (1 children) ambiguous
112+
6: signature Tuple{typeof(+),Ptr{Char},Any} triggered MethodInstance for pointer(::Array{Char,N} where N, ::Int64) (2 children) ambiguous
113+
7: signature Tuple{typeof(+),Ptr{_A} where _A,Any} triggered MethodInstance for pointer(::Array{T,N} where N where T, ::Int64) (4 children) ambiguous
114+
8: signature Tuple{typeof(+),Ptr{Nothing},Any} triggered MethodInstance for _show_default(::IOContext{Base.GenericIOBuffer{Array{UInt8,1}}}, ::Any) (49 children) ambiguous
115+
9: signature Tuple{typeof(+),Ptr{Nothing},Any} triggered MethodInstance for _show_default(::Base.GenericIOBuffer{Array{UInt8,1}}, ::Any) (336 children) ambiguous
116+
10: signature Tuple{typeof(+),Ptr{UInt8},Any} triggered MethodInstance for pointer(::String, ::Integer) (1027 children) ambiguous
117+
2 mt_cache
118+
```
119+
120+
Your specific output will surely be different from this, depending on which packages you have loaded,
121+
which versions of those packages are installed, and which version of Julia you are using.
122+
In this example, there were four different methods that triggered invalidations, and the invalidated methods were in `Base`,
123+
`Distributed`, `JuliaInterpeter`, and `LoweredCodeUtils`. (The latter two were a consequence of loading `Revise`.)
124+
You can see that collectively more than a thousand independent compiled methods needed to be invalidated; indeed, the last
125+
entry alone invalidates 1027 method instances:
126+
127+
```
128+
julia> sig, node = trees[end].mt_backedges[10]
129+
Pair{Any,SnoopCompile.InstanceTree}(Tuple{typeof(+),Ptr{UInt8},Any}, MethodInstance for pointer(::String, ::Integer) at depth 0 with 1027 children)
130+
131+
julia> node
132+
MethodInstance for pointer(::String, ::Integer) at depth 0 with 1027 children
133+
134+
julia> show(node)
135+
MethodInstance for pointer(::String, ::Integer) (1027 children)
136+
MethodInstance for repeat(::String, ::Integer) (1023 children)
137+
MethodInstance for ^(::String, ::Integer) (1019 children)
138+
MethodInstance for #handle_message#2(::Nothing, ::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::typeof(Base.CoreLogging.handle_message), ::Logging.ConsoleLogger, ::Base.CoreLogging.LogLevel, ::String, ::Module, ::Symbol, ::Symbol, ::String, ::Int64) (906 children)
139+
MethodInstance for handle_message(::Logging.ConsoleLogger, ::Base.CoreLogging.LogLevel, ::String, ::Module, ::Symbol, ::Symbol, ::String, ::Int64) (902 children)
140+
MethodInstance for log_event_global!(::Pkg.Resolve.Graph, ::String) (35 children)
141+
⋮
142+
MethodInstance for #artifact_meta#20(::Pkg.BinaryPlatforms.Platform, ::typeof(Pkg.Artifacts.artifact_meta), ::String, ::Dict{String,Any}, ::String) (43 children)
143+
⋮
144+
MethodInstance for Dict{Base.UUID,Pkg.Types.PackageEntry}(::Dict) (79 children)
145+
⋮
146+
MethodInstance for read!(::Base.Process, ::LibGit2.GitCredential) (80 children)
147+
⋮
148+
MethodInstance for handle_err(::JuliaInterpreter.Compiled, ::JuliaInterpreter.Frame, ::Any) (454 children)
149+
⋮
150+
⋮
151+
⋮
152+
⋮
153+
⋮
154+
⋮
155+
```
156+
157+
Many nodes in this tree have multiple "child" branches.
158+
159+
## Avoiding or fixing invalidations
160+
161+
Invalidations occur in situations like our `call2f(c64)` example, where we changed our mind about what value `f` should return for `Float64`.
162+
Julia could not have returned the newly-correct answer without recompiling the call chain.
163+
164+
Aside from cases like these, most invalidations occur whenever new types are introduced,
165+
and some methods were previously compiled for abstract types.
166+
In some cases, this is inevitable, and the resulting invalidations simply need to be accepted as a consequence of a dynamic, updateable language.
167+
(You can often minimize invalidations by loading all your code at the beginning of your session, before triggering the compilation of more methods.)
168+
However, in many circumstances an invalidation indicates an opportunity to improve code.
169+
In our first example, note that the call `call2f(c32)` did not get invalidated: this is because the compiler
170+
knew all the specific types, and new methods did not affect any of those types.
171+
The main tips for writing invalidation-resistant code are:
172+
173+
- use [concrete types](https://docs.julialang.org/en/latest/manual/performance-tips/#man-performance-abstract-container-1) wherever possible
174+
- write inferrable code
175+
- don't engage in [type-piracy](https://docs.julialang.org/en/latest/manual/style-guide/#Avoid-type-piracy-1) (our `c64` example is essentially like type-piracy, where we redefined behavior for a pre-existing type)
176+
177+
Since these tips also improve performance and allow programs to behave more predictably,
178+
these guidelines are not intrusive.
179+
Indeed, searching for and eliminating invalidations can help you improve the quality of your code.
180+
In cases where invalidations occur, but you can't use concrete types (there are many valid uses of `Vector{Any}`),
181+
you can often prevent the invalidation using some additional knowledge.
182+
For example, suppose you're writing code that parses Julia's `Expr` type:
183+
184+
```julia
185+
julia> ex = :(Array{Float32,3})
186+
:(Array{Float32, 3})
187+
188+
julia> dump(ex)
189+
Expr
190+
head: Symbol curly
191+
args: Array{Any}((3,))
192+
1: Symbol Array
193+
2: Symbol Float32
194+
3: Int64 3
195+
```
196+
197+
`ex.args` is a `Vector{Any}`.
198+
However, for a `:curly` expression only certain types will be found among the arguments; you could write key portions of your code as
199+
200+
```
201+
a = ex.args[2]
202+
if a isa Symbol
203+
# inside this block, Julia knows `a` is a Symbol, and so methods called on `a` will be resistant to invalidation
204+
foo(a)
205+
elseif a isa Expr && length((a::Expr).args) > 2
206+
a = a::Expr # sometimes you have to help inference by adding a type-assert
207+
x = bar(a) # `bar` is now resistant to invalidation
208+
elsef a isa Integer
209+
# even though you've not made this fully-inferrable, you've at least reduced the scope for invalidations
210+
# by limiting the subset of `foobar` methods that might be called
211+
y = foobar(a)
212+
end
213+
```
214+
215+
Adding type-assertions and fixing inference problems are the most common approaches for fixing invalidations.
216+
You can discover these manually, but the [Cthulhu](https://github.com/JuliaDebug/Cthulhu.jl) package is highly recommended.
217+
Cthulu's `ascend`, in particular, allows you to navigate an invalidation tree and focus on those branches with the most severe consequences (frequently, the most children).

‎examples/invalidations_blog.jl

Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,56 @@
1+
using SnoopCompile
2+
using SnoopCompile: countchildren
3+
4+
function hastv(typ)
5+
isa(typ, UnionAll) && return true
6+
if isa(typ, DataType)
7+
for p in typ.parameters
8+
hastv(p) && return true
9+
end
10+
end
11+
return false
12+
end
13+
14+
trees = invalidation_trees(@snoopr using Revise)
15+
16+
function summary(trees)
17+
npartial = ngreater = nlesser = nambig = nequal = 0
18+
for methodtree in trees
19+
method = methodtree.method
20+
invs = methodtree.invalidations
21+
for fn in (:mt_backedges, :backedges)
22+
list = getfield(invs, fn)
23+
for item in list
24+
sig = nothing
25+
if isa(item, Pair)
26+
sig = item.first
27+
item = item.second
28+
else
29+
sig = item.mi.def.sig
30+
end
31+
# if hastv(sig)
32+
# npartial += countchildren(invtree)
33+
# else
34+
ms1, ms2 = method.sig <: sig, sig <: method.sig
35+
if ms1 && !ms2
36+
ngreater += countchildren(item)
37+
elseif ms2 && !ms1
38+
nlesser += countchildren(item)
39+
elseif ms1 && ms2
40+
nequal += countchildren(item)
41+
else
42+
# if hastv(sig)
43+
# npartial += countchildren(item)
44+
# else
45+
nambig += countchildren(item)
46+
# end
47+
end
48+
# end
49+
end
50+
end
51+
end
52+
@assert nequal == 0
53+
println("$ngreater | $nlesser | $nambig |") # $npartial |")
54+
end
55+
56+
summary(trees)

‎src/SnoopCompile.jl

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -21,4 +21,8 @@ include("parcel_snoopc.jl")
2121
include("write.jl")
2222
include("bot.jl")
2323

24+
if VERSION >= v"1.6.0-DEV.154"
25+
include("invalidations.jl")
26+
end
27+
2428
end # module

0 commit comments

Comments
 (0)