`Base.specialize_method` CodeInstance not saved in system image

[Keno]

Seen on at least these two PRs:
#58830
#58768

From worker 9:	┌ Info: REPL: trace compile output:
--
  | From worker 9:	│ precompile(Tuple{typeof(Core.kwcall), NamedTuple{(:preexisting,), Tuple{Bool}}, typeof(Base.specialize_method), Method, Any, Core.SimpleVector})
  | From worker 9:	└ precompile(Tuple{typeof(Base.setindex!), Base.Dict{Any, Any}, Any, Char})

From worker 6:	┌ Info: REPL: trace compile output:
--
  | From worker 6:	│ precompile(Tuple{typeof(Core.kwcall), NamedTuple{(:preexisting,), Tuple{Bool}}, typeof(Base.specialize_method), Method, Any, Core.SimpleVector})
  | From worker 6:	└ precompile(Tuple{REPL.var"#61#62"{REPL.LineEditREPL, REPL.REPLBackendRef}}) # recompile

I'm primarily concerned about the Base.specialize_method one - the other one could plausibly be related to the individual PRs, but the Base.specialize_method one is in the compiler and seems to get triggered every time something is inferred. At the very least that's a latency bug.

[Keno]

Ok, so looks like there's two applicable specializations for this method:

m = which(Tuple{typeof(Core.kwcall), NamedTuple{(:preexisting,), Tuple{Bool}}, typeof(Base.specialize_method), Method, Any, Core.SimpleVector})
println("Any specialization")
mi = m.specializations[1]
@show mi
if isdefined(mi, :cache)
    @show mi.cache.min_world
    @show mi.cache.max_world
end
println("Type specialization")
mi = m.specializations[2]
@show mi
if isdefined(mi, :cache)
    @show mi.cache.min_world
    @show mi.cache.max_world
end

./julia --compile=no test.jl
Any specialization
mi = MethodInstance for Core.kwcall(::@NamedTuple{preexisting::Bool}, ::typeof(Base.specialize_method), ::Method, ::Any, ::Core.SimpleVector)
Type specialization
mi = MethodInstance for Core.kwcall(::@NamedTuple{preexisting::Bool}, ::typeof(Base.specialize_method), ::Method, ::Type, ::Core.SimpleVector)
mi.cache.min_world = 0x0000000000002e11
mi.cache.max_world = 0xffffffffffffffff

Apparently during sysimage generation we only ever touch the second one, but as soon as we reload the sysimage, we're primarily using the first one.

[Keno]

So for some reason, in the inference world, there's an :invoke there for the Any version, although it did infer the Type version and while building the sysimage it does use the Type version. Still not sure why.

julia> mm = Base._which(Tuple{typeof(gcd), Int, Int})
Core.MethodMatch(Tuple{typeof(gcd), Int64, Int64}, svec(Int64), gcd(a::T, b::T) where T<:Union{Int128, Int16, Int32, Int64, Int8, UInt128, UInt16, UInt32, UInt64, UInt8} @ Base intfuncs.jl:53, true)

julia> @code_typed world=Base.unsafe_load(cglobal(:jl_typeinf_world, Csize_t)) Base.specialize_method(mm; preexisting=true)
CodeInfo(
1 ─ %1 = %new(Base.Pairs{Symbol, Bool, Tuple{Symbol}, @NamedTuple{preexisting::Bool}}, @_2, (:preexisting,))::Base.Pairs{Symbol, Bool, Tuple{Symbol}, @NamedTuple{preexisting::Bool}}
│   %2 =    invoke Base.:(var"#specialize_method#10")(%1::Base.Pairs{Symbol, Bool, Tuple{Symbol}, @NamedTuple{preexisting::Bool}}, @_3::typeof(Base.specialize_method), match::Core.MethodMatch)::Union{Nothing, Core.MethodInstance}
└──      return %2
) => Union{Nothing, Core.MethodInstance}

julia> Base.code_typed_by_type(Tuple{typeof(Base.:(var"#specialize_method#10")), Base.Pairs{Symbol, Bool, Tuple{Symbol}, @NamedTuple{preexisting::Bool}}, typeof(Base.specialize_method), Core.MethodMatch}; world=Base.unsafe_load(cglobal(:jl_typeinf_world, Csize_t)))
1-element Vector{Any}:
 CodeInfo(
1 ─ %1 =   builtin Base.getfield(kwargs, :data)::@NamedTuple{preexisting::Bool}
│   %2 =   builtin Base.getfield(match, :method)::Method
│   %3 =   builtin Base.getfield(match, :spec_types)::Type
│   %4 =   builtin Base.getfield(match, :sparams)::Core.SimpleVector
│   %5 =    invoke Core.kwcall(%1::@NamedTuple{preexisting::Bool}, Base.specialize_method::typeof(Base.specialize_method), %2::Method, %3::Any, %4::Core.SimpleVector)::Union{Nothing, Core.MethodInstance}
└──      return %5
) => Union{Nothing, Core.MethodInstance}

[Keno]

Alright, I understand what's happening here. The expected situation is that the Type specialization gets invoked and the Any specialization never runs. However, at some point during the sysimage build, well after the typeinf_world, specialize_method gets invalidated. This shouldn't be a problem because it should keep running in the typeinf_world. However,

julia/src/staticdata.c

Line 926 in d48a675

else if (jl_atomic_load_relaxed(&ci->max_world) != ~(size_t)0 || // delete all code that cannot run

then decides to discard it. When we then re-infer everything to build the native code for sysimage generation, this discard pushes inlining into the bail path (because the code for inlining got discarded), which then chooses the Any signature as the compilesig. However, that signature never ran during the sysimage build, so we don't have native code for it. I think the immediate fix is to not discard code instances that may be runnable in the typeinf world. In the future, @vtjnash 's changes in the #58662 sequence might change this again.

[Keno]

I think this also explains various other weird allocations and dynamic dispatches in inference that show up in profiles that I've been meaning to track down.