`--trace-compile` ignores inferred const-return Methods

[topolarity]

If you write a method that ends up being inferred to a Const(...) return value:

julia> const k = rand(1:100)
61
julia> foo() = *(k, 15, 12)
julia> @code_typed foo()
CodeInfo(
1 ─     return 10980
) => Int64

it will not be included in the --trace-compile output:

$ julia --trace-compile=stderr -e "const k = rand(1:100); (foo() = *(k, 15)); println(foo())"
precompile(Tuple{typeof(Base.rand), Base.UnitRange{Int64}})
precompile(Tuple{typeof(Base.println), Int64})
precompile(Tuple{typeof(Base.println), Base.TTY, Int64})
705

Even though we ran our compilation pipeline on foo() for the first time (generating a new CodeInstance), it ended up not needing to compile via LLVM so it is excluded from the output.

A user may hope to use the output of --trace-compile to determine whether they should add precompile(foo, ()) to the pre-compilation workload, If we had reported this MethodInstance, it would have saved them that inference time.

[Keno]

I think it makes sense for --trace-compile to include these. Possibly we may want additional filtering options later.

[vtjnash]

I believe we deprecated trace-compile for this purpose a while ago because of various reasons like this, replacing it with trace-dispatch for users that need more correctness

[topolarity]

I'm not sure --trace-dispatch is a very good replacement for this use case

It doesn't help you to distinguish between whether you are dispatching to existing code or if you are compiling brand new code, which is very useful, e.g., for hunting down invalidations and caching bugs.

[topolarity]

There's actually an additional wrinkle here, which is that these signatures may not be "compileable".

These work fine:

julia> precompile(Tuple{typeof(Base.tail), Tuple{Int}})
true
julia> precompile(Tuple{typeof(Base.isempty), Tuple{Base.SubString{String}}})
true
julia> precompile(Tuple{typeof(Base.haslength), Base.Dict{Int, Nothing}})
true

but these do not:

julia> precompile(Tuple{typeof(Base.tail), Tuple{Any}})
false
julia> precompile(Tuple{typeof(Base.isempty), Tuple{Vararg{Base.SubString{String}}}})
false
julia> precompile(Tuple{typeof(Base.haslength), Base.Dict{_A, Nothing} where _A})
false

The false means that we refused to pre-compile this, in particular because the JIT does not expect to be able to compile these abstract signatures. Its default specialization behavior says that they need to be more specific.

The irony is that the signatures are, in fact, easily compiled to the most precise inference / performant codegen that we have (a side-effect-free function w/ a Const(...) result), but precompile(...) has no way to know that until after it runs inference.

That said, even if precompile(...) did run on these, I don't think these are actually useful to cache right now. Inference can use the results, but dispatch / JIT won't look for code (or a const-return) attached to a widened signature like this. We can probably leave these non-"compileable" signatures out of the --trace-compile output for now, but I wanted to mention it for awareness.

[NHDaly]

Yes, leaving those out makes sense to me, since presumably they will be compiled (and cached?) as part of compiling whatever other functions reached them, during those other functions' compilations?

[topolarity]

Yes, leaving those out makes sense to me, since presumably they will be compiled (and cached?) as part of compiling whatever other functions reached them, during those other functions' compilations?

Yeah, that sounds right to me. In principle, we only need to log the compiled functions that were also the target of dispatches (I think the most compact log would be an intersection of --trace-dispatch and --trace-compile, so that you only see 'roots' of compilation work and you only consider compilation that was done 'freshly' this session)

[NHDaly]

Agreed, except that I think --trace-compile should already be only a strict subset of --trace-dispatch, since I think we only log it during dispatch (except for the cases that you're identifying in this ticket, where we don't do it).

[vtjnash]

Right, I think this PR just makes --trace-compile a much worse version (unreliable, unstable, inaccurate) of --trace-dispatch. I say we just delete this option and call it done.

[KristofferC]

I say we just delete this option and call it done.

For some history, the printing of precompile signatures used to be done by defining the TRACE_COMPILE variable and was used to get some basic precompile signatures for the sysimage. In a series of PRs that were intended to improve Julia latency:

#28075
#28118
#28371
#28419

this evolved to the --trace-compile workflow we all know and love ;).

Now, the point of that is that the only reason --trace-compile is used is to generate signatures to improve latency. If more things should be included in it, then include more things in it. If it should alias --trace-dispatch for best behavior then do so. Arguing what is semantically compilation and not etc is not really useful, anything that is of use to reduce latency for a given workload should be in it. But there is no need to "delete" it (would be breaking), just improve it (or alias it).

[vtjnash]

I think at this point, all parts of all those PRs related to trace-compile have been removed due to the problems with trace-compile

[topolarity]

Agreed, except that I think --trace-compile should already be only a strict subset of --trace-dispatch, since I think we only log it during dispatch (except for the cases that you're identifying in this ticket, where we don't do it).

I don't think that's the case - this ticket is making me realize that I can't precisely say what the definition of --trace-dispatch is either, but it sometimes reports elements that have already been compiled:

$ julia +nightly --trace-dispatch=stderr -e "Core.invoke(Base.getindex, Tuple{Vector{Int}, Int}, Int[1,2,3], 1)"
precompile(Tuple{typeof(Base.getindex), Array{Int64, 1}, Int64})
$ julia +nightly --trace-compile=stderr -e "Core.invoke(Base.getindex, Tuple{Vector{Int}, Int}, Int[1,2,3], 1)"
# no output

It also does not report some dynamic dispatches, depending on where there are found in the cache:

julia +nightly --trace-dispatch=stderr -e "v = Int[1,2,3]; (foo() = v[1]); foo()"
precompile(Tuple{typeof(Main.foo)})
# no entry for: precompile(Tuple{typeof(getindex), Vector{Int}, Int})

despite having a dynamic dispatch:

$ julia +nightly -e "using InteractiveUtils; v = Int[1,2,3]; (foo() = v[1]); display(@code_typed foo())"
CodeInfo(
1 ─ %1 = Main.v::Any
│   %2 =   dynamic Base.getindex(%1, 1)::Any
└──      return %2
) => Any

That's not an issue for pre-compilation lists / reducing compilation latency, but it could be considered a bug if you are using --trace-dispatch to look for dynamic dispatches (do we care about that use case?). Anyway, I'll see what can be done to align these.