Error within compiler/typeinfer.jl (from UB?)

[tecosaur]

Last night I made a type-stable Memoize.jl via a @generated function.

Along the way, I tried something that may well be considered undefined behavior, but was surprised to see a compiler/typeinfer.jl typeassert error regardless.

julia> @generated function ohnoes(::F, args...) where {F <: Function}
       Core.Compiler.return_type(F.instance, Tuple{args...}, Base.get_world_counter())
       end
ohnoes (generic function with 1 method)

julia> ohnoes(sum, [1,2,3])
ERROR: TypeError: in typeassert, expected Vector, got a value of type Nothing
Stacktrace:
 [1] return_type(interp::Core.Compiler.NativeInterpreter, t::DataType)
   @ Core.Compiler ./compiler/typeinfer.jl:1187
 [2] return_type(t::DataType, world::UInt64)
   @ Core.Compiler ./compiler/typeinfer.jl:1175
 [3] return_type(f::Any, t::DataType, world::UInt64)
   @ Core.Compiler ./compiler/typeinfer.jl:1165
 [4] #s1#11
   @ ./REPL[40]:2 [inlined]
 [5] var"#s1#11"(F::Any, ::Any, ::Any, args::Any)
   @ Main ./none:0
 [6] (::Core.GeneratedFunctionStub)(::UInt64, ::LineNumberNode, ::Any, ::Vararg{Any})
   @ Core ./boot.jl:707
 [7] top-level scope
   @ REPL[41]:1

I'm aware that doing word-age stuff within a @generated function is very much not-blessed (not sure if it's explicitly UB from the docs though), but I'm still surprised to see a typeassert error from the compiler.

If there's an invariant the compiler is assuming, I would have thought it would be explicitly checked for, rather than hitting a typeassert at some point.

The line in question is a _methods_by_ftype call, which ends up ccall-ing jl_matching_methods. The ccall itself is typed to return a Union{Vector{Any},Nothing}, but in 10 places (mostly reflection.jl, also bootstrap.jl and typeinfer.jl) a _methods_by_ftype call is asserted to be a Vector, discarding the Nothing case.

Perhaps there's something here that should be more explicitly checked for/errored?

[vtjnash]

It is explicitly UB

[KristofferC]

but I'm still surprised to see a typeassert error from the compiler.

I am curious as to why. An assert failure seems like one of the best scenarios you could run into when you do something like this.

[tecosaur]

From the discussion on Slack this seems to come down to an expectation mismatch more than anything else. I think of type asserts as something applied for internal correctness checks, with dedicated error paths for things a user might do.

[Seelengrab]

It is explicitly UB

Should this be added to #54099, which will hopefully be merged some day?

[vtjnash]

It is already documented that interacting with Core.Compiler in any way is expected to trigger UB, and I thought it mentioned the meaning of "pure" a bit more clearly (e.g. referencing the prohibition on returning any newly allocated mutable objects), but it seems to assume the reader knows that "pure" is a superset of the "consistent" assume_effect, but it pre-dates the documentation for assume_effect: https://docs.julialang.org/en/v1/manual/metaprogramming/#Generated-functions

[Seelengrab]

Yes, I was referring to explicitly mention this in the UB PR as well, so that people don't have to hunt for multiple places that define what is UB :)

[KristofferC]

Closing this as this not being unexpected when doing UB. Someone could add a ref to it to #54099 if so desire.