Why does runtime dispatch allocate when the return type is inferred?

[ToucheSir]

@Benny

The devdocs doesn’t explain many things and I’m not even sure what a box means anymore, but supposing that boxes are referenced heap-allocated anythings, it seems like the arguments and return value being strictly “pointers to boxes” would explain the 2 allocations

I believe so. The missing info you want is likely Memory layout of Julia Objects · The Julia Language, which explains the layout of the jl_value_t*s that jl_value_t *jl_apply_generic(jl_value_t *F, jl_value_t **args, uint32_t nargs) expects. It’s unfortunate that the unrelated Core.Box also exists to cause confusion, but thankfully that doesn’t show up in any of these examples.

@Benny

Interestingly, the box allocation in (1) seems to be elided if the input was already “boxed” to begin with, like an element of Array{Number} (and not Array{Float64}):

Yes, this was what I was alluding to with my first sentence about “requires boxing…” earlier. Array{Number}, is, roughly speaking, an array of already-boxed jl_value_t*s.

@Benny

What is it about boxes or jl_apply_generic that requires this? Can the information in the boxes really not be put on the caller’s stack for the generic call to read from and write to?

The signature of jl_apply_generic is rather restrictive and makes a number of assumptions around the lifetimes of the jl_value_t **args it receives. Part of the linked PR discussion is whether it’d be safe to “fake” jl_value_t *s that point to the stack instead of GC-allocated and tracked memory—the answer appears to be no, not without non-trivial changes on the GC side.

Regardless of what the best solution is, it’d likely require coordinated changes to codegen and the runtime, as noted in [WIP] dynamic dispatch optimization: avoid boxing stack-allocated inputs by NHDaly · Pull Request #50136 · JuliaLang/julia · GitHub. If I had to guess, this level of complexity is a big part of why it hasn’t been done yet.