I was wondering what are the heuristics (if any) that the compiler uses to decide whether to use the @generated branch instead of the normal branch. The docs remain a little bit vague and only talk about this as a performance optimization, but in a trimming world taking the @generated branch or not might mean being able to compile vs not.
For example, say you have a @generated function. It’s a pretty heavy function and the first compilation takes quite a while, but it’s what makes it possible to actually make the verifier happy.
It’s not a performance sensitive function, and when used in a REPL, it would be more than fine to just use dynamic dispatch and don’t incur in the compilation time of the @generated function.
If I were to change that function to an if @generated body, do I get the best of both worlds:
@generated during trimming, and dynamic dispatch in the REPL
or the worst of both:
dynamic during trimming and @generated in the REPL?
a mix of both? Is this something that can be relied upon?
I think you are confusing concepts here but I can’t quite identify what you are mixing up. Perhaps you could write some code (doesn’t need to work/run) to clarify what scenario is unclear to and what you would like to have instead?
@generated itself has not much to do with dispatch. A @generated function simply creates the actual function body via code the first time it runs (for some specific signature). There are some more details to it but this is the bottomline.
I think the poster is referring to the use of if @generated inside a generated function - see optionally generated functions in the manual. Unfortunately, the documentation doesn’t make it clear when it is actually used.
The non-@generated branch does not imply anything about dynamic dispatch (of multimethods), it’s just the behavior of normal generic function bodies without @generated metaprogramming. The @generated branch also does not guarantee a performance optimization; “can achieve high efficiency” does not imply it must. I could easily write a @generated branch expression that is less efficient (and with no multiple methods here to dispatch):
julia> begin
foo() = if @generated; :(Libc.systemsleep(2); 1) else 1 end
@time foo()
@code_llvm foo()
end
2.023341 seconds (2.62 k allocations: 125.938 KiB, 0.47% compilation time)
; Function Signature: foo()
; @ REPL[18]:2 within `foo`
; Function Attrs: uwtable
define i64 @julia_foo_1742() #0 {
top:
; ┌ @ REPL[18]:2 within `macro expansion`
; │┌ @ libc.jl:164 within `systemsleep`
call x86_stdcallcc void @jlplt_Sleep_1746_got.jit(i32 2000)
; │└
ret i64 1
; └
}
No, which is one of the few things the docs does clarify:
However, which implementation is used depends on compiler implementation details, so it is essential for the two implementations to behave identically.
The compiler is free to change with any patch. Julia could theoretically be implemented without a compiler at all, in which case @generated function calls would be a nightmare of runtime Expr processing and eval.
Presumably, your concern with trimming is about the sentences:
Typically, Julia is able to compile “generic” versions of functions that will work for any arguments, but with generated functions this is impossible. This means that programs making heavy use of generated functions might be impossible to statically compile.
At first glance, that doesn’t seem plausible. JuliaC’s trimming is primarily intended to support statically dispatched programs, which ought to provide the “static” types to normally compile @generated functions, prior to trimming. My guess is that “generic versions” refers to non-monomorphic compilation e.g. @nospecialize, in which case @generated function calls would lack information at compile-time. But I wouldn’t expect that to work with trimming, and the docs don’t actually say what optionally generated functions can be good for. I’ve seen a few cases of devs actually giving up on them because the reference implementation’s compiler chose a slow non-@generated branch instead of just being a niche fallback. I bet they also wished they knew more about the compiler heuristics beforehand.
You’re right, the dynamic dispatch implication is just my actual use case leaking through the description.
Thanks for clarifying. After some extra thought, probably the better approach is mimicking rust’s features with @static and Preferences.jl to manually specify which version is compiled; instead of trying to rely on compiler heuristics. Something like:
function myfunc(...)
@static if juliac
_myfunc_gen()
else
normal body
end
end
I don’t think you would even need @static if, let alone embedded inside methods. Unless there’s some JuliaC-dependent expression that can’t survive to macro-expansion in non-JuliaC contexts, you can just conditionally define a @generated method or a normal method in the global scope:
if juliac()
@generated function myfunc(#=arguments=#)
#=metaprogramming body, long compilation=#
end
else
function myfunc(#=same arguments=#)
#=normal body, quick compilation=#
end
end
But that’s the lesser issue. The main issue is the condition. There isn’t any documented Julia function that checks for JuliaC, let alone trimming, and it’s not actually the opposite of having a REPL. People routinely run scripts from the command line without firing up a REPL, so which definition happens in that case? isinteractive() is true for a REPL, but also third-party “interactive” contexts as generally implemented. Whatever the condition ends up being, you’d have to separately test and debug the implementations to never diverge in results, which is generally much harder to maintain than it appears.
Another issue is you’re also taking a choice away: trimmed binaries won’t opt into dynamic dispatch, but it’s very common to be willing to pay compilation costs for a faster run in the REPL. Typically that’s mitigated by package precompilation shifting the cost to environment instantiation instead of each Julia session, but that only works for call signatures you can confidently specify in advance and reuse in practice. You can still offer both versions as separate functions, though that would be annoying as callees to other functions that either have to diverge as well or take a function input.
There’s no documentation saying that the two branches need to have the same Effects. I agree that’s one way to read the documentation, but if it’s true, I’d argue that if @generated is basically impossible to safely use, because which Effects a piece of code has is not stable, and is the result of compiler knowledge.
Effects inference (again, internal compiler stuff) seems to agree on the effects being different, though I’m not exactly sure what systemsleep is doing:
But section on optionally generated functions really does not say anything about effects. The preceding section on generated functions does say side effects shouldn’t occur, but it’s all about what the @generated method body does in the process of making the Expr, not what happens in the Expr itself. The distinction is worded very poorly though, so I’m not certain; the @generated function documentation is due for a rewrite, imo (what then block??)
Regarding the condition, I did try to use Base.generating_output(false) but that also turned out to not be really suitable. I ended up simply relying on a user defined preference through Preferences.jl, which seems to be the general direction on how to recognize trimming state (not the user defined part, but the preference flag).
And at that point, might as well use @static.
Lots of info here thought, thanks! it’s making me more and more scared of using if @generated now. Ensuring same effects seems like a nightmare to enforce and test.
function myfunc(args...)
if juliac_flag()
myfunc_generated(args...)
else
myfunc_normal(args...)
end
end
So long as the juliac_flag() function is a compile time constant, the compiler will only compile the branch that actually gets hit, and redefining the function will automatically cause it to switch:
julia> @generated function foo(x)
println("compiling foo !")
:(x + 1)
end;
julia> bar(x) = x - 1;
julia> toggle() = false;
julia> function foobar(x)
if toggle()
foo(x)
else
bar(x)
end
end;
julia> foobar(1)
0
to be fair, that issue refers to when the code generation itself errors the else branch will be chosen. if the function is already implemented safely (in that the if @generated and the else branches are semantically identical) then that issue should never cause observable problems, only performance problems
Leveraging world ages and recompilation is convenient for interactivity, but relying on compiler optimizations kind of goes against cshen’s preprocessor-like intent, as much as I have raised questions about it. The good news is even -O0 still does constant propagation and dead code elimination, so it could be fairly reliable in the current implementations even if not technically guaranteed.
The bad news is we have to really watch out for whether the flags are constants. For example, isinteractive() accesses a typed global internally, so it doesn’t work as one:
julia> const _isinteractive = isinteractive()
true
julia> foo() = _isinteractive ? 1 : 2
foo (generic function with 1 method)
julia> @code_llvm foo()
; Function Signature: foo()
; @ REPL[4]:1 within `foo`
; Function Attrs: uwtable
define i64 @julia_foo_1399() #0 {
top:
; @ REPL[4] within `foo`
ret i64 1
}
julia> const _isinteractive = false
false
julia> @code_llvm foo() # recompiled
; Function Signature: foo()
; @ REPL[4]:1 within `foo`
; Function Attrs: uwtable
define i64 @julia_foo_1401() #0 {
top:
; @ REPL[4] within `foo`
ret i64 2
}
Just to add if it’s not obvious already, @static if doesn’t allow this kind of recompilation because the @static macro call evaluates the conditions in the global scope and removes the branch from the input expression. In other words, there’s no constant condition left to compute at compile-time or track for recompilation.
