Is there any way to reduce the time it takes to create a julia app with package compiler?
I have found the filter_stdlibs=true arg in the PackageCompiler docs, but I wonder if there’s anything else I can do
The reason for compiling slowly (for any language) is compiling at non-low optimization level (often involving use of inlining, which is on by default in Julia, but I believe disabled by default at lower levels, also settable separately), and/or compiling too much.
I would compile with the latest available Julia version (since they are getting better over time), even 1.10/master for speed. Julia 1.9 RC is around the corner, 1.9 would also be a rather safe bet, and while only stable builds, e.g. 1.8, are recommended for production, I’m recommending newer than that for a non-final build only. If I recall there was some recent improvement for compiling the sysimage, why I’m recommending recent Julia, and explaining:
Julia (when Julia itself is compiled) takes a lot of time compiling its sysimage (basically all the standard library). And I understand since there’s only one sysimage, when you compile your app, you have the (relatively) fixed overhead of triggering compiling all of the standard library as an overhead. I believe there’s no easy solution to strip out parts of the standard library you don’t need, or which parts do not get precompiled. That’s something to look into (one reason I want a cut-down version of Julia, e.g. without LinearAlgebra not all users need). As a stop-gap solution would be to compile at a lower optimization. I don’t belief you could set it separately for your own code than for the standard library (i.e. it’s a global setting, though it IS also settable at a module level).
If you compile often (or just don’t care too much about fastest generated code for your final build) you can run with lower optimization level, e.g. -O0. Then for a final compile build you usually want more optimization, either the default level Julia provides, or even the higher -O3.
I don’t know this really applies to you because while it’s very common for compiled languages to have a debug build that’s fast to compile (but slow at runtime), with Julia you’re more likely to not compile at all until you have your code ready for the final build. In principle the same principle applies for Julia, in case you compile often.
1 Like
Thanks for the reply
I do compile often
I’ve changed the optimization level to 0 as described in here:
I didn’t find much of a difference however
I wouldn’t rule it out as effective, until you tried it the right way. 
It’s just a theory that this will help though.
It’s documented here:
I would it even try with --compile=min -O0 --inline=no --min-optlevel=0 --check-bounds=no
The reason for this way of invoking optimization and other option, is as I suspected, PackageCompiler.jl starts from your julia (with yes, the opt settings you chose, there would only slow down, and you might argue for higher settings, this setting might influence compilation speed (only minimally?), but likely not the resulting binary), but I think it must invoke a new julia as a sub-process, and I’m not sure they inherit the opt settings (probably in some cases, e.g. when using Distributed), here at least you can set it explicitly (to something different), and that setting will affect the opt level of the generated app, plus hopefully the compilation speed.
When I want maximum startup-speed of Julia, I do use --compile=min and min actually means no compilation (the interpreter rater used), so I’m not sure if this actually works for you, try also skipping it and --check-bounds=no which would likely at best provide a minimal speed-up. I think adding bounds checks must be rather fast, and recommended (it’s the default, “auto”, you may want “yes”).
Another option, and you could combine them, is choosing a incremental sysimage build. Non-incremental in the default (70 MB smaller), but likely slower since you actually have to build the sysimage then…
1 Like
Thank you for clarifying.
In the end I went with this:
julia --project=. -e 'using PackageCompiler; create_app("./",".prodcontainer/compiled_app",force=true, filter_stdlibs=false, incremental=true, sysimage_build_args=`--compile=min -O0 --min-optlevel=0 --inline=no --check-bounds=no`)'
I would check with Julia master, since this was merged:
This reduces the Outputting sysimage file… portion from 142s to 20s for me, using 32 cores.
From 454 seconds to 67 seconds, in one fell swoop, you’ve reduced overall build CI times by 30%. (The assert build is even more impressive, from 700 seconds to 101 seconds ).
I think it should help, if you need to generate a sysimage (which is the default, another option is non-incremental).
1 Like
I have noticed an significant reduction in time to create_app(), to almost half. Must be what you shared