I’d like to introduce new package AutoSysimages.jl that aims to automate building process of sysimages. It records precompile statements during normal Julia usage. Then, building a user-specific system image for your project is possible by a single command.
Please find more details in the documentation, or in my recent post:
https://forem.julialang.org/petvana/introducing-autosysimagesjl-gsoc22-3pe8
Tools like that are incredibly useful @petvana , thank you! 
Many thanks for the tool and the wonderful Forem article!
Does your pkg speed up the startup of Pluto notebooks?
That is a nice idea, I wish we had some precompilation in Pluto, some button to press and precompile the workflow.
Not yet, but we can try to integrate it into Pluto.
Oh nice!
Sounds similar to Fezzik.jl (GitHub - TsurHerman/Fezzik: Brute force sysimg building for Julia)
I’ve just tried the Pluto, and it seems to be able to reduce the loading just out-of-the-box.
(Notice, the compilation takes only 60s because I use the very experimental chained build.)
Hi, I gave a try on it but apparently select_packages() is unable to find packages in dev mode.
This is intentional that dev packages are not allowed to be included because such packages are not static and you would need to rebuild the sysimage after any change. Thus, it seems reasonable to include only released packages.
It is possible to revise packages that are compiled into sysimage with Track packages compiled into sysimage by ktdq · Pull Request #688 · timholy/Revise.jl · GitHub
I’m running with dev packages compiled in and startup latencies are gone.
I’d like to get a version that we would call ’juliacompile’ with no questions asked. Then running a script could look like this:
juliacompile benchmark.jl
julia benchmark.jl
No questions should be asked for non-REPL (noninteractive) usage. Thus the following should work (once you create the sysimage). Or you can rename the startup script…
asysimg benchmark.jl
julia benchmark.jl
The only difference for now is that it prints the @info. We can suppress it in non-REPL mode in the future.
Let’s call it the compile step for this.
Do I have to add AutoSysimages to my local project, or is it sufficient if I add it to the global environment?
It should be sufficient to add AutoSysimages into the global environment. Also, AutoSysimages is automatically included to the sysimage for faster startup, unless AutoSysimages is in dev mode.
To be honest, I do not see ANY effect from using AutoSysimages.
What I did (on Ubuntu):
asysimg>
[ Info: AutoSysimages: Copy snooped statements to: /home/ufechner/.julia/asysimg/1.8.1/23RTv7/snoop-file.jl
[ Info: There is no sysimage for this project. Do you want to build one?
> Yes
No
I press enter and select No for the next question (do you want to select packages).
Now it builds a system image.
[ Info: AutoSysimages: Copy snooped statements to: /home/ufechner/.julia/asysimg/1.8.1/23ZRuw/snoop-file.jl
[ Info: There is no sysimage for this project. Do you want to build one?
> Yes
No
[ Info: AutoSysimages: Collecting precompile statements for empty run (-e "")
[ Info: AutoSysimages: Building system image by PackageCompiler.
⠇ [00m:10s] PackageCompiler: compiling incremental system image┌ Warning: There is no statement file
└ @ Main.anonymous /home/ufechner/.julia/packages/AutoSysimages/LTaGS/src/precompile-PackageCompiler.jl:44
✔ [02m:25s] PackageCompiler: compiling incremental system image
[ Info: AutoSysimages: Builded in 147.87036108970642 s
If I now launch julia again and run the same include statement the time to first plot is 17s as before:
ufechner@TUD277255:~/repos/Plotting$ asysimg --project
┌ Info: The package AutoSysimages.jl started!
│ Loaded sysimage: Default (You may run AutoSysimages.build_sysimage())
│ Active directory: /home/ufechner/.julia/asysimg/1.8.1/23ZRuw
│ Global snoop file: /home/ufechner/.julia/asysimg/1.8.1/23ZRuw/snoop-file.jl
└ Tmp. snoop file: /tmp/jl_gJmh8ds5yB-snoop.jl
_
_ _ _(_)_ | Documentation: https://docs.julialang.org
(_) | (_) (_) |
_ _ _| |_ __ _ | Type "?" for help, "]?" for Pkg help.
| | | | | | |/ _` | |
| | |_| | | | (_| | | Version 1.8.1 (2022-09-06)
_/ |\__'_|_|_|\__'_| | Official https://julialang.org/ release
|__/ |
asysimg> @time include("src/plot.jl")
To zoom, draw a rectangle while keeping the right mouse key pressed!
To go back to see the full graph, press <CRTRL><f>
17.618484 seconds (43.41 M allocations: 2.397 GiB, 4.78% gc time, 71.93% compilation time: 12% of which was recompilation)
You can find and clone the example project I used here: GitHub - ufechner7/Plotting
My impression is that it creates a sysimage, but it does not use it later.
Any idea?
It determines the active directory /home/ufechner/.julia/asysimg/1.8.1/23ZRuw correctly, but the sysimage is not loaded (see, Loaded sysimage: Default). I’ll look at it, and moved the discussion to Unable to save/load sysimage · Issue #5 · petvana/AutoSysimages.jl · GitHub. Thank you for the detailed report.
But that means a developer cannot test to compile its own package. And all developers have more than one Julia version installed, so he/she could develop in one version and try/use it in another.
You can explicitly add them into the include list as in the documentation:
“Notice dev packages are excluded unless they are in include list.”
Moreover, you can include all the dependencies which should reduce the loading time in most of the cases.