>
> The following PR introduces features helping compile and use custom system image…s for LanguageServer and SymbolServer. The expected benefits are reduced start-up times for the language server. On my (old-ish) machine, start-up times are reduced from 20 seconds to less than 2. On a newer machine where I also tested this, times went from 15 seconds down to less than 1.
>
> The idea is the following:
> \- \`eglot-jl.jl\` looks at the \`EGLOT\_JL\_TEST\` environment variable; when it is set, the server is configured to read from an IOBuffer that sends it an \`exit\` instruction as soon as it is able to process requests. This allows \`eglot-jl.jl\` to be used as a \`precompile\_execution\_file\` that exercises the (language) server before gracefully exiting.
> \- a \`compile.jl\` script automates the generation of a system image, named after the julia version (and having the relevant extension depending on the OS).
>
> This is what the first commit in this PR introduces, alongside with instructions in \`README.org\` explaining how to compile the system image, and set \`eglot-jl-julia-args\` to use it.
>
> The second commit goes a step further: it automates the search for a suitable system image at server startup, in order to use it if one is found.
> \- a first call to julia allows getting the version number. This involves something like 0.1s additional latency before the server is run, which I think is acceptable but is nevertheless quite a large amount of time for such an insignificant-looking task
> \- a suitable system image is looked for, based on the version number; if one is found, the corresponding \`--sysimage\` command-line switch is generated. If not, nothing happens; no system image is used, so users who have not generated any system image will not be impacted by this.
> \- the whole process handling system images can be deactivated using a new \`eglot-jl-enable-sysimage\` customizable option. This allows users to opt out of the system, even if they have previously generated a system image (for example in case it would become stale for some reason). It also completely removes the julia version overhead.
>
> Additionally, a new autoloaded command allows running the system image generation script from within emacs. The same command could be used to re-generate a system image if the previous one caused problems for any reason.
>
> This is documented in \`README.org\`. I've tested this with Julia 1.3 and 1.4, on Windows and Linux, without encountering any problem (I don't have access to a Mac).
>
> I know you were reluctant to handle system images in \`eglot-jl\`, and were (rightfully) especially concerned about possible ways that a system image would become obsolete. I tried making this implementation as robust as possible, but I might very well have overlooked something. So please do not hesitate to tell me if something bothers you with this proposal.
However the whole system is indeed a bit fragile; it works well in this case, but only because in `eglot-jl` we completely control the environment we want to work in, and the command-line flags provided to Julia. I wonder if such a technique could be robustly implemented in a more general context, but unfortunately I’ve never really taken the time to think it through…
In any case, I’m looking forward to reading what others have to say about this!
---
### Author:  [@airpmb](https://discourse.julialang.org/u/airpmb)
#### Post date: [January 29, 2021, 8:07pm UTC](https://discourse.julialang.org/t/invalidating-sysimages/53922/3 "2021-01-29T20:07:37Z")
Thanks, those are some useful pointers. It sounds like there’s some work ongoing that in the longer term can really improve sysimage workflows.
In the meantime I think what I might try is a simple experiment in my own package with at script that saves a snapshot of Manifest.toml as it was when I ran PackageCompiler last and if there has been a change it will re-run PackageCompiler.
It’s actually so simple that I suspect others are already doing this but I might be missing some nuance… OTOH I also wonder whether any and all changes to Manifest.toml should trigger a rebuild or only packages that I list in the call to `create_sysimage`.
---
### Author:  [@ericphanson](https://discourse.julialang.org/u/ericphanson)
#### Post date: [January 29, 2021, 8:16pm UTC](https://discourse.julialang.org/t/invalidating-sysimages/53922/4 "2021-01-29T20:16:32Z")
It looks like the Julia VSCode extension does it based on timestamps: [julia-vscode/repl.ts at b7a8b6b3fa838a64715f2315d634f2d9d09f79c9 · julia-vscode/julia-vscode · GitHub](https://github.com/julia-vscode/julia-vscode/blob/b7a8b6b3fa838a64715f2315d634f2d9d09f79c9/src/repl.ts#L197)
But they always use all packages in the environment to build the sysimage. I think the right set to watch is the packages you pass to create the sysimage, and all of their dependencies (and their dependencies, etc).
---
### Author:  [@airpmb](https://discourse.julialang.org/u/airpmb)
#### Post date: [January 29, 2021, 9:08pm UTC](https://discourse.julialang.org/t/invalidating-sysimages/53922/5 "2021-01-29T21:08:51Z")
Thanks. Mostly though I don’t run from within VS Code–and it looks like that is only for the REPL in VS Code?
At any rate, just so I actually have some transparency to what’s going on–and also, IIUC, not updating the sysimage will just silently cause much confusion down the road–for now I’m going to try rolling my own script.
---
### Author:  [@ericphanson](https://discourse.julialang.org/u/ericphanson)
#### Post date: [January 29, 2021, 9:13pm UTC](https://discourse.julialang.org/t/invalidating-sysimages/53922/6 "2021-01-29T21:13:16Z")
Oh yeah, I didn’t mean to suggest you should re-use their code (which is in typescript anyway), I just wanted to provide some support to the idea that others are already doing it for various uses 🙂. The VSCode extension does it so you can ask it to compile a sysimage through the UI and it knows when to invalidate it, and automatically start the REPL with that sysimage.
I think it makes sense to roll your own script for now, though hopefully at some point there is more tooling built up around it. In fact, hopefully at some point the package manager itself can be aware of when sysimages are invalidated by changes to the project ([https://github.com/JuliaLang/Pkg.jl/issues/2008](https://github.com/JuliaLang/Pkg.jl/issues/2008)).
---
### Author:  [@airpmb](https://discourse.julialang.org/u/airpmb)
#### Post date: [January 29, 2021, 9:23pm UTC](https://discourse.julialang.org/t/invalidating-sysimages/53922/7 "2021-01-29T21:23:40Z")
> [@ericphanson](#):
>
> hopefully at some point the package manager itself can be aware of when sysimages are invalidated by changes to the project ([Proposal for more first class handing of sysimages in Pkg · Issue #2008 · JuliaLang/Pkg.jl · GitHub](https://github.com/JuliaLang/Pkg.jl/issues/2008)).
Yes I’m particularly excited for that to come about!
So a related thing I’m wondering: Is there a way to determine what packages (modules?) are worth including in the `create_sysimage` call? I mean I guess I can just list every single package in the manifest but over time I’ll start to accumulate packages that I don’t always use.
I’m also still not quite clear on under what conditions I should definitely update the sysimage. For example, do I need to worry about transitive dependencies or do these show up in the manifest as changes anyway? Are there any changes that would not show up in Manifest.toml yet would invalidate a custom sysimage in some way?
---
### Author:  [@ericphanson](https://discourse.julialang.org/u/ericphanson)
#### Post date: [January 29, 2021, 9:46pm UTC](https://discourse.julialang.org/t/invalidating-sysimages/53922/8 "2021-01-29T21:46:09Z")
> Is there a way to determine what packages (modules?) are worth including in the `create_sysimage` call?
I think it’s a bit of a balancing act, because you are locking in those versions, so updating them means waiting for a sysimage rebuild. So I think it’s good for packages you use often but are fairly stable and you don’t often need to update them (or their (transitive) dependencies!). I used to use a sysimage for plotting with Makie.jl, but it’s still in a state of active development and I end up updating a lot, and so I stopped using one. At my work, we also use sysimages in docker images, to package up a piece of code and have it start quickly. There we just put everything except for `dev`’d dependencies into the sysimage (i.e. everything except for the code we’re actually working on).
> do I need to worry about transitive dependencies or do these show up in the manifest as changes anyway?
Yeah, changes to transitive dependencies will show up in the Manifest.
> Are there any changes that would not show up in Manifest.toml yet would invalidate a custom sysimage in some way?
Hm, changing the Julia version would do that, but I can’t really think of anything else.