# How to programmatically update my PackageCompiler's sysimage, using a precompile\_execution\_file?

**URL:** https://discourse.julialang.org/t/how-to-programmatically-update-my-packagecompilers-sysimage-using-a-precompile-execution-file/91050
**Category:** General Usage
**Tags:** precompilation, package-compiler, sysimage, scripting
**Created:** [November 30, 2022, 1:24pm UTC](https://discourse.julialang.org/t/how-to-programmatically-update-my-packagecompilers-sysimage-using-a-precompile-execution-file/91050 "2022-11-30T13:24:30Z")
**Posts on this page:** 2
**Page:** 1

<div class="post-metadata">

### Author: ![bertulli](https://avatars.discourse-cdn.com/v4/letter/b/ecc23a/32.png) [@bertulli](https://discourse.julialang.org/u/bertulli)
#### Post date: [November 30, 2022, 1:24pm UTC](https://discourse.julialang.org/t/how-to-programmatically-update-my-packagecompilers-sysimage-using-a-precompile-execution-file/91050/1 "2022-11-30T13:24:30Z")

</div>

Hi all!

I am using `PackageCompiler` to speed up Julia usage. It works enthusiastically well, and it saves me some time, but I also want to keep the sysimage updated. Specifically, I want to update it (even manually) once in a while, when:

- a package gets updated
- I use a new function in my scripts

First and main question (to avoid the XY problem): **how can I do that?**

I’ve thought I could embed the package compilation in a script to invoke periodically:

```julia
## update_sysimage.jl

using PackageCompiler;

PackageCompiler.create_sysimage(["Package1", "Package2"];
                                sysimage_path="/path/to/sysimage.so",
                                precompile_execution_file="???.jl")

```

Since I run multiple scripts in a specific directory, and since you have to use the `precompile_execution_file` to instruct Julia to detect and compile the functions you use, what should I put in there, to make it aware of new functions? I thought I can simply get the list of the scripts I wrote, and include them all:

```julia
# precompile_execution_file.jl
script_files = readdir(abspath(scripts_source_dir), join=true);

include.(script_files);

```

Now, I think this _should_ work (I’m not sure), **BUT** :

1. Do I do unnecessary computation? Is there a more efficient/idiomatic way?

2. When using `PackageCompiler.create_sysimage`, does the `precompile_execution_file` effectively get run? IOW, will it be run, and modify my data with **side-effects** , or `PackageCompiler` simply _reads_ it to extract the signatures of the functions to compile?

3. do you know any method to automatically detect also the list of packages to pass in the first argument array?

Thanks!

---

<div class="post-metadata">

### Author: ![ffevotte](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ffevotte/32/6587_2.png) [@ffevotte](https://discourse.julialang.org/u/ffevotte)
#### Post date: [November 30, 2022, 8:55pm UTC](https://discourse.julialang.org/t/how-to-programmatically-update-my-packagecompilers-sysimage-using-a-precompile-execution-file/91050/2 "2022-11-30T20:55:58Z")

</div>

> [@bertulli](#):
>
> - do you know any method to automatically detect also the list of packages to pass in the first argument array?

You can use something like this to get a list of all packages added to the current environment:

```julia
julia> import Pkg
julia> packages = Symbol.(keys(Pkg.project().dependencies))
6-element Vector{Symbol}:
 :Revise
 :Debugger
 :BenchmarkTools
 :OhMyREPL
 :Infiltrator
 :ProfileView

```

> [@bertulli](#):
>
> - Do I do unnecessary computation? Is there a more efficient/idiomatic way?
> - When using `PackageCompiler.create_sysimage`, does the `precompile_execution_file` effectively get run? IOW, will it be run, and modify my data with **side-effects** , or `PackageCompiler` simply _reads_ it to extract the signatures of the functions to compile?

Yes, the `precompile_execution_file` will effectively get run, and will potentially have side-effects. This is especially adequate if you have unit tests, because presumably they call a large fraction of your code without having any harmful side effect.

If you have a collection of scripts that actually modify data, then it may not be a very good idea to use it in the `precompile_execution_file`. Or maybe you can setup a sandboxed environment in which the scripts run and alter a _copy_ of your data, but not the real thing?

Alternatively, you can provide a [`precompile_statements_file`](https://julialang.github.io/PackageCompiler.jl/dev/sysimages.html#Using-a-manually-generated-list-of-precompile-statements), in which `PackageCompiler` will simply _read_ the list of methods to compile. But this list has to be generated ahead of time (_e.g_ using `julia --trace-compile` or `SnoopCompile`), and I’m pretty sure at some point you’ll have to actually execute the code you want to compile. But at least you can generate the statements list once, and re-use it multiple times (knowing that if it gets out-of-date, things should still work but the benefits of precompilation may be reduced).
