# Precompilation or sysimg building not working as expected

**URL:** <https://discourse.julialang.org/t/precompilation-or-sysimg-building-not-working-as-expected/1399>\
**Category:** General Usage\
**Created:** [January 10, 2017, 4:50pm UTC](https://discourse.julialang.org/t/precompilation-or-sysimg-building-not-working-as-expected/1399 "2017-01-10T16:50:05Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![jminardi](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jminardi/32/452_2.png) [@jminardi](https://discourse.julialang.org/u/jminardi)\
**Post date:** [January 10, 2017, 4:50pm UTC](https://discourse.julialang.org/t/precompilation-or-sysimg-building-not-working-as-expected/1399/1 "2017-01-10T16:50:05Z")

</div>

Our company has a medium size Julia app that we run from the command line. Because every time we run we start a new julia instance we have a good amount of overheard in loading the libraries we use and priming the JIT compiler.

Our app starts with essentially one function call. If I just call this function twice during the same run of julia the first call takes around 20 seconds and the second takes 0.1 seconds. What I want is for the first call to take the same amount of time as the second call, is this an achievable goal?

I tried using the precompile statement, which gave me some speedup, but not what I wanted. (The 20 seconds quoted above is actually with precompile statement enabled)

I also tried building a new system image with the included script. This works much better, reducing the time down to 1.6 seconds.

Is there any way to get the first run all the way down to the second run time of 0.1, or is that just asking too much?

---

<div class="post-metadata">

**Author:** ![rdeits](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rdeits/32/286_2.png) [@rdeits](https://discourse.julialang.org/u/rdeits)\
**Post date:** [January 10, 2017, 5:06pm UTC](https://discourse.julialang.org/t/precompilation-or-sysimg-building-not-working-as-expected/1399/2 "2017-01-10T17:06:57Z")

</div>

When you say you’re using the precompile statement, do you mean the ` __precompile__ ()` at the top of your module, or are you explicitly calling `precompile(myfunction, (MyInputType, MyOtherInputType))`?

---

<div class="post-metadata">

**Author:** ![jminardi](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jminardi/32/452_2.png) [@jminardi](https://discourse.julialang.org/u/jminardi)\
**Post date:** [January 10, 2017, 5:08pm UTC](https://discourse.julialang.org/t/precompilation-or-sysimg-building-not-working-as-expected/1399/3 "2017-01-10T17:08:12Z")

</div>

` __precompile__ ()` at the top of my module.

---

<div class="post-metadata">

**Author:** ![rdeits](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rdeits/32/286_2.png) [@rdeits](https://discourse.julialang.org/u/rdeits)\
**Post date:** [January 10, 2017, 5:13pm UTC](https://discourse.julialang.org/t/precompilation-or-sysimg-building-not-working-as-expected/1399/4 "2017-01-10T17:13:11Z")

</div>

I’d suggest trying the second form. For example:

```julia
precompile(min, (Int, Int))

```

[SnoopCompile](https://github.com/timholy/SnoopCompile.jl) can also help you figure out exactly which precompile() calls you’ll need. But the first thing to try would just be whatever your 20-second function is.

---

<div class="post-metadata">

**Author:** ![rdeits](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rdeits/32/286_2.png) [@rdeits](https://discourse.julialang.org/u/rdeits)\
**Post date:** [January 10, 2017, 5:16pm UTC](https://discourse.julialang.org/t/precompilation-or-sysimg-building-not-working-as-expected/1399/5 "2017-01-10T17:16:03Z")

</div>

By the way, the reason this is necessary is that, as I understand it, ` __precompile__ ()` can’t generally know what input types to compile your functions for, since that would require knowing exactly what input types the user might supply to your functions (which it can’t possibly know ahead of time). But if you _do_ know, then you can supply those input types when you call `precompile(...)`.

---

<div class="post-metadata">

**Author:** ![jminardi](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jminardi/32/452_2.png) [@jminardi](https://discourse.julialang.org/u/jminardi)\
**Post date:** [January 10, 2017, 5:34pm UTC](https://discourse.julialang.org/t/precompilation-or-sysimg-building-not-working-as-expected/1399/6 "2017-01-10T17:34:36Z")

</div>

I will try with the function, but in the module where we call ` __precompile__ ()` we also exercise our app with real world inputs in an attempt to prime the JIT compiler, which I think serves the same purpose as calling the function with arguments.

---

<div class="post-metadata">

**Author:** ![ihnorton](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ihnorton/32/26_2.png) [@ihnorton](https://discourse.julialang.org/u/ihnorton)\
**Post date:** [January 11, 2017, 3:56am UTC](https://discourse.julialang.org/t/precompilation-or-sysimg-building-not-working-as-expected/1399/7 "2017-01-11T03:56:34Z")

</div>

> [@jminardi](#):
>
> I also tried building a new system image with the included script.

Assuming you mean `contrib/build_sysimg.jl`, did you specify a custom userimg file?

Alternatively, try putting your `precompile` calls in `base/userimg.jl` and rebuild through the usual process (`make`, no need to run `build_sysimg`).

The `build_sysimg` code and output look ok on OS X, but isn’t part of the standard build process and I don’t think it’s really tested. So there might be some subtleties missing on some platforms.

> [@jminardi](#):
>
> Is there any way to get the first run all the way down to the second run time of 0.1, or is that just asking too much?

A no-op run (`time julia -e ""`) on my system is close to .25s, so you might not be able to get a first run down to 0.1s.

---

<div class="post-metadata">

**Author:** ![jminardi](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jminardi/32/452_2.png) [@jminardi](https://discourse.julialang.org/u/jminardi)\
**Post date:** [January 11, 2017, 10:31pm UTC](https://discourse.julialang.org/t/precompilation-or-sysimg-building-not-working-as-expected/1399/8 "2017-01-11T22:31:27Z")

</div>

I did specify a customer userimg file, it is just one line: `using $MyModule`

The function form did not significantly speed up the operation. It is looking like 1.6 seconds is the best I can hope for now. (Much better than the original 20 seconds!)

---

<div class="post-metadata">

**Author:** ![rdeits](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rdeits/32/286_2.png) [@rdeits](https://discourse.julialang.org/u/rdeits)\
**Post date:** [January 11, 2017, 11:50pm UTC](https://discourse.julialang.org/t/precompilation-or-sysimg-building-not-working-as-expected/1399/9 "2017-01-11T23:50:32Z")

</div>

@jminardi out of curiosity, what was the change that got you down from 20s to 1.6s?

---

<div class="post-metadata">

**Author:** ![jminardi](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jminardi/32/452_2.png) [@jminardi](https://discourse.julialang.org/u/jminardi)\
**Post date:** [January 12, 2017, 6:07pm UTC](https://discourse.julialang.org/t/precompilation-or-sysimg-building-not-working-as-expected/1399/10 "2017-01-12T18:07:29Z")

</div>

I build a sysimg with my module included. See here:

[http://docs.julialang.org/en/stable/devdocs/sysimg/](http://docs.julialang.org/en/stable/devdocs/sysimg/)

---

<div class="post-metadata">

**Author:** ![Glen1](https://avatars.discourse-cdn.com/v4/letter/g/ee7513/32.png) [@Glen1](https://discourse.julialang.org/u/Glen1)\
**Post date:** [January 8, 2019, 8:43pm UTC](https://discourse.julialang.org/t/precompilation-or-sysimg-building-not-working-as-expected/1399/11 "2019-01-08T20:43:36Z")

</div>

I can’t get this working … I’m looking to regenerate sys.so

I’ve been trying to regenerate a system image for the last 3 days with a few changes and using “core2” (v1.03)… but for the life of me, nothing is successful.  
— getting random errors …

1. FYI - might want to include Libdl in build\_sysimg.jl…
  - couldn’t get this working (build\_sysimg)… eventually extracted the commands being run(), and seemed to be able to generate a sys.o, but the final cc command produces a sys.so file that is too small and fails when replacing the sys.so… not sure if /…/lib/\* libraries aren’t being linked somehow… ran:  
cd /…/shared/julia/base  
julia -C core2 --output-ji …/…/…/lib/julia/basecompiler.ji --output-o …/…/…/lib/julia/basecompiler.o compiler/compiler.jl  
julia -C core2 --output-ji …/…/…/lib/julia/sys.ji --output-o sys.o -J …/…/…/lib/julia/basecompiler.ji --startup-file=no sysimg.jl  
cc -L…/…/…/lib -shared -ljulia -o sys.so sys.o

but sys.so is way too small…

1. PackageCompiler errors out trying to replace a directory… could dig deeper… but not sure that this is the correct tool for generating sys.so – limited documentation…

- copied base dir…
- build\_shared\_lib(“sysimg.jl”, “sysx”)
- ERROR: LoadError: cannot replace module Base during compilation  
… are we able to use this for a system image?

Is there a process for generating a system image (sys.so) in v1.03 – that works 😉

Thanks