# Compilation time differences with custom sysimg

**URL:** <https://discourse.julialang.org/t/compilation-time-differences-with-custom-sysimg/63365>\
**Category:** General Usage\
**Created:** [June 22, 2021, 1:24pm UTC](https://discourse.julialang.org/t/compilation-time-differences-with-custom-sysimg/63365 "2021-06-22T13:24:00Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![paulmelis](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/paulmelis/32/35063_2.png) [@paulmelis](https://discourse.julialang.org/u/paulmelis)\
**Post date:** [June 22, 2021, 1:24pm UTC](https://discourse.julialang.org/t/compilation-time-differences-with-custom-sysimg/63365/1 "2021-06-22T13:24:00Z")

</div>

I was doing some tests with custom sysimages, based on @kristoffer.carlsson’s JuliaCon [video](https://www.youtube.com/watch?v=d7avhSuK2NA).

```julia
$ cat genimg.jl 
using PackageCompiler

create_sysimage([:Plots]; sysimage_path="sysimage_plots.so")

$ cat precompile_plots.jl 
using Plots
p = plot(rand(2,2))
display(p)

$ cat genimg2.jl 
using PackageCompiler

create_sysimage([:Plots]; sysimage_path="sysimage_plots2.so", precompile_execution_file="precompile_plots.jl")

$ julia genimg.jl
[ Info: PackageCompiler: creating system image object file, this might take a while...

$ julia genimg2.jl
[ Info: ===== Start precompile execution =====
[ Info: ===== End precompile execution =====
[ Info: PackageCompiler: creating system image object file, this might take a while...

$ ls -l sysimage_plots*
-rwxr-xr-x 1 melis users 225341080 Jun 22 14:35 sysimage_plots2.so
-rwxr-xr-x 1 melis users 220192976 Jun 22 14:29 sysimage_plots.so

```

The differences in time to plot clearly show the sysimages at work:

```julia
$ julia testplot.jl 
  8.225244 seconds (10.30 M allocations: 601.803 MiB, 1.68% gc time, 16.18% compilation time)

$ julia --sysimage=sysimage_plots.so testplot.jl 
  5.807044 seconds (8.93 M allocations: 523.475 MiB, 1.51% gc time, 59.42% compilation time)

$ julia --sysimage=sysimage_plots2.so testplot.jl 
  0.389013 seconds (115.43 k allocations: 7.312 MiB, 8.79% compilation time)

```

But I was wondering about the reported compilation times:

- With default sysimg: 0.1618 \* 8.225244s = 1.3308444792 s
- With `Plots` sysimg: 0.5942 \* 5.807044s = 3.4505455448 s
- With `Plots` + script sysimg: 0.0879 \* 0.389013s = 0.0341942427 s

How come the compilation time at runtime is longer for the `Plots` sysimg than for the default sysimg? Wouldn’t one expect less compilation at runtime, as most of it would have been baked into the sysimg? Checking with `--trace-compile` indeed seems to suggest there are mostly fewer methods to precompile at runtime, but also a few extra:

```julia
$ julia --trace=sysimage_default.trace testplot.jl
$ julia --trace=sysimage_plots.trace --sysimage=sysimage_plots.so testplot.jl

$ diff -ur sysimage_default.trace sysimage_plots.trace | grep "^\+precompile"
+precompile(Tuple{Type{Base.Dict{Any, Plots.Subplot{T} where T<:RecipesBase.AbstractBackend}}})
+precompile(Tuple{typeof(Base.empty!), Base.Dict{Symbol, Nothing}})
+precompile(Tuple{typeof(Base.get!), Base.var"#156#157"{Base.Set{Symbol}}, Base.Dict{Symbol, Base.Set{Symbol}}, Symbol})

$ diff -ur sysimage_default.trace sysimage_plots.trace | grep "^\-precompile"
-precompile(Tuple{typeof(Base.print), Base.GenericIOBuffer{Array{UInt8, 1}}, UInt16})
-precompile(Tuple{Base.Sort.var"##sortperm#12", Base.Sort.QuickSortAlg, Function, Function, Bool, Base.Order.ForwardOrdering, typeof(Base.sortperm), Array{Float64, 1}})
-precompile(Tuple{Type{Base.Order.Perm{O, V} where V<:(AbstractArray{T, 1} where T) where O<:Base.Order.Ordering}, Base.Order.ReverseOrdering{Base.Order.ForwardOrdering}, Array{Float64, 1}})
-precompile(Tuple{typeof(Base.sort!), Array{Int64, 1}, Base.Sort.QuickSortAlg, Base.Order.Perm{Base.Order.ReverseOrdering{Base.Order.ForwardOrdering}, Array{Float64, 1}}})
...
-precompile(Tuple{typeof(Plots.get_linecolor), Plots.Series, Float64, Float64, Int64})
-precompile(Tuple{Plots.var"#gr_polyline##kw", NamedTuple{(:arrowside, :arrowstyle), Tuple{Symbol, Symbol}}, typeof(Plots.gr_polyline), Base.UnitRange{Int64}, Array{Float64, 1}})
-precompile(Tuple{typeof(Plots.gr_add_legend), Plots.Subplot{Plots.GRBackend}, NamedTuple{(:w, :h, :dy, :leftw, :textw, :rightw, :xoffset, :yoffset, :width_factor), Tuple{Float64, Float64, Float64, Float64, Float64, Float64, Float64, Float64, Float64}}, Array{Float64, 1}})

$ diff -ur sysimage_default.trace sysimage_plots.trace | grep "^\+precompile" | wc -l
3
$ diff -ur sysimage_default.trace sysimage_plots.trace | grep "^\-precompile" | wc -l
64

```

Another question: compilation of the sysimages themselves takes quite a bit of time, around 3 minutes for the `Plots` sysimg, around 4 minutes for `Plots` + script. Is there a way to get statistics on the number of compiled methods and such, to get some idea of the amount of work done?

```julia
Julia Version 1.6.1
Commit 6aaedecc44* (2021-04-23 05:59 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
  CPU: Intel(R) Core(TM) i5-4460 CPU @ 3.20GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-11.1.0 (ORCJIT, haswell)

```