Permanent recompilation of libraries and performance issues

I have a little test program that uses some libraries

println("load libs ...")
t0 = time()
@time using JuMP
@time using Ipopt

println("run prog ...")
m = Model(Ipopt.Optimizer)

println("set variables ...")
@variable(m,x[1:2])
@NLobjective(m,Min,(x[1]-3)^3+(x[2]-4)^2+x[1]*x[2])
@NLconstraint(m,(x[1]-1)^2+(x[2]+1)^3+exp(-x[1])<=1)
println(m)

println("optimize ...")
@time JuMP.optimize!(m)

println("** Optimal objective function value = ",JuMP.objective_value(m))
println("** Optimal solution = ",JuMP.value.(x))
println()
println("prog took ",round(time()-t0,digits=2)," sec ...")

On every start of the program (I did not change anything and I did not touch Pkg) it takes several seconds for recompilation:

load libs ...
  5.880101 seconds (8.24 M allocations: 636.694 MiB, 6.29% gc time, 8.85% compilation time: 97% of which was recompilation)
  1.396453 seconds (1.36 M allocations: 72.658 MiB, 10.58% gc time, 91.54% compilation time: 100% of which was recompilation)

This cannot be a correct behavior, is it?
The program is overall very slow.

prog took 23.39 sec ...

I wonder if there is something wrong with my Julia installation (Windows 11, fast and modern computer, SSD, CPU far below 100%).

Essentially this is to be expected, as Julia caches an intermediate representation over different sesssions, and by its dynamic nature it needs to be aware that methods may be called with different types and thus compiled for specialization. There is serious work underway in Julia proper and in many packages to improve the situation, in particular by caching machine code. You can find more (and more precise) information in this Juliacon talk. It is expected to see the results in one of the next releases.

That said, there are some general workflow tipps which can drastically improve your turnaround time just now. I did a writeup on this here.

1 Like

Can you test the program? What time takes it on your computer? If I remove the about 7 seconds of library loading, it is finally very slow without using much CPU time.

EDIT:
Checked the hints on your link. Loading the script in the REPL and than calling main leads to execution time of 0.09 sec. So the question would be why Julia isn’t able to remember the compiled version (loading libraries cannot be cached?!) via a cache.

Working only from REPL seems to me quite inconvenient if you are used to start a program from shell …

I came from C++/python and had to take this hurdle as well… How much starting from shell is important for you is of course a matter of personal preferences, for me the advantages outweigh this problem.

With your posted code I get

prog took 12.43 sec 

Wrapping into a module and a function gives on an Intel(R) Core™ i7-9850H CPU @ 2.60GHz

prog took 7.09 sec

for the first run and

prog took 0.02 sec

for the second. So this is consistent to your observations.

To get an impression on the ongoing work on saving compiled machine code, besides the video you can investigate

Did you run it from REPL or from shell?

Putting the Code in a function also results in 12 sec for my computer (running from shell, 7 sec for loading libraries).

By the way: I’m coming from Python so I’m used to run always from shell without delay.

What you are running into is known as “TTFX”, “Time To First X” problem. There are loads of discussions on this on here and across the internet, but the short answer is that if you’re trying to use Julia as a scripting language for small programs you are going to be disappointed at this point, as startup and compilation time will entirely dominate your experience.

For that reason, (probably) 99% of Julians have a workflow which relies on a long running REPL session which means TTFX cost is only incurred once for each type-specific method call. This doesn’t necessarily mean you are actually interacting with the REPL, as lots of front ends are available (VSCode, Jupyter, Pluto notebooks, plug ins for other editors).

If you absolutely have to repeatedly call Julia from the shell, there’s

Targeted at this use case (I have never used it so am not sure it fits the bill for what you’re doing).

If you want to use Julia to build an executable that can be called externally there is also tooling for that in PackageCompiler (full language support, large binaries) and StaticCompiler (experimental, support for a very restricted subset of the language at this point, small binaries).

This is the situation as it stands, but as mentioned above there is a lot of work in this area at the minute so expect improvements in either Julia 1.9 or 1.10 (or both!)

Thank you all for the information. I really like Julia an will try to work around such inconveniences.

EDIT:
I tried DaemonMode.jl and this is really fast. I think I go with this until Julia is faster at this point.

1 Like