Overhead between calling a function and running the first line its code

[maxchendt]

After the packages are loaded, there is a “long” time to wait until the first message indicate that the function called is running. Here is a test example:

println("------------- Loading")
t0 = time_ns()
using LinearAlgebra
using Random
t1 = time_ns()
println("Package loaded:  ", t1 - t0, "  ns")
println("------------- Going main")

t0 = time_ns()


function main() #test time_ns used
    println("\n\n\n------------- running -------------")
    t0 = time_ns()

    N = 7777
    Random.seed!(12357)
    X = rand(N, N)
    x = norm(X)
    display(x)
    t1 = time_ns()
    println("code run time_ns:  ", t1 - t0, "  ns")
end

t1 = time_ns()
println("Reach main:  ", t1 - t0, "  ns  ------------")

main()
nothing

You will see there is a time interval between “Reach main” and “------------- running -------------”, even in a consecutive run.

What jobs do Julia do during this time, just the overhead for calling a function, or else?

[jmair]

Julia has to compile the function upon the first run. When you call it again, it will be instant (within the same REPL session), but the first call will take a second or two to compile.

[PeterSimon]

@maxchendt

You will see there is a time interval between “Reach main” and “------------- running -------------”, even in a consecutive run

What do you mean by “consecutive run”? I’m guessing that you have stored all the code you posted into a file, and that you generate consecutive runs by including that file multiple times. If so, then each time the file is included, the function main is redefined and must be recompiled when called. That’s why you see the delay each time.

[mkitti]

With the way you have setup your code, you are recompiling the function main every time you attempt to run your script.

Try this:

t1 = time_ns()
println("Reach main:  ", t1 - t0, "  ns  ------------")

main()

t2 = time_ns()
println("Reach main 2:  ", t2 - t0, "  ns  ------------")

main()
nothing

The second main should run faster.

Now the next step is to figure out how to cache the compilation. There are a few options:

1. Do not exit Julia. The compile code will be stored in memory (RAM). You can run julia with -i and enter the interactive REPL. Using Revise.jl makes this easier.

2. Use GitHub - dmolina/DaemonMode.jl: Client-Daemon workflow to run faster scripts in Julia . The compiled code will be store in memory.

3. Use GitHub - JuliaLang/PackageCompiler.jl: Compile your Julia Package to create a new system image

4. Create a package. In Julia 1.8 and earlier code inference will be cached to disk. In Julia 1.9, native xode will be cached to disk.

If you are just starting out, using the REPL and Revise.jl is probably the easiest route. Otherwise, I recommend Option 4.

[stevengj]

@mkitti

Do not exit Julia.

There are several cases:

1. You are doing interactive exploration. Solution: use an interactive environment (the REPL, a Jupyter notebook, vsCode, Pluto, …) in which you leave Julia running. (For larger-scale code development, do this in conjunction with Revise.jl by creating a package. See Best practise: organising code in Julia - #2 by stevengj)
2. Your function main() is extremely fast, but you are calling it lots of times. Solution: write your loop in Julia — don’t write a shell script or something. (If you are doing benchmarking, use BenchmarkTools.jl.)
3. Your function main() takes a long time (minutes, hours, …). Solution: use separate Julia runs if you want. The compile time will be irrelevant.

[mkitti]

I do wonder now that we have pkgimgs how long would it take to get scriptimgs . We seem to be getting quite close to just compiling this to a not very portable executable.

[giordano]

@mkitti

I do wonder now that we have pkgimgs how long would it take to get scriptimgs .

Taking advantage of pkgimages in 'scripts'

There’s a lot of benchmark scripts out there right now where Julia is at a big disadvantage due to compiler overhead and such. Has anyone thought out an easy way that we can start taking advantage of pkgimages to reduce this compilation overhead in various benchmarks? Having to bundle all of the user’s code into a dev’d package seems a little heavy. E.g. in the example of n-body Julia #8 program (Benchmarks Game) could we make this significantly faster by bundling the code into a pkgimage, and …

[maxchendt]

environment: Julia 1.9 rc1, and EfficientFrontier v1.3.1

the full code

println("------------- Loading")
t0 = time_ns()
using EfficientFrontier
using LinearAlgebra, CodecXz, Serialization, Downloads, TranscodingStreams
t1 = time_ns()
println("Package loaded:  ", (t1-t0)/1e9)
println("------------- Going main")


function mainS5h()
    xzFile = joinpath(tempdir(), "sp500.jls.xz")  #xzFile = "/tmp/sp500.jls.xz"
    if !isfile(xzFile)
        Downloads.download("https://github.com/PharosAbad/PharosAbad.github.io/raw/master/files/sp500.jls.xz", xzFile)
    end
    io = open(xzFile)
    io = TranscodingStream(XzDecompressor(), io)
    E = deserialize(io)
    V = deserialize(io)
    close(io)
    u = fill(3 / 32, length(E))

    t0 = time_ns()
    P = Problem(E, V, u)
    ts = @elapsed aCL = EfficientFrontier.ECL(P)
    aEF = eFrontier(aCL, P)
    t1 = time_ns()
    println("Status-Segment Method:  ", ts, "  seconds ", (t1-t0)/1e9)
    return ts

end

function main() #Julia 1.9 rc 1
    tm = time_ns()
    println("running ------------- in main:  ", (tm-t2)/1e9)
    N = 7
    mainS5h()
    println("             ---next ", N)
    t = zeros(N)
    for k in 1:N
        t[k] = mainS5h()
    end
    display((minimum(t), maximum(t), sum(t) / N))
end


t2 = time_ns()
println("Reach main:  ", (t2-t1)/1e9)

main()
nothing

the output: first run

julia> include("./Julia/test.jl")
------------- Loading
Package loaded:  0.70569542
------------- Going main
Reach main:  0.032035565
running ------------- in main:  9.420282055
Status-Segment Method:  0.130419639  seconds 1.224862577
             ---next 7
Status-Segment Method:  0.081276494  seconds 0.157252959
Status-Segment Method:  0.079336187  seconds 0.139380063
Status-Segment Method:  0.074937016  seconds 0.140449263
Status-Segment Method:  0.0740619  seconds 0.136816746
Status-Segment Method:  0.07623903  seconds 0.142366487
Status-Segment Method:  0.079054201  seconds 0.141553152
Status-Segment Method:  0.074079641  seconds 0.141342849
(0.0740619, 0.081276494, 0.07699778128571429)

second run

julia> include("./Julia/test.jl")
------------- Loading
Package loaded:  0.000309086
------------- Going main
Reach main:  0.002167689
running ------------- in main:  0.029902096
Status-Segment Method:  0.085099502  seconds 0.147684142
             ---next 7
Status-Segment Method:  0.078142165  seconds 0.144540875
Status-Segment Method:  0.078970079  seconds 0.142851774
Status-Segment Method:  0.07676492  seconds 0.141821136
Status-Segment Method:  0.075969915  seconds 0.136969677
Status-Segment Method:  0.079847105  seconds 0.148844903
Status-Segment Method:  0.073495032  seconds 0.134014575
Status-Segment Method:  0.075658942  seconds 0.143792864
(0.073495032, 0.079847105, 0.07697830828571428)

I am excited that Julia 1.9 rc1 cut the first run of the EfficientFrontier.ECL for “Status-Segment Method” from 7.89 seconds (in Julia 1.8.5) seconds to 0.13 seconds. However, how to reduce the 9.42 seconds in the first run of calling function main? (the data have been downloaded to /tmp/sp500.jls.xz first )