Faster/consistent short lived script

New to Julia
1

Hi,

in a bid to promote julia a bit I’m trying to get it accepted as a supported language for an AI competition.
Unfortunately the competition requires short lived executions with strict timing constraints.
(this is related to my earlier post but I thought I’d better start a separate discussion)

I need to be able to run a julia script with consistent execution time.
Eg. if it is determined that a no-opt script takes 1s then a competition entry will be allowed 3s max.

I have found that even with BuildExecutable that about 1 in 15 runs goes 2s over the average for a no-op implementation (see some stats below).

Is there any way to get this more consistent? Maybe static compilation or doing a pre-compile that gets cached to files?

I have also tried my best to get it statically compiled with no luck (I tried several different versions even compiled from source ; with and without JULIA_HOME environment variable defined):

$ /stf/opt/julia-0.5.1/bin/julia --output-ji=x.ji --compile=all -J /stf/opt/julia-0.5.1/lib/julia/sys.so -H /stf/opt/julia-0.5.1/bin testcompile.jl
ERROR: UndefVarError: JULIA_HOME not defined
in load_juliarc() at ./client.jl:276
in process_options(::Base.JLOptions) at ./client.jl:231
in _start() at ./client.jl:321
in jl_apply at /home/centos/buildbot/slave/package_tarball64/build/ui/…/src/julia.h:1392 [inlined]
in true_main at /home/centos/buildbot/slave/package_tarball64/build/ui/repl.c:123
in main at /home/centos/buildbot/slave/package_tarball64/build/ui/repl.c:243

normal Julia
eg. 1.035582 seconds (288.30 k allocations: 12.316 MB, 6.38% gc time)
avg=2.683333333333334
execution times:
[2.0,1.99,2.04,2.06,2.06,2.05,2.05,3.0,2.03,2.03,2.36,1.94,3.47,6.79,4.38]

with BuildExecutable
eg. 0.985178 seconds (287.32 k allocations: 12.517 MB, 6.63% gc time)
avg=3.8993333333333338
execution times:
[1.78,4.09,3.07,3.52,3.86,3.5,4.05,4.09,3.44,3.22,3.52,7.13,5.5,3.86,3.86]

2

After adding some command-line options, it seems to run much more consistently!
So it was accepted \o/

New commandline options: --precompiled=yes --compilecache=no
stats:

average_run_time: 2.8456
average difference with average_run_time: 0.03516800000000009
max difference with average_run_time: 0.13439999999999985

i.e. in 25 runs the max offset to the average is 0.134 seconds which I think is totally acceptable…

how I calculated my stats if anybody is interested:

python -m timeit -n 1 -r 25 -v -s 'import os' 'os.system("julia --precompiled=yes --compilecache=no -- main.jl 007 test_data/Phase\ 2\ -\ Round\ 1/")'
0.582302 seconds (241.16 k allocations: 10.284 MB)
raw times: 2.84 2.84 2.84 2.82 2.81 2.82 2.83 2.85 2.91 2.85 2.82 2.81 2.81 2.83 2.9 2.94 2.98 2.86 2.78 2.87 2.89 2.82 2.77 2.84 2.81
1 loops, best of 25: 2.77 sec per loop

julia> function stats(txt); A = [parse(Float64,x) for x in split(txt, " ")]; avg = mean(A); B = max(0.0, A - avg - 2); C = abs(A - avg) ; println("avg=$avg avg_delta=$(mean(C)) max_delta=$(max(C...))") ; println(A); println(C); println(B); end

julia> stats("2.84 2.84 2.84 2.82 2.81 2.82 2.83 2.85 2.91 2.85 2.82 2.81 2.81 2.83 2.9 2.94 2.98 2.86 2.78 2.87 2.89 2.82 2.77 2.84 2.81")
avg=2.8456 avg_delta=0.03516800000000009 max_delta=0.13439999999999985
[2.84,2.84,2.84,2.82,2.81,2.82,2.83,2.85,2.91,2.85,2.82,2.81,2.81,2.83,2.9,2.94,2.98,2.86,2.78,2.87,2.89,2.82,2.77,2.84,2.81]
[0.0056,0.0056,0.0056,0.0256,0.0356,0.0256,0.0156,0.0044,0.0644,0.0044,0.0256,0.0356,0.0356,0.0156,0.0544,0.0944,0.1344,0.0144,0.0656,0.0244,0.0444,0.0256,0.0756,0.0056,0.0356]
[0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0]
2 Likes