Running AST directly (for scripts), bypassing parsing; deserialize ok in the sysimage?

General Usage Performance
1

In short it’s faster to run the abstract syntax tree (AST) than parse a script. I was testing with:

However it’s not currently because (i.e. would be if deserialize in the sysimage, assuming it doesn’t slow down startup much):

$ hyperfine 'julia +1.13 -e "using Serialization; forms = deserialize(\"input.jast\")" >/dev/null'

708.8-191.2 = 517.6 ms after accounting for startup.

But deserialize only costs 0.3935 ms for my example code (35% faster than parsing the code, if I recall). Because of the precomilation overhead it’s much more now, shown by:

$ hyperfine 'julia +1.13 -e "using Serialization; for i in 1:1000 forms = deserialize(\"input.jast\") end" '
Benchmark 1: julia +1.13 -e "using Serialization; for i in 1:1000 forms = deserialize(\"input.jast\") end"
  Time (mean ± σ):      1.201 s ±  0.076 s    [User: 1.058 s, System: 0.141 s]
  Range (min … max):    1.104 s …  1.328 s    10 runs

$ julia +1.13 --trace-compile=stderr -e "using Serialization; forms = deserialize(\"input.jast\")"
precompile(Tuple{typeof(Serialization.deserialize), String})
precompile(Tuple{typeof(Serialization.deserialize_fillarray!), Array{Any, 1}, Serialization.Serializer{Base.IOStream}})
precompile(Tuple{typeof(Serialization.deserialize), Serialization.Serializer{Base.IOStream}, typeof(DataType)})

So are we willing to have deserialize in the sysimage?

Currently 0.508 sec slower than running directly:

$ hyperfine "julia +1.13 -e 'using Serialization; forms = deserialize(\"input.jast\"); foreach(eval, forms);' 25000000 >/dev/null"

$ hyperfine "julia +1.13 fasta2.jl 25000000 >/dev/null"


using Serialization

src = read("fasta2.jl", String)

forms = Any[]
pos = 1

while pos <= lastindex(src)
    ex, nextpos = Meta.parse(src, pos; raise=false)
    ex === nothing && break

    push!(forms, ex)

    nextpos <= pos && break
    pos = nextpos
end

open("input.jast", "w") do io
    serialize(io, forms)
end