Sysimage -> Invalid signature in function definition

Why does running against a custom sysimage change the behaviour of my program? I thought, at most, it would accelerate the wrong things.

jlfmt.jl
$ cat jlfmt.jl 
using JuliaFormatter
function @main(ARGS::Vector{String})::Cint
    try
        for f in ARGS
            if f == "-"
                print(format_text(read(stdin, String)))
            else
                format_file(f)
            end
        end
        return 0
    catch e
        @error "err:" e
        return 1
    end
end
beer.jl
$ cat beer.jl 
bottle_word(n) = n == 1 ? "bottle" : "bottles"

function f(s)
    return 3s^2 + 2s
end

function beer()
    for n = 99:-1:0
        if n > 0
            next_n = n - 1
            next_text =
                next_n > 0 ? "$next_n $(bottle_word(next_n)) of beer on the wall." :
                "no more bottles of beer on the wall."
            take_phrase =
                n == 1 ? "Take it down and pass it around" :
                "Take one down and pass it around "
            println(
                Core.stdout,
                "$n $(bottle_word(n)) of beer on the wall, $n $(bottle_word(n)) of beer.",
            )
            println(Core.stdout, "$take_phrase, $next_text\n")
        else
            println(
                Core.stdout,
                "No more bottles of beer on the wall, no more bottles of beer.",
            )
            println(
                Core.stdout,
                "Go to the store and buy some more, 99 bottles of beer on the wall.",
            )
        end
    end
end


function @main(args::Vector{String})::Cint
    beer()
    return 0
end

Under normal Julia, this is fine:

$ time julia jlfmt.jl beer.jl 

real	0m1.331s
user	0m1.120s
sys	0m0.202s

But I compiled a sysimage with

$ time julia --startup-file=no --project=@helix-lsp -e 'import Pkg; Pkg.add(["LanguageServer", "PackageCompiler"]); Pkg.update();using PackageCompiler; create_sysimage([:LanguageServer, :JuliaFormatter], sysimage_path=dirname(Pkg.Types.Context().env.project_file) * "/languageserver.so")'

and it breaks the script:

$ time julia -J/home/kousu/.julia/environments/helix-lsp/languageserver.so  jlfmt.jl beer.jl 
┌ Warning: Failed to format file /home/kousu/src/beer.jl due to a parsing error, skipping file
│   error =
│    ParseError:
│    # Error @ line 36:10
│    
│    function @main(args::Vector{String})::Cint
│    #        └─────────────────────────┘ ── Invalid signature in function definition
└ @ JuliaFormatter ~/.julia/packages/JuliaFormatter/zLsPw/src/JuliaFormatter.jl:448

real	0m1.765s
user	0m1.493s
sys	0m0.249s

What is going on? How did I mess up precompile tools so badly? Shouldn’t it go looking in the regular libraries to get uncompiled versions?

PrecompileTools.jl and system images are two very different things.

And to create a custom system image is useful, but not all packages work as expected. I do not include Plots.jl or PyPlot.jl in my custom system image, for example, because they do not work as expected. So just try out which packages cause issues when including them in your system image and exclude them.

And feel free to create an issue in the package or in PackageCompiler.jl if you encounter an issue. These issues often can be related to global variables that can get calculated and frozen at compilation time and that are different at runtime.

1 Like

It looks like this is a breaking bug in JuliaFormatter. The sysimage was just an accidental trigger, because I ran it in --project=@helix-lsp which was a clean env and therefore installed JuliaFormatter@2.2.

Rebuilding with this got around it:

-Pkg.add(["LanguageServer", "PackageCompiler"])
+Pkg.add([Pkg.PackageSpec("LanguageServer"), Pkg.PackageSpec("PackageCompiler"), Pkg.PackageSpec(name="JuliaFormatter", version="1")]
Solution
$ time julia --startup-file=no --project=@helix-lsp -e 'import Pkg; Pkg.add([Pkg.PackageSpec("LanguageServer"), Pkg.PackageSpec("PackageCompiler"), Pkg.PackageSpec(name="JuliaFormatter", version="1")]); using PackageCompiler; create_sysimage([:LanguageServer, :JuliaFormatter], sysimage_path=dirname(Pkg.Types.Context().env.project_file) * "/languageserver.so")'

   Resolving package versions...
    Updating `~/.julia/environments/helix-lsp/Project.toml`
⌃ [98e50ef6] ↓ JuliaFormatter v2.2.0 ⇒ v1.0.62
    Updating `~/.julia/environments/helix-lsp/Manifest.toml`
⌅ [a80b9123] ↓ CommonMark v0.9.1 ⇒ v0.8.16
  [a8cc5b0e] + Crayons v4.1.1
⌅ [864edb3b] + DataStructures v0.18.22
⌃ [98e50ef6] ↓ JuliaFormatter v2.2.0 ⇒ v1.0.62
  [70703baa] - JuliaSyntax v0.4.10
  [bac558e1] + OrderedCollections v1.8.1
        Info Packages marked with ⌃ and ⌅ have new versions available. Those with ⌃ may be upgradable, but those with ⌅ are restricted by compatibility constraints from upgrading. To see why use `status --outdated -m`
Precompiling packages finished.
  5 dependencies successfully precompiled in 40 seconds. 50 already precompiled.
✔ [07m:10s] PackageCompiler: compiling incremental system image

real	9m32.731s
user	21m39.131s
sys	0m20.219s
$ time julia -J ~/.julia/environments/helix-lsp/languageserver.so jlfmt.jl beer.jl 

real	0m1.125s
user	0m0.858s
sys	0m0.260s

You’re right. It was late and I misspoke. They have very similar names!

Thanks for taking the time to point me right :slight_smile: