State of Compilation to Single Executable for Julia

General Usage
1

This seems like the right forum part to post this to, but please put it to the correct one if necesssry.

I saw some messages on the forum about compiling to executables, but nothing about the current state.

  • What is the current state?

  • Is PackageCompiler.jl ready for convenient everyday use to produce single fully functional binaries needing no fiddling?

4 Likes
2

My experience (1.1 and 1.2 with PackageCompiler) has been that it works well - we are using the binaries in production and the process to generate them is simple (no manual tweaks). But there are some caveats.

  1. Compilation is slow - 3+ minutes for even trivial applications. That means if/when I encounter a bug, either with the compiler or with the resulting binary, it can be a slow process to find a fix/workaround.
  2. Binaries are large because they basically include the entire Julia runtime (~180MB each).

So I’d say if you can live with those limitations, it’s a good solution.

10 Likes
3

@Usor here’s the next gen package compiler https://github.com/KristofferC/PackageCompilerX.jl

It’s still a WIP

1 Like
4

That is mostly a playground for me so I wouldn’t look too much there (especially not at this stage)

7 Likes
5

Thanks. That I cannot live with, I think.

1 Like
6

Thanks. I think I’ll keep an eye on that.

7

What is exactly necessary to speed up development of this state?

8

Nothing really, but it shouldn’t take too long.

I just want to point out that even with a perfectly working PackageCompiler, it will take a while before you will be able to create functional binaries without fiddling if you want to use a lot of packages. That is because many packages encodes things like absolute paths in their source code (by using the build step of package install to create a file with absolute paths that is then included). This means that they are fundamentally non-relocatable. Fortunately, we have something called the Artifact system in 1.3 which should make the build step pretty much obsolete. It might take a while for packages to start using that and for that new system to work perfectly though.

13 Likes
9

Thanks. In your reasonable estimation, considering all the knowledge you have, if I were to ask you how long it would be before this state is achieved, what would you say? A year?

10

It depends a bit what you want. If it is just a simple script or packages that are simple and behave well when it comes to relocatability then on the order of a month maybe. After that, it depends a lot on how the ecosystem will behave when it comes to adopting artifacts and “jll” packages (which are binaries and libraries built on top of the artifact system)

9 Likes
11

Excellent.

12

Do you use it on Mac or on Linux? For the later I couldn’t even make transferable app from the examples. Maybe I did something wrong though.

13

I’m building under Linux.

It’s probably worth starting a new thread if you have specific errors you’re trying to debug. The module dependencies you have do make a big difference.

As a basic example though:

$ cat ./hello.jl
Base.@ccallable function julia_main(ARGS::Vector{String})::Cint
    println("Hello Julia")
    return 0
end
$ PKGDIR=$(julia -e 'println(abspath(joinpath(dirname(Base.find_package("PackageCompiler")),"..")))')
$ julia $PKGDIR/juliac.jl -vae ./hello.jl
$ /tmp/builddir/hello

Binary will run wherever, provided /tmp/builddir/hello.so and libjulia.so are present on the target system.

6 Likes
14

Thanks for the answer! My bad, I forgot that I was actually using ApplicationBuilder.jl, which wraps PackageCompiler. And there I even opened an issue, but got no response. But you’re right, here is not the best place for this discussion.