Building -- and releasing -- a fully self-contained Application written in julia

-– This post is a follow-up to my previous post on compiling a binary for distribution: How to compile a portable binary (at least across macs) with `juliac.jl`. —

More cool news!! You can write software in Julia, and then compile it into a standalone application, and then distribute and/or sell that program! That’s a thing now! :smiley:

I’ve put the content from last post (compiling for distribution) into a script, build_app.jl, and added logic to turn it into an app. It compiles a julia program into a binary and then wraps it up with all its necessary dependencies into a proper macOS .app bundle. I put the script in a package called ApplicationBuilder.jl.
https://github.com/NHDaly/ApplicationBuilder.jl

And check it out: I used it to build a simple game, written entirely in Julia! You can see the code and download it here (even if you don’t have julia installed):
https://github.com/NHDaly/PaddleBattleJL/releases/latest

So far, build_app.jl only supports macOS, but we will extend it to build apps for Windows and Linux as well! @ranjan has been helping with that – thanks @ranjan!

In the rest of this post, I’ll talk in more detail first about how build_app.jl works and then second about what I had to do to make Paddle Battle into an app.

Most importantly, there are couple changes you have to make in your program to support it running as an app (skip to the code changes section).


Turning code into an app #

You could divide the build-and-release process into three steps: 1. compiling a binary, 2. bundling an app, and 3. releasing and distributing it.

1. compiling a static binary #

juliac.jl, found in PackageCompiler.jl, is a tool for producing a statically-compiled, standalone binary.

It runs your code through julia, outputting a dynamically-linked shared library, and then (optionally) compiles a c program into a binary that invokes that library. For our purposes, it turns .jl files into an executable binary.

This tool is super cool, and the majority of what I’ve done here is due to the awesome progress @lucatrv, @sdanisch, @jameson, @viralbshah, @TsurHerman and others have made there.

2. packaging an app bundle #

A static binary can give you a run-time performance boost by precomputing the compilation, but it won’t run on other people’s machines unless it is distributed together with all the libraries and resources it depends on, including the julia runtime.

That’s what an Application is for: it bundles together an executable binary and all its dependencies. build_app.jl creates Applications.

At least on macOS (which is all I’ve tackled so far), a .app bundle is just a folder that contains a specific structure, including a specific place for the executable and another place for supporting libraries.

build_app.jl simply creates that directory structure, and then puts the right things in the right places. It uses juliac.jl to compile your code, and puts the resultant binary (and all the supporting julia runtime libraries) into Contents/MacOS. It copies any libraries and resources you supply into the right places. These behaviors are controlled with flags.

Consider the example program from the repo, hello.jl. A “hello world” example app has to be different from a generic julia program: it cannot simply println("hello world"), since anyone who double clicks the app wouldn’t see the output! Instead, the simplest thing I could think of for it to do is open a window in your browser to supply its greeting:

    open(filename, "w") do io
        write(io, "hello, world\n")
    end
    run(`open file://$filename`)

When you bundle it into an app, like so:
$ julia build_app.jl examples/hello.jl "HelloWorld"
you can double click the resultant HelloWorld.app, and it says “hello” from the browser. If you look inside the app, you’ll see the binary and the julia runtime libraries:

HelloWorld.app
  ↳Contents
    ↳MacOS   # executables go here
      hello             # the compiled c program that invokes hello.dylib
      hello.dylib       # our julia code ended up compiled in here
      ... libjulia.0.6.2.dylib ...  # the supporting runtime libraries. lots of these...
    ↳Resources
      HelloWorld.icns  # if you don't specify a custom icon, this is copied from the Julia app.

(The Apple documentation says that supporting libraries and frameworks are supposed to go in Contents/Frameworks, but I thought it made sense to keep the julia runtime libs next to the executable, since organizationally, they are very tightly linked to it. It’s also nice to keep them separate from any user-supplied, app-specific libraries.)

This example is simple because it doesn’t have any extra dependencies. If your app has other binary dependencies (like SDL for Paddle Battle, discussed below), those have to be copied in as well. The -L flag is for that. If you have other resources like images or sounds, the -R flag will copy those into Contents/Resources.

And that’s basically it for bundling!

As far as I know, this is new – has anyone been doing this already? If so, how are you doing it? Is it similar to what I did here?

3. Releasing your code. #

At this point, HelloWorld.app is ready to be shared! However, before it will be accepted into formal distribution channels (app stores, etc), there are a few more needed steps.

Apple has a list of the things it requires here: Distributing software on macOS - Apple Developer

At least on macOS and Windows, one required step is code-signing. build_app.jl supports signing your app with the --certificate flag. And to support sandboxing on Mac, pass an entitlements file to --entitlements.

Putting all this together, here is the build command I used for Paddle Battle (see PaddleBattle/build.sh):

julia ~/src/build-jl-app-bundle/build_app.jl --verbose \
 -R assets -L "libs/*"  --icns "icns.icns" \
 --bundle_identifier "com.nhdalyMadeThis.Paddle-Battle" \
 --certificate "Developer ID Application: nhdalyMadeThis, LLC" \
 --entitlements "./entitlements.entitlements" \
 --app_version=1.0 "main.jl" "PaddleBattle"

Pulling in external libraries #

The last detail to cover is the libraries passed to the -L flag. If your code depends on any external binary dependencies, you need to copy them to your binary so they’re available on other computers. If any of them are unix-style libraries (.dylibs on macOS), and if they in turn depend on any other libraries, you have to make sure that all their dependency paths are changed to be relative paths rather than absolute paths, and that they don’t reference any dependencies outside your app.

This means, you have to find all your app’s dependencies, copy them, and modify them to point to relative paths. Here is an explanation on that:
macos - Why is install_name_tool and otool necessary for Mach-O libraries in Mac Os X? - Stack Overflow

We’d like to add a utility function for this as well, but it seems difficult to generalize.

Note that it can be hard to tell if you’ve gotten all the external dependencies, since the app will still run on your computer just fine. You can test it by sending your app to a computer that’s never had julia installed, or by inspecting all the app’s dependencies. I found this StackOverflow post to be particularly helpful: StackOverflow–Failure digitally signing a Mac app outside Xcode, especially the export DYLD_PRINT_LIBRARIES=1 trick.


Paddle Battle #

As an end-to-end test case, I made a simple Pong clone called Paddle Battle. The game is written entirely in Julia, using the SDL.jl graphics library, which was nicely put together by @jonathanBieler.
https://github.com/NHDaly/PaddleBattleJL

You can download the game from the releases tab, or from my game company’s website (I had to make one to satisfy Apple’s Developer requirements):
http://nhdalyMadeThis.website

The game is pretty simple because I want it to act as an example, with all its code available.

Code changes for building as an App #

The most pertinent parts of the PaddleBattle code are the changes required to compile it as an App.

julia_main(ARGS::Vector{String})::Cint #

julia_main is the entry point for any executable compiled by juliac.jl. It is the first part of your julia code that will be called.

One thing this means is that functions executed at global scope won’t be run when your program is started. That is, unless they’re used to initialize global-scoped variables. Instead, global-scoped function calls are executed during compilation. So you have to be conscious of that.

All your code, then, should be invoked from julia_main. Here it is for PaddleBattle:
https://github.com/NHDaly/PaddleBattleJL/blob/aa61980/main.jl#L508

change_dir_if_bundle() #

The next big thing is that when your code is executed from within the application, its current working directory (cwd) will change depending on how it was invoked. On a mac, if double-clicked through Finder, the process’s cwd will be /.

Therefore, if your code interacts with the filesystem at all (loads resources), it needs to be able to reliably detect when its running as a compiled app, and then navigate to a familiar location. The way I handled this for Paddle Battle is to copy all necessary resources into Contents/Resources/, mirroring the structure of the repo root directory. Then, if the code is executing in the app, it changes directory into Resources, and the rest of the program can proceed identically, unaware of whether it’s an app or a normal julia program.
https://github.com/NHDaly/PaddleBattleJL/blob/aa61980/main.jl#L484-L500

I think most apps will want to do almost exactly that, so I’ve added this as a utility function in ApplicationBuilder.jl. Your code can simply call this function from julia_main(), and it should just work:

Base.@ccallable function julia_main(ARGS::Vector{String})::Cint
  ApplicationBuilder.change_dir_if_bundle()
  # ...
end

(Detecting the location of the compiled code is a bit tricky, and we’re still working out the best way to do that. For now, it uses the Base.julia_cmd() trick above.)

Package dependencies #

The last change is one that needs to take effect during your code’s compilation, not at runtime. (So, remember, that means it has to be globally-scoped code.)

If your code or any of the Packages it uses make ccalls, the relative path of the shared library is baked in at compile time. This means that while your code is being compiled, it needs to use the correct, relative path locations for its ccalls.

In the case of Paddle Battle, the only package I used was SDL.jl, which makes calls into three shared libraries. Before compiling any functions that make ccalls into those libraries, I need to change the location where it will look for those libraries!

SDL.jl uses BinDeps.jl to manage its dependencies, so it stores the libraries’ paths in the variables libSDL2, libSDL2_ttf, etc. The code needs to detect when its being compiled for an app, and in that case override the locations for those libraries, such that the ccalls will be compiled with a relative path.

To do that, I’ve set up build_app.jl to set an environment variable that indicates the code is being compile for an apple bundle. PaddleBattle checks for that environment variable, then overrides the locations of those libraries before they’re used. It does this in a global-scoped if-block right after using SDL:

using SDL

# Override SDL libs locations if this script is being compiled for mac .app builds
if get(ENV, "COMPILING_APPLE_BUNDLE", "false") == "true"
  #  (note that you can still change these values b/c no functions have
  #  actually been called yet, and so the original constants haven't been
  #  "compiled in".)
  eval(SDL2, :(libSDL2 = "libSDL2.dylib"))  # `eval` executes this line inside the SDL2 module.
  eval(SDL2, :(libSDL2_ttf = "libSDL2_ttf.dylib"))
  eval(SDL2, :(libSDL2_mixer = "libSDL2_mixer.dylib"))
end

(Note that those lines have to come after using SDL, so that your definitions override the default, and not the other way around!)

The old values for those globals were absolute paths (e.g. /Users/nhdaly/.julia/v0.6/Homebrew/deps/usr/lib/libSDL2.dylib). I set the new definitions to be simple relative paths – just the name of the lib. That’s because the -L flag will copy the binary into the .app bundle’s Libraries folder, and build_app.jl adds a library search path to the compiled binary to search in that folder. Note also that the -L step happens before compilation, because the libraries have to be available in order for the ccalls to compile. (The Libraries directory is then made available via LD_LIBRARY_PATH while compiling, and set via install_name_tool on the final binary.)

You’ll have to do this for any filesystem dependencies a package has, not just binary dependencies. For example, if your app uses Blink.jl, you’ll likely also want to include main.js, main.html, and other resources. Here is what @ranjan used for an app that uses Blink:

using Blink
if get(ENV, "COMPILING_APPLE_BUNDLE", "false") == "true"
     eval(Blink.AtomShell, :(_electron = "Julia.app/Contents/MacOS/Julia"))
     eval(Blink.AtomShell, :(mainjs = "main.js"))
     eval(Blink, :(buzz = "main.html"))
     eval(Blink, :(resources = Dict("spinner.css" => "res/spinner.css",
                                    "blink.js" => "res/blink.js",
                                    "blink.css" => "res/blink.css",
                                    "reset.css" => "res/reset.css")))
     eval(HttpParser, :(lib = "libhttp_parser.dylib"))
 end

And that’s it! There are just a few code changes you have to make, and a little bit of build configuration, and then in theory you should be able to turn any julia program into a compiled, portable, distributable app!

The next step for Paddle Battle is putting it on the Mac App Store. I’ve gotten a Developer ID from Apple, and I’m going to start the App Store submission process this week. I think it will be really cool for Julia to have presence on the app store! I will post another followup post when that’s completed.

96 Likes

So cool! Thank you very much for the writeup.

6 Likes

Wow! Thanks for this @NHDaly! I think this should be pinned or posted somewhere for more permanent reference. Maybe expand the README in your ApplicationBuilder package?

3 Likes

Crosslinking Julia on iOS and Android

1 Like

Awesome work! I will definitely be checking this out and would be willing to lend a hand (time permitting). In that vein, have you considered using the various @ paths from DYLD for macOS? I think you should be able to implement change_dir_if_bundle() and handle updating the SDL library paths using @loader_path or @executable_path (see man dyld for more info). Alternatively, adding your local bundle’s resources path to DYLD_LIBRARY_PATH might fix the SDL library location issue in a more generic fashion (so that even dependencies not using BinDeps.jl should work).

1 Like

Awesome work! I will definitely be checking this out and would be willing to lend a hand (time permitting).

:smile::smile: Thanks, that would be awesome!

I think you should be able to implement change_dir_if_bundle() and handle updating the SDL library paths using @loader_path or @executable_path (see man dyld for more info).
Alternatively, adding your local bundle’s resources path to DYLD_LIBRARY_PATH might fix the SDL library location issue in a more generic fashion (so that even dependencies not using BinDeps.jl should work).

Sadly, I don’t think that is sufficient in this case, at least not without some changes to julia. The problem is that ccall compiles the paths its given directly into the machine code, not using @ paths at all. That means that resultant binary will look for those paths exactly, even if the lib is present in an @ path.

For example:

julia> using SDL2; init() = ccall((:SDL_Init,SDL2.libSDL2), Int32, (Int32,), 0)
>> init (generic function with 1 method)

julia> @code_typed init()
>> CodeInfo(:(begin
        SSAValue(0) = (Base.checked_trunc_sint)(Int32, 0)::Int32
        return $(Expr(:foreigncall, (:SDL_Init, "/Users/daly/.julia/v0.6/Homebrew/deps/usr/lib/libSDL2.dylib"), Int32, svec(Int32), SSAValue(0), 0))
    end))=>Int32

And so unfortunately that string ends up in the binary, in a way that can’t be changed without recompiling.

I’m sure there is a better way to show this, but I found the string using $ hexdump sdl_test.dylib. You can see that the .dylib contains the string literal (scroll right):

...
026b4d50  5f 63 63 61 6c 6c 6c 69  62 5f 2f 55 73 65 72 73  |_ccalllib_/Users|
026b4d60  2f 64 61 6c 79 2f 2e 6a  75 6c 69 61 2f 76 30 2e  |/daly/.julia/v0.|
026b4d70  36 2f 48 6f 6d 65 62 72  65 77 2f 64 65 70 73 2f  |6/Homebrew/deps/|
026b4d80  75 73 72 2f 6c 69 62 2f  6c 69 62 53 44 4c 32 2e  |usr/lib/libSDL2.|
026b4d90  64 79 6c 69 62 00 5f 63  63 61 6c 6c 5f 53 44 4c  |dylib._ccall_SDL|
026b4da0  5f 47 4c 5f 53 65 74 41  74 74 72 69 62 75 74 65  |_GL_SetAttribute|
...

And you can see that the .dylib is not recorded as a binary dependency in the normal way:

$ otool -L -v ./sdl_test.dylib
./sdl_test.dylib:
        @rpath/sdl_test.dylib (compatibility version 0.0.0, current version 0.0.0)
        @rpath/libjulia.dylib (compatibility version 0.0.0, current version 0.0.0)
        /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.50.4)

So I do change the binary’s @ paths to look inside the binary (here), but you have to also change the code to use relative paths so it will look in the @ paths.


And it’s even harder for Blink, since it’s opening file resources through an absolute path, not calling a unix-style dll (As one example, it opens Electron.app, which was renamed to Julia.app when installing Blink).

So sadly I don’t think that would work. That said, you did make me think that maybe this could be somewhat automated by overriding or modifying BinDeps.jl, so that it produces relative paths when being compiled ApplicationBuilder.jl! That way the user wouldn’t have to make this change themselves… It would only work for dependencies using BinDeps (so not Blink files, for example), but it could make it easier in some cases! I’ve opened an issue for that idea here:

Sorry I never updated this earlier, but I did publish the App on the Mac App Store!

April 26th, in fact, so i’m almost three months late with this post!! :persevere::persevere::persevere:

https://itunes.apple.com/us/app/paddle-battle/id1370991412?mt=12

So that’s exciting!

I’ll be giving a talk at JuliaCon2018 in London next month, which will be a tutorial on how to compile, bundle, and release your own programs written in Julia from any OS! :slight_smile: See you there!

30 Likes

Oh hey that’s awesome, congratulations! This is one of the very few times that I’ll actually go to the Mac App Store willingly! :wink:

1 Like

:smiley: Thanks!!

This is one of the very few times that I’ll actually go to the Mac App Store willingly! :wink:

Haha yeah agreed…

Many congrats for JuliaCon2018!!! I am happy to hear that, and would love to read your presentation (unfortunately I will not be there), maybe you will publish it afterwards?

1 Like

:smiley: Thanks @lucatrv! :slight_smile: It’s been really fun working with you on all of this!! I’ll definitely publish it afterwards. And I think the talks might even be recorded :fearful:

2 Likes

I see people linking to this thread from Reddit. I think this package is outdated, and redundant with PackageCompiler.jl. Am I wrong?

Yes. PackageCompiler is what should be used now.

3 Likes

+1.

I have added a deprecation notice to ApplicationBuilder. I’m sorry I hadn’t done that earlier! :slight_smile: Thanks for the reminder.

There are still some things that ApplicationBuilder provided that PackageCompiler doesn’t, like a Windows installer and MacOS .app bundling. But I don’t think many people were using those, and the code is now out of date. If anyone else wants to pick up the package again though, please let me know! It shouldn’t be very hard to get it up and running again.

2 Likes

About “Almost everything this package did is now supported in PackageCompiler.jl itself, out of the box.”

What did you have in mind with “almost”, wasn’t it a bit more user-friendly (faster to compile?), while you get the same end-result? Or are you just being conservative, in case you’re not sure the other package has all the same features?

Also would you know if:

is also outdated, redundant with PackageCompiler.jl? I’m not sure, it seems to have other priorities.

It doesn’t load on Julia 1.5.1, but seems to work on 1.3:

[..] ERROR: Failed to precompile StaticCompiler [81625895-6c0f-48fc-b932-11a18313743c] to /home/pharaldsson_sym/.julia/compiled/v1.6/StaticCompiler/9DKHt_ElFpo.ji.

on 1.3:

julia> @time using StaticCompiler
[ Info: Precompiling StaticCompiler [81625895-6c0f-48fc-b932-11a18313743c]
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/7/lto-wrapper
OFFLOAD_TARGET_NAMES=nvptx-none
OFFLOAD_TARGET_DEFAULT=1
Target: x86_64-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Ubuntu 7.5.0-3ubuntu1~18.04' --with-bugurl=file:///usr/share/doc/gcc-7/README.Bugs --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++ --prefix=/usr --with-gcc-major-version-only --program-suffix=-7 --program-prefix=x86_64-linux-gnu- --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --libdir=/usr/lib --enable-nls --enable-bootstrap --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --with-default-libstdcxx-abi=new --enable-gnu-unique-object --disable-vtable-verify --enable-libmpx --enable-plugin --enable-default-pie --with-system-zlib --with-target-system-zlib --enable-objc-gc=auto --enable-multiarch --disable-werror --with-arch-32=i686 --with-abi=m64 --with-multilib-list=m32,m64,mx32 --enable-multilib --with-tune=generic --enable-offload-targets=nvptx-none --without-cuda-driver --enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu
Thread model: posix
gcc version 7.5.0 (Ubuntu 7.5.0-3ubuntu1~18.04) 
 18.829799 seconds (1.45 M allocations: 77.920 MiB, 1.07% gc time)

One intriguing compiler is:

(@v1.6) pkg> add https://github.com/jpsamaroo/BPFnative.jl

and GPUCompiler.jl it builds on, while neither is general-purpose, and the former for running Julia code in the Linux kernel, not having anything to do with GPUs… despite that dependency.

Hmm, no I don’t think this could have been any faster to compile, since this package was always calling PackageCompiler under the hood.

The main thing that ApplicationBuilder did was wrap up various configurations to PackageCompiler, which now already has sane defaults for create_app(), and then apply some postprocessing steps, which i’m not sure if PackageCompiler is doing or not. We were doing extra rpath stuff afterwards for MacOS relocatability, and creating a windows installer, and some things like that.

Finally, the other main thing was trying to figure out a way to package up binary dependencies, but it never worked very well, and I think just encouraging julia packages to use Artifacts instead is a better approach.


But yeah, like I said, I might be wrong about the above. If there’s still useful things this is doing, I’m happy to add other maintainers so we can revive it! :slight_smile:

1 Like

Don’t expect StaticCompiler to do much useful right now. Development of that is on hold. I’m holding out for Keno’s Compiler3 work and updates to GPUCompiler.

The goal is small, self-contained applications. If it works, it’ll end up being some static subset of Julia meant to target standalone libraries and cross compilation to WebAssembly and embedded targets.

14 Likes