[ANN] AppBundler.jl - Bundle Your Julia GUI Application

Package Announcements
,
1

I am excited to announce AppBundler.jl :partying_face:

The package offers recipes for building Julia GUI applications in modern desktop application installer formats. It uses Snap for Linux, MSIX for Windows, and DMG for MacOS as targets. It bundles full Julia within the app, which, together with artifact caching stored in scratch space, allows the bundling to be done quickly, significantly shortening the feedback loop.

The build product of AppBundler.jl is a bundle that can be conveniently finalised with a shell script on the corresponding host system without an internet connection. This allows me to avoid maintaining multiple Julia installations for different hosts and reduces failures due to a misconfigured system state. It is ideal for a Virtualbox setup where the bundle together with bundling script is sent over SSH after which the finalised installer is retrieved.

The configuration options for each installer bundle vary greatly and are virtually limitless; thus, creating a single bundling configuration file for all systems is impractical. To resolve this, the AppBundler recipe system comes into the picture. AppBundler provides default configuration files which substitute a few set variables specified at the Project.toml in a dedicated [bundle] section. This shall cover plenty of use cases. In case the application needs more control, like interfacing web camera, speaker, host network server, etc., the user can place a custom snap.yaml, AppxManifest.xml and Entitlements.plist in the application meta folder, overloading the defaults. Additional files can be provided easily for the bundle by placing them in a corresponding folder hierarchy. For instance, this can be useful for providing custom-sized icon sizes. To see how that works, explore AppBundler.jl/examples and PeaceFounderClient where you can check out the releases page to see what one can expect.

All recipes define a USER_DATA environment variable where apps can store their data. On Linux and Windows those are designated application locations which get removed with the uninstallation of the app, whereas on MacOS, apps use ~/.config/myapp and ~/.cache/myapp folders unless one manages to get an app running from a sandbox in which case the $HOME/Library/Application Support/Local folder will be used.

Thought has also been put into improving the precompilation experience to reduce start-up time for the first run. For MacOS, precompilation can be done before bundling in the /Applications folder by running MyApp.app/Contents/MacOS/precompile. For Linux, precompilation is hooked into the snap configure hook executed after installation. For Windows, a splash screen is shown during the first run, providing user feedback that something is happening. Hopefully, the cache relocability fix in Julia 1.11 will allow us to precompile the Windows bundle as well.

The most challenging aspect of this package is crafting recipes. So far, I have yet to manage to get sandboxing to work on either platform which prevents applications from being accepted in corresponding marketplaces. In particular, the issues are:

  • Windows can start Julia and crash immediately. See issue #52007
  • MacOS can start GUI but is unresponsive. Present in both GTK and QML.
  • Linux can start GLWF by linking with the mesa-core22 snap content package. GTK apps with mesa-core22 segfaults, but works if unlinked. QML draws a window but does not render content #191

It would probably be reasonable to integrate PackageCompiler.jl for completeness and to resolve the sandboxing issues with Windows and MacOS in that way. One MacOS application is still standing in the Mac app store developed with now deprecated ApplicationBuilder.jl so the path is clear. However, at the moment, I am only considering investing time in it once PackageCompiler issue #164 is resolved, as there would be no benefit for the PeaceFounder project.

Another avenue worth exploring could be making a GitHub workflow, but making a Windows build setup seems like a pain in the ass. At the moment, I am doing postprocessing with shell scripts for MacOS and Windows.

30 Likes
2

I’m guessing this doesn’t handle edge-cases such as when user-dirs.dirs exists, XDG_CACHE_HOME is set, or someone has run SHSetKnownFolderPath on Windows?

I ask because these sort of edge cases motivated me to make GitHub - tecosaur/BaseDirs.jl: A cross platform implementation of the XDG Directory Spec.

4 Likes
3

This is good. I did not understand what the XDG environment variables were all about, but now, reading the specs, it makes sense. Your package looks like something which should work within the GUI application. Still, it needs to include the cases for situations where it is run from a snap environment, which has its own data directories SNAP_USER_DATA and SNAP_USER_COMMON where XDG directories are often pointed to. It is trickier for UWP, which does not provide an environment variable where user data is stored. I had to infer from the installation folder name and process it to get its location in the $LOCALAPPDATA/Packages directory.

2 Likes
4

This is pretty great, thank you for creating it! Is flatpak or appimage support on the horizon? Asking as I have the impression these are more widely supported than snap on linux.

2 Likes
5

Currently, my focus is making Snap support for the Linux platform better, and I have no plans to add Flatpak support as that would duplicate efforts. One of the key advantages Snap offers over Flatpak is the ability to create bundles entirely on other platforms using the widely available mksquashfs utility, which is also built with Yggdrasil. This contrasts with Flatpak, where flatpak-builder is necessary, and manual implementation would require considerably more effort. Depending on it would result in a finalisation step similar to MSIX bundles on Windows. In contrast, Snaps, coupled with a configure script that handles precompilation post-installation are complete and do not need any additional post-processing steps.

Another point worth noting is that Snaps are a good format for distributing server-side applications, which aligns well with the needs of my project. Therefore, while Flatpak has its merits, Snaps are easier to build and are more versatile.

2 Likes
6

It sounds like what might work best for you then is handling the special Snap/UWP cases yourself, and then using BaseDirs.jl for the rest (e.g. config/cache).

Oh, you could also manually do something like

if haskey(ENV, "SNAP_USER_DATA")
    ENV["XDG_DATA_HOME"] = ENV["SNAP_USER_DATA"]
end

before loading BaseDirs (or just call BaseDirs.reload() afterwards) and handle your special cases that way.

1 Like
7

This is supercool, as one of the limitations of Julia is that there was not an easy way to distribute double-clickable apps. Thanks for doing this!

Who know, maybe in the future this can be integrated into VScode, making creating apps as easy as it was in CodeWarrior!

3 Likes
8

Thanks, this is superfast to “build”, as expected, since it’s just bundling, not compiling (it’s not either or, both could be done theoretically, or partially compiling), first time slower if downloading stuff…, and also compressing a bit slower, though not very:

julia> @time AppBundler.bundle_app(Linux(:x86_64), ".julia/packages/AppBundler/bFhGI/examples/qmlapp/", "build/MyApp3-x64")
[ Info: Rule with origin linux/meta is skipped as not found in default or override path.
  Activating project at `/tmp/temp_env`
  Activating project at `~/.julia/environments/v1.10`
  5.528476 seconds (839.69 k allocations: 83.795 MiB, 0.33% gc time)

The docs seem very extensive and good, though I may look into improving minor points.

I’m trying out all the (GUI) examples, and as you can see they are rather large as expected:

$ du -hs build/*
696M	build/MyApp2-x64
1017M	build/MyApp3-x64
216M	build/MyApp4-x64
963M	build/MyApp-x64

MyApp4-x64 is the QML one, but note redone from the 5x larger uncompressed one, MyApp3-x64, done with (as in docs … except):

julia> AppBundler.bundle_app(Linux(:x86_64), ".julia/packages/AppBundler/bFhGI/examples/qmlapp/", "build/MyApp4-x64", compress=true)

I kind of expect the mousetrap example to be the smallest one, but it’s can’t be built since it doesn’t have a Manifest file, and I don’t know how to make one, only now needing it! I reported it and promised a PR when I figured it out, and he doesn’t have time now, so also if others want to do it or explain, even point to docs on that, which is elusive to find.

I’ll try to make an even smaller example, try a text-based “hello world” I expect should work, and may try it on the recent platform Julia game (with the Julia game engine).

I can run the apps, by locating the code in the folders (will finish, figure out one file-installation), but at least the one-file compressed is not the way to go for that:

shell> chmod u+x ./build/MyApp4-x64

shell> ./build/MyApp4-x64
/bin/bash: line 1: ./build/MyApp4-x64: cannot execute binary file: Exec format error

shell> file ./build/MyApp4-x64
./build/MyApp4-x64: Squashfs filesystem, little endian, version 4.0, xz compressed, 226076129 bytes, 21347 inodes, blocksize: 131072 bytes, created: Thu Jan 18 13:02:12 2024
9

I wrote a blog post reflecting on the future of AppBundler :dizzy:

On the potential of integrating bundling toolchains, which would make it easier to write CI deployment actions and integration tests. I discuss current issues in resolving sandboxing issues for MSIX and Snap and some further ideas about making Julia distributions for environments that take a lot of time to precompile.

I am also writing an NLNET grant application and would appreciate feedback, which is linked at the bottom of the blog post.

4 Likes