I’ve been doing some small proof-of-concepts while getting deeper on some of the technical aspects of Julia around compilation and deployment packaging. I’m drawn to Julia as a result of having seen the challenges with the 2-language problem in our domain. As further background, a key need for our work is developing engineering modeling, simulation and analysis tools, which we largely run in a desktop environment due to a need to manage an air-gapped arrangement. As a tool developer supporting engineers and analysts, my ideal would be an ecosystem where they could develop functionality – or I could assist – and then easily deploy this in a heterogeneous desktop environment. Examples of such tooling might be MATLAB’s GUI, Mathematica’s UIs, or the canonical “VisualBasic experience” – code, compile, deploy. Some important requirements implied by this:
Access to the whole of the Julia language an ecosystem
Usable by the average Julia coder
Cross-platform tooling, cross-compilation
Deploy to targets without a full Julia installation (the tooling itself and the compiled products)
Leverage Julia’s “native” UIs (e.g. Makie, Genie), while integrating cleanly into a full desktop tools environment.
Having read a number of threads, as well as worked on some PoC implementations, I have the following questions:
1. Feasibility: Will Julia’s future compilation infrastructure support such a tool? My concern here arises from threads such as this where it seems that there may be an inevitable trade in functionality, and potentially a need to be an advanced Julia programmer, to fully leverage a compiler. There seems to be some uncertainty in the community about this, so I’m inquiring at a technical level whether, given the design of Julia, this (as defined by the above requirements) is even possible?
2. Desirability: There seems to be anecdotal evidence in these threads and others that there is interest in this use case. We also have discussions such as this one in which we have an excellent demonstration of using a “VB experience”-type tool (Lazarus) to build a UI wrapper for Julia and get it deployed, though with a few bumps along the way. Moreover, since we have to fallback to Object Pascal for the UI, we’re back to the 2-language problem. My question is whether there are any survey results that shed more light on this, or whether such a question about core use cases and workflows could be considered for future iterations of the Julia Developer Survey in order to provide better insight into the level of interest for this?
The average Julia coder today has a local installation of Julia, downloads packages of source code, precompiles package-specified call signatures for a particular environment of versioned packages and Julia, and JIT-compiles other evaluated source code on demand. AOT compilation of binaries and standalone deployment is far less mature, though it obviously has its places so there’s a lot of interest or active development.
I don’t know all (or maybe most of) the ways to do GUIs over Julia code, but it typically involves scripts and web apps. Some interactive graphics are also often done with some plotting libraries and notebooks. As you’ve linked, it’s possible to use PackageCompiler.jl to deploy a standalone GUI app, but that’s relatively rarer for now.
There are many good reasons for that, which you have somewhat brought up.
Even if you don’t run into some of the rough edges, PackageCompiler products are often too big. It bundles the Julia runtime (which lets users not have to install it themselves), the packages, and whatever you specified for precompilation. That is far less than “access to the whole…ecosystem”, but it’s already a pain to distribute and can’t fit in some contexts like Arduino microcontrollers. Arbitrary eval and JIT compilation isn’t necessary to print “hello world”, but a Julia process can’t tell even if provided a script that only contains println("hello world").
The obviously follow-up idea is to ditch interactivity and bundle minimal code. That is exactly the trimming option of juliac, the upcoming successor to PackageCompiler. Until it comes out and further develops, we don’t know how it’ll do trimming or how well, but there’s no fundamental difference from familiar executables and shared libraries. Trimming costing functionality is not an alarming limitation; if you want your binary to just take up 1-2MB, you really don’t want the full Julia runtime around or compile arbitrarily more code. That will limit how we write the source code as well; for one example, we’ll have to specify all the code to compile ahead-of-time and remove runtime dispatches that try to JIT compile. If you don’t want those limitations, juliac defaults to no trimming, like PackageCompiler.
We don’t have cross-compilation of Julia code for different OSs and CPUs yet. As far as I’ve read, it’s technically possible but needs significant work; linking a much better comment from last year. This doesn’t really concern the typical workflow of locally compiling code on demand in varying package environments, so this work will probably need juliac to develop a bit. Cross-compilation is routinely used to run supportable Julia on GPUs, but that doesn’t amount to the same thing.
So while you can do some of the things you want, the Julia ecosystem isn’t as capable or mature as others. If you’re just curious and would like to witness the developments, that’s no problem. If you need to do real work, you need to consider how Julia is used in practice. I wouldn’t really know, but there does seem to be people using Julia on some level for modeling and simulation. Ask around, look at what they do, and see if that can benefit your own job.
The two-language problem doesn’t assert using >1 language is fundamentally bad, and as far as I know, no language has ever aspired to be the only useful one. Julia’s own core is written in C, many Julia packages wrap binaries compiled (and cross-compiled) from other languages, and there are a few packages for interop with other interactive languages like Python and R. The problem is developing in a more convenient, often interactive language then reimplementing in another language to perform efficiently enough in practice, and Julia doesn’t prevent that in all circumstances e.g. microcontrollers. Writing Julia to do one thing and Python to do another in cooperation is not the two-language problem. Writing Python that already performs nearly optimally because >90% of the executed code was compiled from C is not the two-language problem.
I am working on the AppBundler.jl project, which addresses many of the listed pain points for the Julia GUI application deployment. Indeed, Julia does not cross-compile, which can be resolved using continuous deployment services. You may need to deploy your own CI infrastructure to match security constraints for an air-gapped arrangement.
Currently, AppBundler.jl supports bundling using package images as a compilation product. Integration with sysimages using PackageCompiler is underway and will arrive at the beginning of autumn once other tasks are finished.