Static compilation of a mixed python - julia application

General Usage
1

I have a python application which is enhanced with julia in the performance critical parts. The julia source is called with the julia modle of python, however, the full python code is executed with the python-jl interpreter, that, from my understanding, is actually julia running python with PyCall.

I have the need to compile it as a static executable that can be run on any machine. I know that for pure julia programs there are a bunch of utilities, however, if the main file is a python one that needs to be called with python-jl (which is actually julia running python), are there “easy” solutions?

(I do not care if the executable is 2 Gb because it needs both julia and python runtime)

2

The easiest solution is likely PackageCompiler.jl, should work with all Julia code (there are very few exceptions, I believe LoopVectorizaion.jl is (only?) one, too technical to get into here why but it’s only to provide speedup, so not strictly needed).

PackageCompiler can be used in several ways, not just to compile full apps, also libraries, or to sysimage, what you want, seemingly. I believe all the ways have some drawbacks, mainly the app or library is not going to be tiny (100+ MB if I recall realistic). It’s rather large but can be made smaller through non-default options. First is to check if this works at all for you.

Compiling with it, even for a full app, will not provide you with just one .exe file, but a folder of files, but I’ve seen a YouTube video about how to use it and make a (Windows) installer (i.e. one file), I don’t think it’s outdated, but you need to do things a bit differently (also because of Python):

You can find some ways elsewhere how to distribute Python apps (without Julia) as an .exe, but they might be incompatible with PackageCompiler. I suppose Python itself, and your Python code can be incorporated into the builder as explained above.

What is your goal with “the need to compile it as a static executable”? Is it just easy distribution (and/or) fast startup (“TTFX”) of Julia? Or to hide the source code (it’s possible with non-default Julia options, otherwise it will be bundled with)?

It’s very possible you don’t actually need to AOT compile Julia, and then I would suggest juliacall (part of PythonCall.jl), a PyCall replacement, that will download Julia for the Python user.

StaticTools.jl is another option, it only works with a restricted subset of Julia, but if it works (or if not, then with slightly changed Julia code), then you get tiny (on the order of kilobytes for a small demo) .dll (or .so etc.) or apps.

3

The python package julia_project may help. This is for exposing a Python interface to the user that has Julia dependencies. It manages compiling and loading an image using PackageCompiler.jl.

I’m a bit confused about what you are doing. It sounds like you are calling from julia → python → julia. Probably just a misunderstanding.

Your Python project will depend on julia_project. The goal was to isolate the user from Julia to the greatest extent, which is probably more than you need if this is only for your own use. But it is quite modular.

It will

  • search for Julia on your machine, and offer to download it if not found
  • Instantiate the Julia project defined inside your Python project. (i.e. download the Julia Packages)
  • Optionally compile a system image that will be automatically loaded when you next import the Python module (and call an init function)

All of this is optional and all questions are asked at the beginning and stored, so you don’t have to monitor the process. They can also be controlled by env vars for automated installation.

1 Like