Juliacall and small binaries?

General Usage
, , ,
1

With the imminent release of v1.12, I’m wondering how small binary compilation might subsequently facilitate Julia integration into non-Julia codebases? Obviously, one could write wrapper packages in [R] or Python, which call binaries compiled using Julia, and then load those packages into the Python, [R] package/script. But could there be a more direct route, integrating Julia code into the python or r codebase?

I’m imagining a juliatrim’‘’ ‘’’ code block that checks if the relevant compiled files already exist for the given architecture, directly calling those binaries (without requiring a Julia installation on the computer). If the binaries do not exist, then it calls Julia to compile to specified architectures, saving to the project folder under some (modifiable) convention and then calls the binaries. Thus, a team member who does have Julia can run the compilation and push the binaries. Then, team members who do not program in Julia and have no interest in installing Julia can seamlessly use functions written in Julia where these functions are helpful for adding performant features.

Something like this would provide the most seamless way to, eventually, grab share from other languages. Python or [R] codebases can persist while an increasing share of the code itself is written in Julia. Eventually, with enough of the code written in Julia itself, teams will opt to ditch what has become an unnecessary Python wrapper. It’s a gradualist approach that does not require forcing users into an all-or-nothing refactor, slowly habituating teams to a new language.

2

sounds brittle, the increase in performance and what not does not sound like it will be worth it given we already have Guide · PythonCall & JuliaCall

3

The problem with JuliaCall is twofold: (1) you end up getting the overhead of spinning up a Julia session. This adds 2Gb of RAM requirements to the python code, and, although TTFX has improved, causes a compilation delay. (2) Relatedly, A JuliaCall code block necessitates that everyone working on or using a repo must have Julia installed. Some team members will be categorically opposed to installing Julia but might not care if there exists a Julia code block if they do not need to install Julia.

See: What's bad about Julia?.

The -trim feature addresses the above problem. Probably, the intended way that -trim solves the above problem is by allowing Julia to be the ‘second language’ in Python or [R]. Julia compiles the binaries, a wrapper package written in [r] or python calls the binaries, the wrapper package and binaries are installed from e.g. CRAN, and the python/[R] code using the binaries loads the wrapper package. However, particularly in the data science space, many users will not be writing wrapper packages, particularly if the intended use is not a major development effort. Directly integrating the ‘second language’ into the first-language’s codebase will lower the perceived costs of adoption.