Any existing efforts on cross-compiling Julia programs across different platforms?

Hi everyone,

I’m looking at possibilities for cross-compiling Julia programs, which would enable compiling on one platform to run natively on another without setting up separate environments or relying on emulation. This would make it easier for us to compile binaries for multiple platforms from our platform of choice.

I searched for prior discussion and asked around on various Julia communities outside of the official Discourse and from my understanding native support for cross-compiling isn’t currently available in Julia.

I’m interested in contributing towards developing this capability and would like to connect with any existing efforts. If anyone is working on this or knows of groups focusing on enhancing Julia’s compilation process, I would appreciate guidance on how to collaborate or contribute to these efforts.

Using the latest stable version of Julia, my goal is to contribute effectively and avoid duplicating any existing work. Any pointers or information on ongoing projects would be highly valuable.

Thank you!

- Barzin

3 Likes

SyslabCC: Suzhou-Tongyuan’s proprietary Julia AOT compiler is now available for free use (personal & educational license only) - Community - Julia Programming Language (julialang.org)

The closest thing is StaticCompiler.jl / GPUCompiler.jl

In the GPU land, we are effectively cross-compile Julia and similar efforts exist for other exotic execution target, but it also allows us to sidestep some cross-compilation concerns since we ignore ABI issues in foreign-calls.

For https://arxiv.org/pdf/2208.01154, I wrote a minimal cross-compiler from x86_64 to AArch64, but it was a lot of manual fiddling.

It should be fairly straightforward to add target support to the compiler infrastructure
Choose target ISA in `@code_llvm` and `@code_native` · Issue #52949 · JuliaLang/julia · GitHub see choice to Target ISA in @code_llvm and @code_native by arhik · Pull Request #54589 · JuliaLang/julia · GitHub for an attempt.

The hardest question though is that Julia is “define by running code”, so you often encounter

if Sys.iswindows()
     f() = ....
else
     f() = ....
end

Effectively making defining target-specific decision about the availability/implementation of methods by running code at the top-level. This extends to 32bit vs 64bit, where the definition of Int differs.

So getting the first 30% for cross-compilation is probably straight-forward. Just add an option to the code-generator to use a different target (and ensure that the JIT cache doesn’t see the “wrong” code).

The harder question is how to deal with this “static” definition of alternative codes, in the GPU world we use something called “overlay method tables”,
but this feels like we ought to integrate it deeper into the runtime.

I am envisioning something similar to rusts target-features, that tags methods in the method-table and then one could compile a call-graph using a set of tags. Kinda running code on x86, but using a shadow call-graph for Aarch64.

The issue of data-types is also hard, but WIP: World-age partition bindings by Keno · Pull Request #54654 · JuliaLang/julia · GitHub might get is into the neighborhood of making it feasible.

So, long story short, depending on your needs it ought to be doable, but you are opening a rabbit hole.

Oh, and we need to update our LLVM builds to build for all targets.

9 Likes

The simple answer is, yes there is a lot of ongoing discussion in it, and it’s one of the things we (JuliaHub) will be requiring so there has been some efforts to start scoping out the work, but it’s not something that is far enough along to have a definite roadmap.