I am trying to use PackageCompiler to build a stand-alone app. Since I need to potentially ship it to different host systems, I need to cross-compile my project. The PackageCompiler’s docs say nothing about it (they really just explain how to use the basic tool), but peeking into the source code I found the variable
- `cpu_target::String`: The value to use for `JULIA_CPU_TARGET` when building the system image.
which in turn, in case of Linux-x86_64, is "generic;sandybridge,-xsaveopt,clone_all;haswell,-rdrnd,base(1)". My question is: how can I build a cross-platform app with Julia? Say, I need to run it on Linux-arm, or MacOS. Should I set the cpu_target variable? How?
I’m not familiar with PackageCompiler internals, but my guess is that you don’t find anything about cross-compilation in the docs simply because you can’t: julia isn’t a cross-compiler at the moment. JULIA_CPU_TARGET is only to choose a specific microarchitecture within the host architecture, but you’ll likely be unable to target a different architecture, and a different operating system.
If your code is on GitHub, perhaps you can use GitHub Actions CI/CD service to build your application.
I see, thanks. It could be possible in principle, tho, right? For instance, translating Julia code to LLVM, then compiling with clang passing the cross-compilation flag?
In principle, yes, that’s actually what GPUCompiler.jl does (without involving clang) for generating code for GPUs and other kind of devices, but
on x86_64 the llvm which comes with julia by default doesn’t have the arm backend, so you can’t that architecture, and viceversa
you can’t really target different operating systems as there are lots of implicit assumptions that the host and the target OS are the same, so even without the same architecture the LLVM IR code you’d generate on Linux for Windows would be wrong, whenever something system-level is required.
So in the end the practice is much far from the theory. Not that cross-compilation isn’t desirable, quite the contrary, but it’s just not something anyone had a burning need for to the point of tackling all these issues.
LLVM-IR itself is agnostic to the OS/platform you target, but julia itself (which produces that LLVM-IR) isn’t aware of that. You can think of it more-or-less as julia assuming that all code it generates will run on the same machine it is generated on (with all nastiness that entails, like OS specific bindings etc). The detailed implications of what that means and how incompatible the generated code is depends on your usecase.
As far as I know, that’s more a case of not all backends supporting everything in the IR, right? Are there actually some special invocations in the IR that only work on a single backend?
I think llvm.vscale is aarch64-specific in practice? And I’ve seen lots of (maybe undocumented) llvm.<backend> intrinsics.
It’s true that you can have “valid” target-agnostic LLVM IR to a certain extent, but it may not be optimal for some targets, or actively bad, for example if you get vector registers width wrong.
Maybe it’s a far fetch suggestion, but if you consider you PackageCompiler App as a “binary” you are building for multi-platforms, maybe you can use BinaryBuild.jl. I don’t know if it is possible (how to get the app artifact?) but it is how I would try.
Thanks to all! Well, that’s a bummer, having binary code from pure Julia would have been just perfect. A possible workaround I can think of is to have multiple versions of Julia installed with juliaup, then using PackageCompiler with Qemu, but I don’t know if it’s possible. Anyway, for now I think I’ll stick with requiring Julia on the target machine.
For a Julia example, someone explained to me that something as simple as if Sys.iswindows() include("windows_only_functions.jl") end runs on Windows, after which those compiled functions will only be suitable for Windows. A cross-compiler needs to know the target at source level.
To expand on what Benny said, making the Julia compiler able to cross-compile would be only the first step, because of user code (in packages, mostly) that assumes it’s executing on the host.
It also occurs to me that things like this do exist. Java is often compiled to bytecode that can be used on any platform with a Java virtual machine (JVM) to handle the platform-specific interpretation or compilation. The thing is, what if there’s some platform-specific features the bytecode and JVM do not abstract away yet? Java provides the Java Native Interface, which in part allows Java to use platform-dependent functions in other languages, even assembly (there are also other alternatives for this purpose). However, the bytecode will be locked into platforms that the source code implements and checks.
Foreign function interfaces and handling platform-specific features in source code are pretty cool and often necessary, and I’m sure we’d like a cross-compiler that can accommodate those, not relying on changes to LLVM.
