I have a large Julia project with tens of thousands of lines of code. It requires two custom C++ libraries (I wrote), along with a combined 13 foreign dependencies, these are libraries not maintained by me.
As far as I see it there are 3 options to distributing this Julia package - for all platforms - like this:
pre-compile all libraries, host them somewhere as artifacts
This would be the best option but I don’t have access to a mac or Window 11 machine to pre-compile them on. I can’t afford to buy them just for this.
run build.jl as root
This would give me full control of the machine so I can just ask their package manager on debian/fedora, or homebrew on mac to install all the dependencies for me, then it’s a simple make install for my libraries.
This would require root access and it would modify files unrealted to Julia so I’m very hesitant about this but it’s starting to look like the only way forward.
Use BinaryBuilder.jl
I have been unable to get it working, I have tried for hours. The wizard crashes for me and the sandboxed environment does not allow me to use a package manager. I tried checking for _jll packages of my dependencies but some are missing, so I have no way to install that dependency into the sandbox, afaik.
I am also unsure as to how binary builder would work on windows, it’s a linux environment and it uses GCC, not MSVC. Does this mean windows users will have to install MinGW?
Here is a complete list of my dependencies, if it helps:
Thank you for pointing this out, I should’ve read the intro page more carefully. If this is the case than this entire endeavor is pointless anyway since I need to ship for windows.
I will have to manually build all dependencies during build.jl then. Thank you for taking the time to respond
I think there was a bit of miscommunication. BinaryBuilder can build for windows, which I have the impression is all you want. Do you actually need BinaryBuilder to run on windows? You already said that you are not using windows yourself. You can use BinaryBuilder to make compiled jll of all your custom C++ dependencies for any platform, right?
What do I do about the dependencies that do not have a jll package yet? I would have to also write binary builder package for those in order to be able to use them, but those libraries are not maintained by me and I have no experience compiling them. Or am I misunderstanding something about how binary builder dependencies work?
I can’t be sure that the target system has those libraries installed globally. The julia package installation should be a one-liner I don’t want to ask my users to install dependencies for me.
Maybe getting some inspiration by packages like CUDA and AMDGPU can help. For instance, AMDGPU.jl relies on a very large and difficult to compile set of libraries, which are all distributed as jlls through the Yggdrasil repository. Someone used BinaryBuilder to prepare building recipes for all of them. However, AMDGPU.jl also expects kernel drivers to be installed for the GPU. These have to be installed by the OS administrator and the Julia package simply tries to have clear error messages (with good suggested next steps) if the drivers are missing.
You will have to decide for yourself whether the dependencies you have are something reasonable to distribute through Yggdrasil (I would say any userspace dependency should be done that way) or should be the responsibility of the OS admin (e.g. kernel drivers) (in which case I would suggest sticking to very detailed error messages, not shelling out to apt-get and company).
Back to the question of dependencies: Some time ago I was working on the tectonic package. You can see its build script here: Yggdrasil/T/tectonic/build_tarballs.jl at master · JuliaPackaging/Yggdrasil · GitHub – you can see how in it you can declare that to build tectonic you need to first pull in other jll packages from Yggdrasil – these are build-time dependencies.
I would suggest starting with the simplest dependency you have and compiling it only for a single architecture, to make sure things work and you are comfortable with the workflow. I personally jump between using the wizard and editing the build recipe manually. You can also submit draft PRs to the Yggdrasil repo, and other folks might be able to help you there as well.
At this rate I think I will simply attempt to track down a download link for each OS-specific version of each shared library I need, download them locally, then build against that, possible using BinaryBuilder for that step only, which should only be a few lines.
I saw that GLFW.jl appears to do the same. I don’t think it’s feasible for me to compile libGL.so, which is OpenGL, from source, as I expect it itself to have a number of OS-specific, driver-specific dependencies, who may themself not have a jll package yet.
I do realize now that taking over the users PC and installing things globally for them would be inappropriate. Thank you again.
It seems to me this is asking the wrong questions:
OpenGL is bundled on macOS, you will probably want to link against the system OpenGL library (but that only goes up to OpenGL 4.1, see e.g. here for more information.
as far as I know, the story is somewhat similar for Windows
CxxWrap, like nowadays most Julia packages with C/C++ code dependencies, is using JLLs, so of course its C++ components are available that way
libjulia_jll is a bit of a red herring: it only exists to facilitate building of JLLs which need to link against Julia, and thus is only meant to be used as a build dependency for JLLs. If you are building code on your own resp. a user’s machine, you should link against the libjulia provided by the Julia version used by you / your user
In general, based on my experience, I think it is far easier to support a couple JLLs than to try and build stuff on user’s systems. The latter IMHO really only makes sense if there is just a handful of users to whom you are ready to provide direct support. Otherwise, it just doesn’t scale, because there will be an endless stream of issues with it:
people who don’t have the required compilers installed;
or just in a version that is incompatible with what you need;
people who have subtly broken versions of some of your dependencies installed;
people whose system simply is different in a way you did not expect;
differences in versions of the various dependencies you could not anticipate;
people for whom your code works, until they update something unrelated, and then your code breaks (but of course in the bug report the preceding update won’t be mentioned because in the user’s mind it is not related);
and many, many more.
Most of these problems just don’t exist for JLLs, or in a much, much smaller scope.
How do I build against something on the users system using BinaryBuilder? Afaik it is completely sandboxed and I have no access to the users globally installed libraries, or is that incorrect?
For example the build environment for macOS in BinaryBuilder gives you the macOS SDK, so that you can effectively link to system libraries (the SDK contains only stub libraries, not the actual implementations, but that’s enough for dynamic linking purposes).
You don’t do that. You build nothing on the user’s systems, that’s the point.
Instead you build the code once into a JLL (via Yggdrasil) and then your package depends on that JLL, so that the resulting binary artifacts are automatically downloaded by Julia’s package manager
I wanted to offer an update to anyone stumbling on this post in the future.
OpenGL libraries for linux-architectures are provided by GLU_jll, while the OpenGL extensions can be loaded using GLEW_jll. All the gnome-related dependencies (GTK, GLib, etc.) are also dependencies of GTK4_jll, so just depending on it also automatically pulls those in.
I have been unable to get PkgConfig-based packages working inside the binary builder sandbox (which is how you usually build against GTK4 on desktop), so I ended up having to rewrite most of my CMakeLists.txt allowing overrides for every single include path and library path, then specify those as build variables within the binary builder sandbox. This way they can be loaded using the basic find_library cmake command.
BinaryBuilder does not seem to recognize gitlab links as repos, it will download the html of the page as opposed to the repo, so I was forced to use github-mirrors of all of them or find a direct download link.
I don’t think it is possible to build against a specific local system-supplied OpenGL or Julia library, I am reliant on whatever libraries GLU_jll and libjulia_jll provide.
Thank you all who donated their time to help me, if nothing else you convinced me that binary builder would be worth the effort and while I haven’t shipped yet, I’m hopefuly that the ~12h I spend on this will make things easier down the line.
Note, while I believe OpenGL is still available on macOS, in some form, it’s deprecated, so you may want to consider rather relying on Metal.jl or something. You should be able to confirm “deprecated” at Apple (note, I went to the link and got a pop-up stating so, but going out and back in, it no longer comes up, and only this outdated doc, without that word):
As explained there even if drivers have OpenGL extensions on non-Mac (or possibly maybe in the driver itself on Macs?), no extensions are usefully available on Macs, and no way around that, so also neither can cross-platform code rely on extension available in all cases. So I suppose you’re not trying to do use such. And with core OpenGL gong away at some point (will Apple really get away with that?), then what’s the best way to target its equivalent functionality on Macs (Metal), but also on other platforms? I’m thinking somebody must have thought of the in a cross-platform way (with or without Julia’s involvement). The future on other platforms is Vulkan, and you may want to use that, there is Vulkan.jl, but Vulkan is not officially supported on Macs… I believe there’s a Vulkan-to-Metal translation layer. Since you’re using GLFW.jl, which is for/depends on OpenGL, using it might also be in doubt (for Macs). Will it/or does already support Vulkan, or Metal, in addition to OpenGL? It would be nice to know of a good cross-platform way (with Julia), solid for the future. I suppose this post can and should be split off to a different tread by someone.
EDIT: I see now, even if OpenGL goes away in macOS, you need not worry too much, since there’s this project if you only used the OpenGL ES 3.0 subset of full OpenGL:
Thanks for the good link, so if you rather wish to migrate to Vulkan, than you can also use this to support it on MacOS (should this project be wrapped as a JLL for Julia?):
MoltenVK is designed to be an implementation of a Vulkan 1.2 subset that runs on macOS, iOS, and tvOS platforms by mapping Vulkan capability to native Metal capability.