Porting Julia to new platforms

Internals & Design
1

I cloned the Julia repo to my SmartOS box, intending to build it. I found out that:

  • it has a Makefile, not a configure script or a CMakeLists.txt file or the like;
  • it failed to build;
  • somewhere in the source code, there’s a file with lots of #defines relating to specific operating systems.

Is there a porting guide for people who want to get Julia to run on new OSes or processors? Is someone willing to get it to build with CMake or some such build system, to make it easier to port?

Besides illumos, I’d like to get it to run on DragonFly BSD, and I’ve been wanting to buy a big-endian Power box (Talos II, unless they come out with something new by the time I have enough money) running one or more of Adélie Linux (musl), FreeBSD, or OpenBSD to test portability of software I write.

2 Likes
2

I’d first check whether some of the big dependencies are supported or not in SmartOS.

llvm works on BSDs, and probably Solaris too. But I don’t think libuv supports Solaris (not sure, you’d have to check).

4 Likes
3

Recently julia merged the basic support for riscv, which can be used as a reference:

#system:riscv - Pull requests · JuliaLang/julia

Is someone willing to get it to build with CMake or some such build system, to make it easier to port?

Before starting actual CMake support work, I’d suggest opening an issue or posting a discussion on discourse.

julia does not currently support builds using MSVC.
As a Windows user, I’m interested in adding support for CMake and similar cross-platform build systems.

RFC: A transition to cmake as a build / configure system · JuliaLang/julia · Discussion #50194

2 Likes
4

On my SmartOS box:

[root@orshemesh ~]# pkgin se libuv
R-fs-1.6.3           Cross-platform file system operations based on 'libuv'
libuv-1.48.0 =       Cross-platform asychronous I/O
lua51-luv-1.43.0.0nb1  Bare libuv bindings for Lua
lua52-luv-1.43.0.0nb1  Bare libuv bindings for Lua
lua53-luv-1.43.0.0nb1  Bare libuv bindings for Lua
lua54-luv-1.43.0.0nb1  Bare libuv bindings for Lua
py310-uvloop-0.20.0  Fast implementation of asyncio event loop on top of libuv
py311-uvloop-0.20.0  Fast implementation of asyncio event loop on top of libuv
py312-uvloop-0.20.0  Fast implementation of asyncio event loop on top of libuv

I develop on Linux and have compiled some of my C++ projects on Windows, using Ninja (which was a lot faster than Nmake when I first tried it) and the compiler from MSVC. I’ve used the MSVC GUI to debug programs, but I build them on the command line. I’ve also used gcc in Windows; IIRR gcc has 10-byte floats but MSVC doesn’t.

5

The discussion on GitHub mentions Meson. I have no experience with Meson, so can’t compare it with CMake, but if Meson makes cross-platform building easier, it’s fine with me. Meson depends on Python, which CMake doesn’t.

6

I understand that sentiment, it is born out of the feeling “Wouldn’t my life be much easier if Julia used a standard build-system, like CMake or Meson”.
I am a big fan of Meson ( and after years of fighting with CMake for LLVM a strong dislike for CMake).

But it is important to ask the question: Why do the Julia developers didn’t use CMake? In many ways we have to solve three problems:

  1. Dependency management (and control). There is a tight coupling between a few dependencies (most notable LLVM and libuv) and we want to minimize the possible configuration set instead of having developers run into the same bugs multiple times, just because they don’t yet have a patch for a dependency.
  2. We need to build the Julia runtime and the compiler (this would be trivial for any build-system and even with Make it’s not a big deal)
  3. We need to bootstrap Julia itself, the Julia standard lib etc…

If you propose to switch the build-system to Meson or CMake you would need to answer the question: How would this make it easier for everyone developing the Julia core.

Right now as someone who just wants to work on a standard lib all I have to do is enter “make” and I get a reasonable approximation of a build that your CI system would also provide.

If you want to port Julia to a new platform the build-system really shouldn’t be the hardest part, you likely will need to turn off the BinaryBuilder provided caches, and build everything from scratch.

4 Likes
7

I don’t suggest doing that, since it has already been done multiple times and the outcome wouldn’t be much different than the last time this has happened. Valentin elaborated a little bit more about the motivations.

2 Likes
8

I looked at the pull request and there’s a lot of files changed. Is there a list of steps to port Julia to a new platform? Try to compile, and if you get this error, do that?

How hard would it be to add powerpc64be, which I mentioned above? Powerpc64le is already available.

9

I highly doubt that what you ask for exists.

10

Big-Endian is hard, we have no Big-Endian support so it is not just going in and porting it to a new architecture, but also going in and rooting out all the small little endian assumptions in the runtime, the language and Julia libraries.

Note that we currently don’t have an active maintainer for ppc64le and are considering dropping support for it due to the ongoing maintenance burden.

6 Likes
11

I want a Power box specifically to test that software that claims to work on big-endian architectures actually does.

If I get a Power box, which will take several more months, can I become the maintainer of ppc64?

12

In principle, yes, but it does require profound knowledge about both Julia and PPC64. It is very unlikely that we will be able to support -BE and there is very economic incentive to do so.