FYI: Cosmopolitan for e.g. Julia portability (potentially)

reconfigures stock GCC and Clang to output a POSIX-approved polyglot format that runs natively on Linux + Mac + Windows + FreeBSD + OpenBSD + NetBSD + BIOS with the best possible performance and the tiniest footprint imaginable.

See at: Cosmopolitan Libc: build-once run-anywhere C library

Cosmopolitan makes C a build-once run-anywhere language, similar to Java, except it doesn’t require interpreters or virtual machines be installed beforehand.
[…]
For details on how this works, please read the αcτµαlly pδrταblε εxεcµταblε blog post.

I proposed the rather cool αcτµαlly pδrταblε εxεcµταblε way back for PackageCompiler.jl, but then the issue was closed (I do not recall if Cosmopolitan existed then and what I see now claimed), and possibly Cosmopolitan, building on it, makes portable Julia now more viable if anyone feels like trying a hack to make it work.

Platform Agnostic C / C++ / FORTRAN Tooling

Who could have predicted that cross-platform native builds would be this easy? As it turns out, they’re surprisingly cheap too.

Julia could be the first portable language/binary that includes fast JIT of native code.

I’m thinking of the showstoppers, Julia is implemented by several languages, C, C++ (not STL for itself? while “LLVM makes heavy use of the C++ Standard Template Library ( STL )” and others?), FemtoLisp (should probably work) and Fortran.

This wouldn’t make Julia fully portable, to e.g. ARM and POWER, but I can live with that. :slight_smile: And it might even be possible ARM according to the docs; or MacOS ARM emulation could work, but Julia itself could generate ARM code…?

Also Linux, means traditional ELF binaries, doesn’t imply would work on Android. That would be cool, if you first install Ubuntu on Android, which already can run Julia, potentially such (ARM) Julia build could work?

At some point there was unofficial Julia-lite, without OpenBLAS if it would help (there are also some interesting (partial) Julia replacements or also system OpenBLAS could be used), similar could be done for problematic dependencies, e.g. using Fortran, and still be useful.

It’s a bit unclear (since issue open), but it seems Python 2.7 already works this way (and other languages close, e.g. PHP):

and:

Cosmopolitan works great with C++ but very little effort has been made to integrate it with an existing STL.

STL will likely be a showstopper, unless we could use system LLVM and it’s the only STL using code? At one point there was Cfront “C with classes” (later renamed C++) to C transpiler, I guess nothing similar exists anymore…

1 Like

imho: only for the simple code …

For a complex codebase: it is not so easy to implement.
Now we have a

  • “Linux (Glibc)” and a “Linux (Musl)” based Julia * ( 32bit / 64 bit )
  • Windows (7+ / 10+ ) * ( 32bit / 64bit )

imho: the X86 - there is a new 4 micro-architecture feature level
( IMHO: not the tiniest footprint including all )

> x86-64: CMOV, CMPXCHG8B, FPU, FXSR, MMX, FXSR, SCE, SSE, SSE2
> x86-64-v2: (close to Nehalem) CMPXCHG16B, LAHF-SAHF, POPCNT, SSE3, SSE4.1, SSE4.2, SSSE3
> x86-64-v3: (close to Haswell) AVX, AVX2, BMI1, BMI2, F16C, FMA, LZCNT, MOVBE, XSAVE
> x86-64-v4: AVX512F, AVX512BW, AVX512CD, AVX512DQ, AVX512VL
2 Likes

Thanks for the link, I’m aware of feature levels, but wasn’t familiar with these exact terms: “x86-64-v2, x86-64-v3, and x86-64-v4”.

That doesn’t seem to be any problem. There’s not a different Julia binary (per OS) for those anyway. With a portable Julia binary across OSes, nothing changes. It (the core, not dependencies) can just target the most conservative x86-64 (and just ignore 32-bit, not a huge concern for me). At runtime when you invoke the backend, LLVM, it will generate appropriate code, for e.g. latest, even if LLVM itself is compiled for oldest. And if you want you can override just as you currently can.

You are quoting me, quoting the project “the tiniest footprint”. Yes, Julia will not be small, or smaller. That refers to the Cosmopolitan. Getting something like this working with Julia would be interesting and a first step, then making smaller might be a later step.

Yes, seems not supported. You can still just use specific Julia target that or some platform Cosmopolitan doesn’t support. Musl, is not needed for Linux except for some very obscure distros like Alpine. Getting Windows, Mac and Linux most use would be awesome, and I wouldn’t rule out Musl could be supported later.

The julia binary is actually only 12320 bytes. What’s taking the most space are dependencies:

-rwxr-xr-x 1 pharaldsson_sym pharaldsson_sym 184609760 mar 25 02:40 sys.so
-rwxr-xr-x 1 pharaldsson_sym pharaldsson_sym 71850032 mar 25 02:30 libLLVM-11jl.so
-rwxr-xr-x 1 pharaldsson_sym pharaldsson_sym 39025344 mar 25 02:41 libjulia-internal.so.1.6
-rwxr-xr-x 1 pharaldsson_sym pharaldsson_sym 30415776 mar 25 02:30 libopenblas64_.0.3.10.so
-rwxr-xr-x 1 pharaldsson_sym pharaldsson_sym 22521088 mar 25 02:30 libstdc++.so.6.0.26
-rwxr-xr-x 1 pharaldsson_sym pharaldsson_sym 7382888 mar 25 02:30 libgfortran.so.4.0.0
[…]

sys.so is the sysimage, and while it’s e.g. sys.dll on Windows, it’s not clear to me Cosmopolitan can’t work for such (e.g. could it be statically linked?). Worst case (for proof-of-concept), you could duplicate all dependencies, have .so and .dll, doubling in size… or tripling for MacOS too. Doesn’t sound nice, but you could also have one preferred platform, say Windows (as it’s very popular) with its DLLs, and dynamically download .so etc. on first use if you happen to run on Linux (or discover preinstalled Julia’s .so files). This likely solved potential STL issues, as they only cover libLLVM-11jl.so (or maybe more), but not the main julia or julia.exe?

imho: this is a default minimal Docker linux for testing .
and the docker official minimal images - based this linux.

for example postgres … has a debian(glicbc) + alpine(musl) version

for julia - it has a minimal disk savings

julia 1.6        22b56ebea464   8 days ago      486MB
julia 1.6-alpine 89e8f10e3115   3 weeks ago     408MB
1 Like

I’m mainly thinking of portable Julia for non-server use, to eventually support apps for end-users, and wouldn’t you then not be unlikely to use Docker?

I’ve never used it, nor had the need, I at least think it’s mainly for servers, and then you likely want to support only one infrastructure (or you can at least choose).

Linux is most used anyway for servers (even most used on Microsoft’s Azure), and I’m not too concerned about server use (current Julia binary options wouldn’t or needn’t go away). Even for Windows servers, Docker “runs on WSL 2”, so you would still use the same Julia Linux binary.