Julia in Linux distributions

Community
1

I’m not pushing anyone, just curious.

Is there, or should, Julia 1.0 be supported in Linux distributions?

I know the prebuilt binaries work, and that e.g. old Julia versions where added, I believe stuck at 0.5 in Ubuntu.

I understand maintaining Julia versions wasn’t a priority with Julia a moving target, thus it was stopped.

We may never replace Python/Perl, or be the default for new code, and to be relied on Julia would have to be preinstalled in all of the major distributions, so is it pointless to try (we can still make Julia apps, and distribute as binaries; or some other I’m not up-to-speed on, Docker/Snap/Flatpack etc.; is distro support/PPAs simply an outdated method, except for core functionality?)?

Still, is it now time, or wait for Julia 2.0…? :slight_smile:

I see there’s also a long thread I didn’t read:

–

Also, to really be able to rely on any Julia runtime (or any other alternative, Python/Perl, PowerShell), you would want the major platforms Windows (and pre-Windows 10?) having it to (and macOS, that only has Python 2, splitting Python support across Unix…), and there’s no such thing for any language (except JavaScript/WebAssemly…) so maybe prebuilt binaries are the thing to do, even though a like the idea of source code distributed.

1 Like
2

It would be nice, but as we pointed out in the other thread, a lot of repos simply move too slowly for Julia, at least as things stand right now. (Happily, Julia is already up-to-date in Arch :slightly_smiling_face:)

It would be nice if we could update the PPA for debian, but nobody has volunteered to maintain it. This is something I had considered a while ago, but as I’m increasingly on Arch now, I’m not going to volunteer for it myself.

3 Likes
3

Distribution-Specific Packages

The following distribution-specific packages are community contributed. They may not use the right versions of Julia dependencies or include important patches that the official binaries ship with. In general, bug reports will only be accepted if they are reproducible on the official generic binaries on the downloads page.

See here.

In theory, all major platforms (Windows, Linux, macOS) are supported, just not through their dedicated or chosen channel of distribution but through the official binaries from the website directly.

4

I think the OP was more interested in Julia being bundled in the official repo of a distro so that it would be easier to use Julia throughout the operating system, and especially so that the repo can also have packages which run Julia code (which is ultimately something we are very much going to want to be able to have).

I have given some thought to using Julia for more OS configuration type stuff such as display configuration, network configuration and custom udev drivers. Unfortunately the latency and lack of compile caching right now means that Julia is definitely in no position to replace Python for most of these kinds of tasks. Ultimately it would be great if Julia be a seamless drop-in replacement for Python (or Ruby, perl or even bash) by keeping it’s compiled binaries in the appropriate places in the OS so that it’d run when scripts are entered (i.e. compile caching), but for the foreseeable future it seems more likely that if Julia is used in this way it will involve fully compiled binaries from PackageCompiler.jl or whatever form that takes once it’s a full language feature.

I have been toying with the idea of writing a joystick driver for myself using PackageCompiler as a fun exercise. I’ve been itching to do something like this once I realized how easy it can be to write basic hardware drivers for linux. It’s always fun to get Julia involved with stuff that nobody would have imagined it doing a few years ago :slightly_smiling_face:.

5 Likes
5

For reference, the progress of the inclusion of julia 1.0 in distros’ package repositories can be tracked here: julia package versions - Repology.

At the time of writing, it is available, among a few others, in Arch (naturally), in Fedora Rawhide (the next version, I suppose?), and most notably, in Ubuntu 18.10.

4 Likes
6

Yes, Fedora 29 (to be released at the end of the month) already includes Julia 1.0.1.

1 Like
7

Thanks for the repology.org link.

I had in Ubuntu 16.04 julia0.5.2-xenical2 and I still have it after updating to Ubuntu 18.04.1.

While current Julia is in 18.10 (that’s not LTS), there seems to be none in 18.04

confirmed in Ubuntu – Package Search Results -- julia

I assume only reason I have any Julia is because I upgraded (would not have with a fresh install).

Is there a way to retroactively add Julia in old distros (I mean other than Ubuntu 18.04 Backports?), e.g. into 18.04 (I’m guessing 18.04.2 could be for that, but not wanted for 18.04).

If julia repo in 16.04 would upgrade from 0.4.5 (that’s no longer maintained by the Julia community) to 1.0.1, it would break code, for those few(?) already/still using, while there would be an upside to haveing latest. Should there be a new julia-stable (or julia1) for old and new distros, to not have that conflict?

The issue of Julia supported in at least the main distros, Ubuntu, Fedora (CentOS), Debian, Mint etc. will I guess resolve itself eventually, but I at least view it also as a promotion to have it in them by default (sooner rather than later). I would personally be ok with a cut down version, e.g. that excludes [Open]BLAS etc. Just having something small with few (big) binary dependencies (I’m not sure we should rely on distro packages to be up-to-date) could work, and if you need say BLAS, you could add it as a package. You would at least be able to from within Julia (and having an Internet connection…).

We have a long way to get Julia included in all Linux distros (or other Unix), I see e.g. from repology, no Linux Mint has and neither:

Absent in repositories
[…] AIX Open Source Packages
[…]
CentOS 6
CentOS 7
CPAN [these few and RubyGems, seem not to apply…]
MetaCPAN
CRAN
[…]
EPEL 6
EPEL 7
Exherbo
F-Droid
[…]
HP-UX 11.31
[…]
KDE neon Dev Stable
KDE neon Dev Unstable
KDE neon User
KDE neon User LTS
[…]
OpenBSD Ports
OpenIndiana packages
OpenMandriva 3.x
[…]
Raspbian Testing
[…]
Scientific Linux 7.x
[…]
Ubuntu 12.04
Ubuntu 12.04 Backports
Ubuntu 14.04 Backports
Ubuntu 16.04 Backports
Ubuntu 17.10 Backports
Ubuntu 18.04
Ubuntu 18.04 Backports
Ubuntu 18.10 Backports
Ubuntu 18.10 Proposed

8

What do you mean by “right now”? I agree lots is happening, e.g. 1.0.2, and 1.1 planned. A Julia package (or julia-stable as I proposed) at 1.0.1 could be upgraded to 1.0.2 as that’s only a bugfix release.

To 1.1, I think not(?), as it follows semantic versioning, while I’m not sure. But is it any difference from Python3 that means Python 3.6.5 for me and could be older in older distro. And while I’ve never seen more than one concurrent 3.x in the same distro, there’s 3.x and 2.x concurrent.

Would we need Julia1.0 and Julia1.1 packages etc.?

9

Yes, I was thinking of the possibility of relying on Julia within each distro [version], but also across versions, as you don’t want Linux more fragmented than it already is… I understand e.g. Arch doesn’t install Python by default (and if you do Python 3), so Python 3 (or old 2.7) scripts made e.g. for Ubuntu aren’t usable in that system (Arch used I believe bash to e.g. install packages).

It’s news to me that writing Linux [kernel] hardware drivers is easy (I’ve never tried, and maybe it just is, even for the kernel). I assume these examples you list however wouldn not be kernel drivers, and would only live in userspace.

The kernel itself is famously written only in C, not C++ (while similar for WIndows, I believe a possibility to make drivers for Windows and macOS in C++), at least not written in a GC language, nor the drivers. You may not get any other than C code, even for drivers mainlined, while I think you could use e.g. Rust language (that neither does have a tracing GC; you could however say it’s design is some form of GC), and possibly Julia…

It’s not that it isn’t possible to use a GC in a kernel (e.g. done in Microsoft’s research OS), and you would be brave to try. :slight_smile:

I assume as with “printer drivers”, that aren’t really, they are filters and live in userspace, what you have in mind is, e.g. a joystick driver, is more like a filter for USB, not in kernel space (while USB sub-system itself you rely on is there). [A lot of drivers that are in the kernel, e.g. GPU drivers, often also have a userspace counterpart.]

10

For example, it would probably already have been too late for Julia 1.0 to be included in Ubuntu 18.10. That’d be a problem for most people. It’s true that we’re entering a more stable era now, but patches tend to be more important for younger languages. I posit that the average Julia user cares more about the latest minor version of the release than the average Scala, Python or Ruby user, simply because it’s a (much) younger language.

Sorry, perhaps I was overzealous, driver may not be the right word. I’ve seen a couple of tutorials where you can write some simple “drivers” using udev and some other tools. Although, at the moment, I can’t find any good ones, what I saw was certainly more sophisticated than what little I can find right now on the Arch wiki…

11

For perspective from somebody trying to get Julia updated to 1.0 in a distro’s package repository (https://github.com/void-linux/void-packages/pull/3242), there are a lot of things frustrating about Julia’s requirements. The first is obviously the dependency on a specific, patched version of llvm, but then you’ve also got things like https://github.com/JuliaLinearAlgebra/Arpack.jl/issues/5 going on in the package ecosystem. I would definitely say that Julia as it is now isn’t nearly ready to be a first-class citizen in a linux distro’s package manager, but the way we’ll get there is by including it and ironing out bugs/bumps as they come along. I’m hopeful that this will only get better and better. Maybe sometime in the 1.x release series we’ll even be able to use the OS llvm. :wink:

4 Likes
12

That problem isn’t likely to ever go away though, is it? Don’t other LLVM languages have this problem?

13

Julia team has been pretty good about pushing patches to LLVM upstream (see https://github.com/JuliaLang/julia/pull/28809#issuecomment-423529753 for my general analysis of the state of the patches still needed for LLVM 7), so I’m hopeful that it’ll go away some day. And no, Julia is relatively unique as far as I can tell in requiring a patched version of LLVM: it’s fairly normal to require a specific major version of LLVM, but Julia is the only language I’m aware of that requires it’s own patches. Looking at void-packages since that’s what I’m familiar with:

  • Rust uses latest packaged llvm.
  • Crystal uses latest packaged llvm.
  • ldc (D compiler) uses latest packaged llvm.
  • ispc (some sort of SIMD compiler) uses latest packaged llvm.
  • Pony uses packaged llvm 3.9
  • Swift isn’t packaged.
  • Julia 0.6 uses packaged llvm 3.9, but had lots of problems, and probably shouldn’t have been done that way.
6 Likes
14

Yes, Fedora 29 (to be released at the end of the month) already includes Julia 1.0.1.

I tested it. However, it appears that many packages will notl compile. I tried to install HypothesisTests and StatPlots for example. Both fail to install/precompile.

15

Yes, packages using BinaryProvider which use BLAS need an ILP64 OpenBLAS. That’s currently waiting on this PR. In the meantime, the latest version in the Copr should work.

16

@Palli I startedusing Ubuntu in Denmak early this summer, and noticed that the sJulia shipped by the upstream Debian was 0.4. I joined the Denian Julai developer’s list.
The reason is that Julia uses a library which talks with web proxies, and this library had a license which was not compatible with Debian. I have had issues with that library int he past - getting Julia to work through some company web proxies can be impossible.
So let’s make it clear. Debian has strict rules on acceptable licenses. Which is what we as a community need.

However this issue is now fixed - I believe the proxy library updated their license.
Julia 1.0 is now available for Debian and hence Ubuntu.

1 Like
17

@tot_rispekt I have a dual boot into Fedora, but version 28. I will give those packages a try.
HAve you find this is specifically with Fedora 29? could it be an LLVM issue?

18

@tot_rispekt I just used vagrant to install the latest test build of Fedora 29 - in fact the Fedora Scientific spin, which is supposed to contain lots of scientific packages
Welcome to the Fedora Scientific 29 Guide — Fedora Scientific 29 Guide

When you add HypothesisTests is is Arpack.jl which fails to build.
You an fix this by using the master version if Arpack
add Arpack#master
But then when you test HypothesisTests - wow!
‘’‘’
ERROR: could not load library “/usr/lib64/julia/sys.so”
/usr/lib64/julia/sys.so: failed to map segment from shared object
ERROR: LoadError: IOError: write: broken pipe (EPIPE)
Stacktrace:
‘’’

19

there is a middle ground to repository installation and updating here: R has good ubuntu integration but it is not native. It requires the end user to add a repository first. thereafter, it lives in the apt ecosystem. this means that an ‘apt update’ pulls down the latest version.

the advantage is that the packages are then automatically updated when a new version is posted (without having to check the website), and that they will be installed always in the same location. but julia is relatively painless, because it has an easy install and is fairly intelligent about figuring out where it was installed. IMHO, nice but not essential.

20

Note that integrating into the OS’s package distribution mechanism is

  1. expensive to maintain (in terms of time & effort), as one need to be familiar with yet another set of tools (eg for Ubuntu: deb packaging, related CI, etc),
  2. inferior to Pkg(3), as the latter allows multiple parallel versions of packages to coexist peacefully on your hard disk, and the transition to contributing is just a dev away