Plans for Updating Julia

Tooling
, ,
1

I’m aware of four systems for Julia version management:

                            | Manual   | jill   | UpdateJulia | juliaup                   | juliavm
status                      | official | stable | stable      | stable / known bugs       | stable?
version multiplexing        | N/A      | no     | yes         | yes                       | yes
coexist with manual install | yes      | yes    | yes         | no                        | ?
operating systems           | all      | Linux  | all major   | Windows / all (with bugs) | Unix
language                    | N/A      | Shell  | Julia       | Rust & Shell              | Shell     
dependancies                | none     | none   | Julia       | none                      | none
ui                          | N/A      | cli    | Julia REPL  | cli                       | cli
sloc in src                 |          | 284    | 297         | 2306                      | 219            
github stars                |          | 229    | 34          | 123                       | 49

To quote @greatpet “I hope Julia will officially ship with one of these installers / updaters. Fragmentation in this area is no big deal but still unfortunate for newcomers.”

My thoughts (full disclosure, I am the author of UpdateJulia) are manual installation is tiresome and likely seriously slows the adoption of the latest version. We can only consider making Jill an official installer if it is cross-platform. UpdateJulia is silly because it only works if you already have Julia. Juliaup is an order of magnitude larger than it needs to be and will consequently be more difficult to maintain. On the other hand, jill and juliaup are available through standard distribution mechanisms users likely already have on their machines, and UpdateJulia already comes with the general registry.

Question: do we want an external tool available through standard software installation channels which manage Julia versions from the outside, or do we want to make Julia itself available through standard software installation channels and manage Julia versions from the inside (like Pkg)?

Question: have folks tried multiple/all four of these methods, and what parts of each do you like/dislike?

Question: what are some good best ways forward here?

8 Likes
2

Looks like you missed jill.py. This is a reimplementation and extension in Python of the shell script jill that you mention above.

3

asdf is multiplatform (linux/macos) and can also install/update Julia.

nixpkgs and AUR repos have the latest version as well, and both use official julia binaries.

So, on linux and macos there are lots of existing solutions. Users can install/update julia via the same means as other software. Why is there a need for a julia-specific installer at all? Imagine if all software came with one, and needed to be installed/updated differently…

1 Like
4

I’m on Ubuntu Linux and I’ve always downloaded new Julia versions and set up the PATH environment variable by hand. It’s easy enough that I haven’t been motivated to use any of the unofficial updaters, but on the other hand I’ve often been lazy and not upgraded to the latest minor version till a long delay. I’m happy to get software from the official Ubuntu repo or third party repo (PPA), or maybe Snap store. I don’t think I’ll install brew or Nix just for Julia. Again it’s no big deal but an officially blessed / bundled update mechanism would be nice. A bigger concern is that if I deploy applications to users who do not program in Julia, I can’t expect them to diligently keep Julia up to date.

1 Like
5

Julia is available in plenty of system package managers - brew on Mac and choco on Windows are both pretty good and up to date on Julia versions.

If you don’t have a package manager and don’t want to install Julia “manually” there is JuliaUp. If this were a bit more official and advertised on the Julia download page (along with package manager options) that might make it easier for newcomers.

6

Aside, I don’t much like this trend of “taking X seriously” posts. They feel a bit snarky against the people who already do take such things seriously.

11 Likes
7

The plan is that once juliaup works well enough on all platforms, it will become the official means of installing Julia. On Windows we’ll recommend getting juliaup through the Windows store and on other platform people can use curl -sSfL https://install.julialang.org | sh to install juliaup. They can then use juliaup to install and manage various Julia versions and to update juliaup itself (except on Windows where the store update mechanism will update it). We will replace the manual Julia installation instructions on the downloads page with instructions to get juliaup on various platforms and how to use juliaup to install Julia.

We can only consider making Jill an official installer if it is cross-platform.

Correct, so that’s not an option. It also depends on Python which isn’t available everywhere, is a very heavy dependency and isn’t especially portable. It’s serves as an excellent proof of concept but can’t realistically be the official installer.

UpdateJulia is silly because it only works if you already have Julia.

Yes, this seems like a critical bootstrapping problem.

Juliaup is an order of magnitude larger than it needs to be and will consequently be more difficult to maintain.

Not sure what this means. Why is it too large? Maintaining and developing it seems to be happening just fine.

17 Likes
8

Windows Store does not work well (or barely at all) on my work laptop, but, fortunately, you can update (and perhaps install) with winget.

4 Likes
9

@DNF, regarding the use of winget when Microsoft Store Apps are blocked by work PC’s admins, I have just tried it and got:

PS C:\WINDOWS\system32> winget install julia
Multiple packages found matching input criteria. Please refine the input.
Name  Id              Source
-----------------------------
Julia 9NJNWW8PVKMN    msstore
Julia Julialang.Julia winget

Would you or someone know why there are two Julia packages listed?

NB:
Fyi, I have tried to install the one from the msstore source but bumped into an issue:

Verifying/Requesting package acquisition failed: no store account found
10

Well, I don’t think mine is actually blocked, since it works intermittently. I’ve not noticed having two Julias. I installed first time (somehow) from the store, and update from winget.

11

Julia in msstore is juliaup.

1 Like
12

winget currently shows software from two “catalogues”: a more or less free-for-all community repository, and every app from the Windows Store. The community repository is older, and we added the current non-juliaup installer to that back when it came out. At some later point they started to surface all the apps from the Windows Store, and now both the current default Julia installer and Juliaup show up there. My understanding is that the next version of winget will automatically prioritize a Windows Store version, if it exists. We could presumably also remove the non-juliaup version at some point.

If there is any more detail you can provide how that manifests itself, please let us know as much as possible over at GitHub - JuliaLang/juliaup: Julia installer and version multiplexer!

Fyi, I have tried to install the one from the msstore source but bumped into an issue:

Verifying/Requesting package acquisition failed: no store account found

I think they are still working out how the MS account requirements from the store translate into the winget story…

2 Likes
13

It might be a good idea to specify for various platforms where Julia installations are installed and how they are named, and maybe how and where meta data is stored. This way people could manage or at least choose Julia installations with other tools. Then rustup would be designed to conform to this specification. Currently, it looks like each of the installation tools installs Julia in its own, idiosyncratic location.

juliaup does look pretty complicated compared to, say jill.py. And using rust is buying into some complexity and barrier to contributions. It’s worth it if high performance is a requirement, but that’s not the case here. Is the main reason juliaup is written in rust because rustup is? That said, a better choice for a strictly universal installer isn’t obvious to me. Maybe go. I can understand that rust has proven to be better than Julia itself and c++.

I wonder how often Julia is installed in a python-less system. Python has big advantages over rust. It is always installed on linux and macos. It can easily be installed on windows. But, admittedly, that’s (perhaps) not as easy as just going to the windows store. I looked briefly into building a stand-alone Python app for jill with pyinstaller. The resulting executable is 155MB. And the build feels fragile. Startup is a bit slow unless you build an app in a folder. It’s probably difficult at best to make anything close to as lean and fast as juliaup. I don’t see a clear path to a universal installer written in Python.

Still people install Julia for different reasons. I am interested in installing it to support Python projects that depend on Julia, in which case only jill.py makes sense. Partly because juliaup is not ready yet. But juliaup would have to be very convenient and easy to integrate with Python to convince me to use it. Maybe that will happen. There may be other reasons that make Python, or another tool, attractive.

1 Like
14

Juliaup will install all Julia installations in $JULIA_DEPOT_PATH/juliaup, which has the nice propery that all data related to Julia is inside one folder, alongside e.g. artifacts or compile caches.

Juliaup does have a bit of a history. I believe it was originally written in Julia itself and shipped using PackageCompiler.jl. This made the resulting binary quite large though and difficult to handle. It was then reimplemented in C++, but that has of course all the usual drawbacks when it comes to memory safety, portability and lack of convenient package management for external dependencies. At that point Juliaup still only supported Windows. David eventually decided to rewrite it in rust, which had the nice side effect that it made support for Mac and Linux much more feasible. (I think he mentioned it might have also been partly out of curiosity for the language.)

As an occasional contributor, I must say that rust has really nice tooling for these kind of CLI applications, especially the exception handling works quite well for this. The code is pretty well structured, so I don’t think the implementation is as intimidating as it’s made out to be in this thread. Go probably would have been an option as well, but I don’t think it has major advantages over rust for CLI application and I have heard it described as generally being more verbose than rust.

14 Likes
15

Juliaup consists of two binaries: juliaup and julialauncher.

julialauncher gets exposed to end-users as julia via a symlink on Linux/Mac and an execution alias on Windows. When a user installs Juliaup, and then types in julia, that will actually start julialauncher. julialauncher handles a few things: it handles the +channel command line argument, reads the Juliaup configuration file, when in standalone mode occasionally checks for new versions of Juliaup itself, and then launches the correct actual Julia binary. As it essentially injects itself into almost every Julia launch, it needs to be really fast. And small and self-contained, so that it can actually be fast. That means it can essentially be written in C, C++ or Rust (or maybe Go? Not familiar enough with it) in terms of mainstream languages. The original version was in C++, but that was painful because even very simple things seem often very complicated in C++, plus it was just a major pain to get simple functionality like downloading files, extracting archives etc. working. I started looking at Rust at the same time, and it really is fantastic for this kind of software.

juliaup itself was originally written in Julia, as @simeonschaub rightly pointed out, and then compiled with PackageCompiler.jl. But that was never really a smooth experience, for many reasons. The resulting binary distribution was gigantic (couple hundred MB), which really is not ideal for a simple small command line tool, startup latency was not fantastic (not horrible, but not great), compile times were really long and it was all just really rough. So then at some point, when I had already ported julialauncher to Rust, I just ported the rest over as well. And I’m very, very happy with that choice :slight_smile: Rust is fantastic for this kind of software, the tooling is excellent etc. All the things that make Rust very different from a dynamic language like Julia, e.g. static typing, very different error handling model etc, make it a complete non-starter for scientific code, but for something like Juliaup it is a really good match (much better than a dynamic language like Julia, IMO).

Just to be clear, Rust compiles standalone binaries, end-users don’t need Rust or anything related to Rust installed on their system to use Juliaup.

I think in general a default Julia installer should “just work”, without users having to install anything else first. Plus, Python setup on Windows is kind of mess, IMO… I really wouldn’t want the default Julia setup story to get entangled in that.

29 Likes
16

Apple will no longer bundle python (2 or 3) with MacOS: macOS Monterey 12.3 will remove Python 2.7 (/usr/bin/python) – macmule

5 Likes
17

No.

Having Python as an dependency for installing Julia would result in “ok, forget about Julia” for me. My main hope in using Julia is: no R and no Python anymore. Even as a dependency which I don’t have to touch, just install, I would hate it. I hate both, R+Python, already … as a recent story: just this weekend I had to install a new R package (bioconductor). I knew at the beginning it will be pain. Hours of hours to install some dependency which popped up during some steps. At the end, no success, because pandoc had to be a newer version than debian jessie provides for some font of type woff2. Nothing I could overcome but it was just enough for me. You know? It’s only about building the vignette(docs) of some package which failed after HOURS OF DEPENDENCIES… It ruined my weekend and now I refuse to solve the next step just because the weekend is already ruined, no, I decided to enjoy my last hours of this weekend.

For all who want to help now: no need, I can overcome any R problem for myself after decades of fighting R/bioconductor/CRAN.

The point is: R and Python are good for academics where time (at the beginning of the PHD) doesn’t matter and problems don’t emerge because you start vanilla.
But whenever you have to think about time and future maintainability you would NEVER choose R or Python. NEVER! Period.

:wink:

Choosing rust for juliaup is not a problem at all. It’s providing a small executable for all major platforms. That’s a good reason and a good choice. Contributing by learning rust is a no-brainer.

I would be in heaven of software development, if Julia itself could provide this feature of small executables (as I am not getting tired to say that).

4 Likes
18

Will a Windows installer still be available? I’ve never had a problem with using the installer and don’t really like the Windows Store.

3 Likes
19

@Lilith You might want to correct your table: juliaup doesn’t have Rust as a dependency (for the user) and it can also coexist with manual installs (that’s how I’m using it right now).

7 Likes
20

I want this too. I like juliaup, but more often I need the standalone installer. juliaup is for me the tool for installing Julia as a windows integrated (no PATH e.g.) Julia. But whenever I want to experiment with something I install a second Julia from the windows installer (without any integration).
This is my Windows Start menu:

image
image311×547 31.5 KB

Only 1.7.1 is per juliaup :wink:

Well, it needs some cleaning I guess.

3 Likes