Recommand `curl | bash` as default installation is not ideal?

Right now if you visit Download Julia you see something like this:

I think it’s reasonable for people to frown upon this, for one, it’s not a reproducible way to install Julia, it depends on what you get at the moment you run this command, and it’s not tracked by distribution package manager. People may also feel unsafe (you can argue this is not well justified) because you can detect curl | bash and many articles written against this pattern.

It is very unfortunate that this is a common pattern now, for example, looking at Rust:

on the plus side they have --proto '=https' --tlsv1.2 which may be just a bit more guardrail in place, but at the same time that shell script lives on a different domain (rustup.rs) so if someone sees this line in another context, it may be confusing.

Is there anything to be done?

Looking at packaging for rustup across landscape, it’s at least distributed on:

So if they want, they can probably add a “use your linux distribution pkg manager to get rustup” before the “curl | bash”. We (Julia) currently can’t, because we’re not well packaged on linux distributions. But I’m curious to what people think, once juliaup is more widely distributed, should we recommand it over curl | bash, or at least in parallel?

4 Likes

My long-term hope is that we can get juliaup packaged as julia on as many platform package managers as possible, so that the curl option is essentially the fallback for systems where we aren’t in the default system way of installing stuff.

On Windows we are there already: Julia in the Windows Store is just Juliaup.

On Mac/Linux we are in a couple of package managers (brew etc) but at the moment as juliaup, whereas I would want that to eventually become julia.

The most frustrating situation is on Mac: there doesn’t even seem to be a system solution we might be able to hop on eventually… My sense is that we are stuck with curl there for good.

10 Likes

IIUC, for Mac the best solutions are curl or “download the Mac app” for regular use, and brew (or maybe port) and conda for users who have already bought into one of those ecosystems.

Conda also gives us pixi et al for free. But I’m not sure whether juliaup is a reasonable thing to put in conda, given the potential for binary incompatibilities (e.g Julia requires specific OpenSSL versions).

1 Like

Not true (yet at least).

I think we could easily put Juliaup into conda. Juliaup can just be handled by itself, none of the choices there have an impact on what binaries Julia itself will/has to use. We already have this kind of model: Juliaup is in brew, it is on the Rust crates.io thingy, and in each case it will be compiled with whatever that platform “uses”, and then Juliaup itself downloads the “official” binaries for Julia itself that bring everything needed along.

Having said that, my experience with conda over the years has been so bad that I would almost prefer if Juliaup was not in there :wink: Maybe it is just me, but I’ve never had a conda install that wasn’t completely broken after a while…

3 Likes

https://anaconda.org/conda-forge/juliaup

2 Likes

I agree that curl | bash feels bad, but there are several meaningful reasons why I don’t find it to be actually abhorrent:

  • You’ve already got to put a lot of trust into *.julialang.org and your https connection to it. If you don’t trust that, then you shouldn’t be running the executables from there either — a shell script somehow feels sketchier but is fundamentally no different than manually downloading and running Julia itself.
  • Julia doesn’t need admin permissions, and it’s not run with sudo
  • The thing it installs is very self-contained, as are the julias it manages

If adding those extra https flags to curl provide a meaningful benefit, we should copy that improvement. I bet that was added after we “forked” those idioms from rustup.

The biggest thing I don’t like about curl | bash is that it normalizes running code from potentially arbitrary websites, and while that’s no different from downloading and running an executable, I think there are some folks who have the “don’t run untrusted .exes” more firmly lodged in their heads than “don’t copy-paste-run this code.” In fact, pasting code into the command prompt is a common scammer technique for malware injection — probably for this very reason.

It’s also true that the distro packaging landscape is very different these days. They’re much more amenable to *up-style packages and less demanding about the sorts of bug-prone dynamic linking that plagued packaged Julias of the bygone era.

All that to say: yes. It’s be great to distribute juliaup through official platform channels if someone can undertake that work and make it happen in a robust manner. In the meantime, this works.

11 Likes

Ah, maybe it’s purely a package problem then. But if you load the JLL OpenSSL before the Python one in a session with PythonCall, and the two versions are not the same, Python will not be able to successfully call into OpenSSL. (There’s a fix for this in CondaPkg.jl for this exact reason).

I guess the problem is getting Pkg info to Conda, from the Conda end…(since the Julia/Pkg end is solved via CondaPkg!)

A bigger problem is coordinating JLL versions between different packages, since Julia JLLs often don’t share the upstream project’s exact version number (take Proj_jll as an example, which multiplies all version numbers by 100).

Pixi Global: Declarative Tool Installation | prefix.dev seems not bad, we already works with pixi via conda:

> pixi global install juliaup
✔ Installed package juliaup 1.17.10 h8fae777_0 from conda-forge
  These executables have been added to /home/akako/.pixi/bin
   -  julia
   -  juliaup

! To use them, make sure to add /home/akako/.pixi/bin to your PATH
> /home/akako/.pixi/bin/julia
2 Likes

But doesn’t this shift the problem only to the question how to install pixi?

Official suggested method:

curl -fsSL https://pixi.sh/install.sh | bash
4 Likes

Yeah but if you’re on Mac, you don’t have a package manager

Most Mac users I am aware of, do have a package manager, brew, though of course that is also something you have to install and they also have a curl/bash install command line instruction as their default install.

3 Likes

:wave: Somewhat butting in here, but @jling shared this thread with me.

As has already been noted, there isn’t a cross-platform (Linux, macOS, and Windows) base package management system and so at some point you’re going to need to bootstrap something to get going. I think that doing that earlier/lower level and only once is more tolerable, and pixi’s install script allows for you to specify the version of pixi you want as well, so it is even reproducible through time. (c.f. the pixi installer script options docs — as I’m new I can’t include a link it seems, but they’re right on the landing page).

While I’m not going to say that conda-forge is the panacea to all the world’s packaging problems (e.g. Julia packaging is amazing!), it is pretty good that through just conda-forge and pixi you’ve now gotten a reliable and painless installer for every OS that you support on conda-forge that you can also trivially uninstall.

3 Likes

So, my general view on these non-system package managers like pixi/conda/brew etc is that I’m all in favor of putting Juliaup/Julia into those, ideally under the name julia.

But at the same time I don’t see a scenario where any of these would become the “default” recommended way to install Julia. I think one really important thing for Julia is that we have a one-step setup for Julia on each platform. In my mind that pretty much rules out any of these alternatives as our default, where you first have to install the third-party package manager, and then you can use that to install Julia.

Now, I could see a scenario where one of them is so widely used and clearly the only game in town that everyone uses, that one might jump onto it as a default. But at least at the moment, I don’t see any of them reaching that kind of market share that would justify that.

12 Likes

honestly I find that confusing. Even on windows, when installing “julia” I doubt if I’m installing the correct thing (which should be juliaup). What’s the advantage of this name clash?

4 Likes

The julia package installs Julia on your computer. That means julia now is on your PATH, and it also means you get another command line utility that you can use to manage Julia versions. I think the right way to think about Juliaup is that it is a tool that comes by default with a Julia installation, in the same way that a package manager comes along etc.

5 Likes

Can something like that be written in the description of the package in the Windows store? (and other similar places)

Well, there is the App Store. Not sure if that can install command-line tools though?

3 Likes

My sense was that it doesn’t support command-line tools. But I also didn’t find really great documentation on that, so I might be wrong… But if it did support command-line tools, then we should definitely try to go that route.

2 Likes

I’m not sure if that helps, but Xcode is distributed via App Store, I think, and XCode then used to install Command Line Tools (which is then the pre-requisite to install brew, used to install juliaup, used to install Julia, used to…).

1 Like