[ANN] Juliaup Preview - Julia version manager and Windows Store installer

Klaas_Pauly · August 7, 2021, 9:30am

Would there be a way of installing and using juliaup on Windows Server (2019), which doesn’t support the Windows Store?

davidanthoff · August 13, 2021, 1:52am

Not at the moment, but I think we have a roadmap now to enable that. I rewrote juliaup in Rust last week, and now we can compile and run it pretty much on any platform. There are still a lot issues that we need to resolve before this is ready, but the plan now is very much to make this available on all platforms, not just Windows and the Windows Store.

BLI · August 13, 2021, 6:20am

I tried to use the Windows Store installer on my home computer. This is what I did:

Uninstalled Julia, Python, etc. via the Control Panel. I kept VScode on my computer.
Used FileExplorer to remove all I could find related to Julia, Python, etc. Also in %appdata%
Downloaded the Julia app from Windows Store. Worked smoothly.

I then opened VScode and added the Jupyter notebook extension from Microsoft.

OK – now, VSCode uses the latest version of Julia in the terminal window (v. 1.6.2). However, when I try to run Jupyter notebook, VSCode tries to find the old version of Julia I had on my computer (1.6.0, I think), and gives an error message when it doesn’t find that version.

So – apparently, there is something referring to Jupyter version that I didn’t delete… Any ideas of where that file is found?

davidanthoff · August 13, 2021, 6:59am

Yes, two reasons (most likely)

You still have an IJulia.jl kernel registered. They don’t go away when you delete or uninstall Julia, you would have to manually delete them.
The VS Code Jupyter extension is surfacing these IJulia.jl kernels (which in your case are still registered but don’t actually exist anymore). To use the new native notebook feature in VS Code you need to enable that in the settings for the Julia extension, and that should then show a new kernel that is the Julia version you installed from the Store. That kernel is “registration” free.

BLI · August 13, 2021, 1:33pm

Found it… it was under the Julia section of the Julia extension… I first looked under the Jupyter section of the Julia extension.

davidanthoff · August 13, 2021, 9:11pm

I just released version 1.1 to the Windows Store.

This is both a big and small update at the same time. In terms of user facing changes, it is very minimal:

There is a progress bar for downloads.
There are new -h and --help flags.
Generally the command line interface is a bit richer.

Under the hood, this is a complete rewrite Everything is written in Rust now. There are some major benefits to this new approach:

Everything is much smaller. The previous juliaup was a couple hundered MB, now it is about 2.5 MB. When you install from the store, the total size now is what you would expect, a little more than 100 MB. Updates to juliaup that don’t include updates to the bundled Julia version are really small and fast now.
juliaup has excellent startup performance
The code is much more robust. Rust really forces you to handle errors and catches many via its static type system.
This all paves the way to a completely cross-platform version of juliaup.
No more C++! The launcher was previously written in C++ (because it needs to be really as fast as possible), but now is also written in Rust, which makes it way, way simpler.

At the same time, a rewrite always is a risk, and I’m sure I outsmarted Rust’s type checker somewhere and managed to introduce some new bugs. So, please be on the lookout for those and report them either here on even better over at https://github.com/julialang/juliaup. I’ll be monitoring both places and will try to fix anything that comes up quickly.

How do you get the new version? Well, just wait, it is auto-update You could also check in the Windows Store for updates, but it will probably take a little longer before the files have made it through the store submission pipeline.

essenciary · August 16, 2021, 1:04pm

Thanks, loving this!

Just got it updated through the Store but there seems to be an issue with the Julia versioning - looks like it’s recommending a downgrade?

Not sure what to do.

davidanthoff · August 16, 2021, 5:23pm

Ah, I forgot to remove something there from an older version. Two options: 1) I’m going to push out an update that fixes this very soon, 2) you can delete a file called juliaup-versionsdb-winnt-x64.json in your ~/.julia/juliaup folder and then it should also be fixed.

Roger-luo · September 15, 2021, 8:10pm

I also hoped to have something like rustup over jill.py, avoiding an extra step installing pip3 would be VERY convenient especially when I use AWS instances pip3 isn’t always available in the AMI.

I was hoping to do this in pure Julia tho so I made a very experimental thing called Ion: https://github.com/Roger-luo/IonCLI.jl and I haven’t got much time to polish it recently. It’s a binary built in pure Julia and thus has a much larger size. And my hope is to get smaller binary when the Julia runtime separated from codegen and llvm. I was hoping it to be more a CLI for more complicated julia workflows like some of the features cargo provides.

I’m very excited to try out juliaup it seems much stable than Ion and smaller too. Thanks for making this!

oheil · February 11, 2022, 11:53am

At work the Microsoft store is blocked. (therefor winget -s msstore refuses too)
So I downloaded the Windows Release Zip file
https://github.com/JuliaLang/juliaup/releases/download/v1.5.36/
from github and unpacked it.
I installed Julia with

juliaup add release

As the integration is done via Windows Installer I am now missing the entries in
...\AppData\Local\Microsoft\WindowsApps\...

Would it be a nice feature if juliaup could do this, like calling

juliaup integrate

which would create the links in the WindowsApps-folder and what else has to be done?

davidanthoff · February 11, 2022, 4:47pm

Yeah, we clearly need some solution when the Windows Store is blocked… My best guess right now is that we should try to make the shell/script based installer that we have on Linux and Mac also work on Windows for those situations.

I do think that in those scenarios we’ll have to modify PATH manually, the entire execution alias thing really only works if a piece of software is installed via an MSIX… The only other option I wanted to recheck is whether standalone MSIX installs now work without the sandbox, in which case we could also try that.

johnh · February 28, 2022, 7:44am

Perhaps not the best thread to ask these questions:

juliaup has channels - there seems to be no method to display what channels are available

for beta when will this switch to using the 1.8 series - I recognize there will be a lag

carstenbauer · February 28, 2022, 2:10pm

Seems like @davidanthoff has made the necessary in the juliaup repo and started the new cargo release process (IIUC): https://github.com/JuliaLang/juliaup/commits/master

I’m not familiar with cargo releases. What’s the typical delay? It’s 8 hours already since the relevant commit.

johnh · February 28, 2022, 2:22pm

Thankyou @carstenbauer I really was not hassling. I am just curious about the process - if we expect folks to use juliaup we as a community should check that updates are arriving.

tbeason · February 28, 2022, 2:28pm

I would also be curious to know the process. I saw that 1.8.0-beta1 was actually tagged several days ago. I would have thought that perhaps juliaup was somewhat automated and picked up the the new release right away, but I guess the process is much more manual.

carstenbauer · February 28, 2022, 2:42pm

Note that even with a manual juliaup self update I don’t get 1.5.37 (which should give me Julia 1.8.0-beta1 on the beta channel.)

simeonschaub · February 28, 2022, 3:32pm

The promotion to the release channel still needs to be manually approved.

davidanthoff · March 1, 2022, 8:10pm

The process is probably a bit too convoluted at the moment. I monitor the release channel on Slack and try to understand when things are ready in terms of official builds being available for downloads, but there are always many steps discussed there, and it is not entirely clear to me when I can start to move things on the juliaup side. But when I have the sense that things are ready, I bump a build for juliaup. It takes about 10-15 minutes for the build to finish, then I deploy it to the dev channel of juliaup itself. For that build to make it through the Windows Store infrastructure it typically takes a few hours. Once that has happened, I test whether everything works, and then I trigger the release of the build to everyone. At that point the build has to go through the Windows Store process again, and that typically also takes a few hours, although with the latest build it was more like a day.

kristoffer.carlsson · March 13, 2022, 8:34pm

Ideally there would be one source of truth for the statuses of the releases. I recall that you did not want to use the file (versions.json) that all the rest of the ecosystem use for this. I think this was due to file size? Surely, that could be worked around by e.g publishing a hash if the file and only redownload it on changes.

davidanthoff · March 13, 2022, 9:29pm

I definitely want things to eventually just depend on versions.json and get rid of all manual steps. I think we want to get there in two phases:

Some changes to versions.json will always require release of a new complete version of juliaup, i.e. pushing of a new version to the store, the various package managers etc. In particular, whenever the release version of Julia changes, that is the case, because juliaup internally ships/bundles that particular build. I think to automate this more, we probably need some way that an update to versions.json triggers a GitHub workflow in the juliaup repo, and that workflow then prepares a PR for a new version of juliaup or something like that. I could see a scenario where things are fully automated with no user input for this, but maybe that is further down the road, for now it would just be great if there was something automatically kicked off.

A more refined second solution would allow the juliaup client to download new version info without updating juliaup itself. This would be useful whenever a new Julia build is released that is not the main released version at the moment, i.e. new LTS versions or prerelease builds. It would also be especially useful down the road if we succeed in putting juliaup into say the default Ubuntu package manger that apt interacts with, where we won’t be able to update juliaup regularly. This entire scenario is much more complicated, though. We need to deal with things like download size, the size of the version DB on disc (to make sure we don’t spend too much time checking things from a large JSON file whenever julia launches), the stability of the schema of things, handling race conditions on the update side of the version DB file, the interaction of updates to the versions.json file with updates to juliaup itself etc. An earlier version of juliaup actually had all of this implemented, but it was too brittle, so I removed it until we can implement a more robust solution.

I think for now just having the first option would already be great. Having the second option would at the moment be a nice to have things, but wouldn’t actually make the end-user experience that different, so I’m tempted to wait a bit with that. I do think we really need the second part if we get serious about getting juliaup into slower moving package managers, though…