ANN: julia-build, jlenv, jlenv-plugins and jlenv-cookbook

Ann: jlenv & plugins - Robust Julia version management

This is a 1.0.0 release of the Julia port of the rbnev/pyenv tools and plugins.
As well as some projects supporting Julia configuration and version management.

NOTE:
You can install different versions of julia using the julia-build script.
As you will see this script will be deprecated with the next major release of jlenv. It will be replaced with a tool that verifies GPG signatures and can run stand alone or as a jlenv plugin.

More information about jlenv is here

Where a tool has been forked, it has generally been from the ruby code base.
Each tool is under develpment, with the immediate aim being interoperability.
I have added (bats) preliminary tests for the plugins I selected.

The biggest area of ‘wet-paint’ is around the proper/best practice use of
Julia environment variable settings - runtime and build time.

Released

The components considered released are ‘pinned’ on the jlenv organization GitHub page.
These projects have test suites that pass.

  1. jlenv
  2. jlenv-vars
  3. jlenv-each
  4. jlenv-update
  5. julia-build

Forthcoming/Underway

jlenv-cookbook:
The Chef Git Resource breaks when used in no-TTY environments, this breaks
dokken test environment. A workaround is in place and being tested.
Likely released before the next major version of jlenv.

julia-install:
A standalone script to install Julia versions. This port is underway.
and will be released when the test suite passes and jlenv is
compatible with the julia-install conventions/assumptions.
Likely released with the next major version of jlenv.

jlenv-install:
A plugin that adds julia-install functioanlity to jlenv. This needs jlenv and
julia-install to work off the same conventions/assumptions.
Likely released with the next major version of jlenv.

jlenv-installer:
This is a install convenience script.
Likely released with the next major version of jlenv.

chjulia:
A simpler Julia switching tool - the port of chruby. Eventaully this will be added as jlenv plugin,
jlenv-chjl. Currently the jlenv and chjulia scripts have different
conventions/assumptions about where Julias are installed - these need to be
reconciled.
Likely released with the next major version of jlenv.

6 Likes

Any indication when the next version will be released? Right now jlenv-installer does not work (probably because it doesn’t exist yet, the file is still called rbenv-installer in the repository). As an avid user of pyenv for Python I’m really looking forward to using a tool for Julia that works in the same way.

Hmm, jlenv-installer is kinda ready - caveat emptor :wink: . Issue reports and PR’s welcome.

I think you meant jlenv-install plugin and that is not yet released. Sorry about that.

If you don’t mind the bleeding edge try the jl branch in my repository of julia-install - this is what the jlenv-install plugin will invoke under the covers.

Alternatively, try the julia-build - linked to in the OP, that should work and won’t be changing except for bug reports and version releases.
Be forewarned that julia-build will be deprecated as soon as julia-install is ready (verifies source signatures), at which point the jlenv-install plugin will likely be ready too.

Feedback and issues reports welcome - PR’s too.

There are a few things that need to be in place before turning the jlenv-install plugin loose:

  • Internalising the correct way to manage Julia’s packages across multiple version installs, and multiple users.
  • Are we going to support installing to system wide (-1), or rather just allowing point to a system install if asked to (+1). The issue of package management setup in the case of multiple users requires some thought.
  • Sync jlenv and chjulia and julia-install in terms of folder locations and package management
  • Ensure that jlenv, pyenv and rbenv don’t stand on each others toes

Hope that helps.

@taqtiqa-mark

I am curious how jlenv-install will keep a list of available julia versions. I think it would be beneficial to hook into the github api to get a list of the releases.

You probably are right. I prefer loosely coupled systems, and eventually I’d like that julia-install could be pointed at a configurable endpoint:

  1. Allows the community to have a fallback/DR source
  2. Allow an organization to control what versions are available to be installed.
  3. HTTP & Git protocols are the only requirements

So tying to GitHub’s API works against those objectives - at least makes them harder and constrains choices.

Hope that makes sense?

1 Like

Eh no, I do mean jlenv-installer. Check the repository yourself: https://github.com/jlenv/jlenv-installer/tree/master/bin. No jlenv-installer file present, only an rbenv-installer. So the instructions given:

# with curl
curl -fsSL https://github.com/jlenv/jlenv-installer/raw/master/bin/jlenv-installer | bash

# alternatively, with wget
wget -q https://github.com/jlenv/jlenv-installer/raw/master/bin/jlenv-installer -O- | bash

Simply can’t work: there’s no jlenv-installer in that location (I know, I tried). And given the file has not been renamed, I have no guarantee that it’s been modified enough to be used, so I don’t dare change the instructions to run rbenv-installer instead.

So I repeat my question: when can we expect jlenv-installer to be ready?

Apologies. Try now. Please note:

  1. As with all these curl | bash type scripts - read it before you run it.
  2. Also there are as yet no tests in this repo - PR’s welcome.

Hope that helps?

1 Like

Perhaps an interesting balance would be writing a simple github bot that will make a pull request for every julia release. This way the versions are still loosely-coupled, but there is a level of automation that hopefully minimizes errors from human editing and minimizes the propagation time between a julia release and its availability in jlenv

TLDR; There is a script in jelnv/julia-versions/update.sh that you can use in some automagic chain with wei’s pull. This is out of jlenv’s scope for the reasons below.

You’ve correctly identified the jlenv is opinionated. The opinions are that only bash(other shells should work but YMMV - see test suite), HTTP and Git are required.

To be clear “the opinions of jlenv” right now is just me - until other contributors step forward. At that point things may change or fork as Linus intended.

From the perspective of what jlenv aims to achieve - minimally constrained users - that is a seismic imbalance: You’ve just tilted (imbalanced) everything toward github. jlenv aims to let you make your source of truth about Julia versions be any (gitlab/bitbucket) git repositry.

Again, not being tied to github, or git lab, etc etc and not being automated are features not a bugs.
In the TLDR I’ve pointed out how anyone who wants that sort of setup can achieve it. Jlenv wouldn’t add anything worthwhile.

You are correct there is human dependent delay. Again that is a feature not a bug - think of the case where the human delay is \infty - equivalent to choosing to not make that version available.

Having said that, the intention of jlenv/julia-versions is to track Julia’s releases - your fork of jlenv/julia-versions isn’t bound by that, and you can still use jlenv confident your choices about what <your>/julia-versions contains will not adversely affect jlenv’s function. If jlenv/julia-versions isn’t updating fast enough for you please make a PR for that version.

Right now julia-install doesn’t allow you to point to another source of julia versions that it knows about - but it will in the fullness of time :wink:

Thanks! I’ll try it out.

I’m a data scientist, not a developer, so not sure I can help with PR’s, but if I see any issues I’ll report them.

Just a quick question: you mention jlenv, rbenv and pyenv possibly standing on each other’s toes. I’m using pyenv already. Are there any specific things I should be on the lookout for?

Cheers, and thanks for your hard work!

As a general note, all these *env so far allow functions to be overridden - you can see this in the test suite where tests redefine some functions. IMO this is a no-no, and jlenv will eventually disallow this sort of ‘hijacking’.

The only concrete issue I have been able to find where this might be the culprit is between rbenv and pyenv was not reproducable.
So strictly nothing yet that concerns jlenv, but I wouldn’t be surprised if something like that prior experience popped up.

That would be appreciated.

Frak!

$ curl -fsSL https://raw.githubusercontent.com/jlenv/jlenv-installer/master/libexec/jlenv-installer | bash
Installing jlenv with Homebrew not yet ported. Contributors needed…
bash: line 84: : command not found

I should have expected it when I read that the script would first detect whether Homebrew was installed. I happen to use Linux Homebrew (one reason being that it’s the easiest way to install pyenv on Linux!). Is there any way bypass the test? And if not, what would you require to make jlenv available via Homebrew apart from time?

Disclosure:
I know nothing about the MacIntosh world - apart from it being insanly expensive.

By passing the test wouldn’t fix anything would it?

Happy to bypass the test if you can explain/describe/document how another user would use homebrew with the test circumvented.

I don’t foresee doing whatever it is Homebrew needs to install and manage jlenv.

Am very happy to accept PR’s and answer questions.

Me neither. Like I said, I’m running Linux. I just happen to have the Linux version of Homebrew installed (Homebrew on Linux — Homebrew Documentation).

Wouldn’t it? Wouldn’t removing the test allow jlenv to be installed normally despite Homebrew being installed on my machine?

If you don’t, why have the test in the first place? People can’t just use Homebrew to install something that is not available as a Homebrew recipe in the first place. Since it seems you’re not planning to have jlenv available as a Homebrew recipe, why are you even testing for that option?

If other people want jlenv available as a Homebrew recipe, but you don’t want to handle that yourself, let them package it themselves, and install it themselves using brew install jlenv. But you shouldn’t have to worry about it in your own install script.

This has been bugging me - I’ve decided to remove all ‘support’ for any package managers. I think that will resolve your issue.

Note sure when I’ll have time to get to this. PR’s welcome.