Install IJulia for JupyterHub (multiple-user Jupyter)

General Usage
1

Hello, I am trying to install a Julia (0.6) kernel for JupyterHub, the multi-user version of Jupyter.
I already have JupyterHub working with Python3 and R kernels, I just need to add Julia.

I did try (from Julia run as root) Pkg.add("IJulia"), but that install the julia kernel only for the root user.
I then tried to create a symlink (ln -s /root/.local/share/jupyter/kernels/julia-0.6/ /usr/local/share/jupyter/kernels/julia-0.6) but that make JupyterHub crash.
There isn’t an option for multi-user installation like for R kernel IRkernel::installspec(user = FALSE) ?

Antonello

1 Like
2

I did try the following:

mkdir /usr/local/share/jupyter/kernels/julia-0.61
cp /root/.local/share/jupyter/kernels/julia-0.6/logo-32x32.png /usr/local/share/jupyter/kernels/julia-0.61
cp /root/.local/share/jupyter/kernels/julia-0.6/logo-64x64.png /usr/local/share/jupyter/kernels/julia-0.61
cp /root/.julia/v0.6/IJulia/src/kernel.jl /usr/local/share/jupyter/kernels/julia-0.61/
nano /usr/local/share/jupyter/kernels/julia-0.61/kernel.json (just changed the path of kernel.jl)
{
“display_name”: “Julia 0.6.1”,
“argv”: [
“/usr/lib/julia-0d7248e2ff/bin/julia”,
“-i”,
“–startup-file=yes”,
“–color=yes”,
“/usr/local/share/jupyter/kernels/julia-0.61/kernel.jl”,
“{connection_file}”
],
“language”: “julia”
}
systemctl start jupyterhub

Now I can see in jupyterhub the Julia kernel, but it still crashes

3

We had similar problems when setting up IJulia for Jupyterhub. The main problem is that you can not yet create a system-wide Julia package directory and the IJulia kernel assumes that the IJulia package is installed and usable. However, you probably can not assume that users have already run Pkg.init() and Pkg.add(IJulia) when they login to jupyterhub. We saw two possibilities:

  1. Use a global package directory, so instead of calling in your kernel file /usr/lib/julia-0d7248e2ff/bin/julia directly rather invoke a shell script that contains something like export JULIA_PKGDIR=/root/.julia;/usr/lib/julia-0d7248e2ff/bin/julia $* The disadvantage here would be that users can not install their own packages
  2. Invoke a shell script like in (1), but instead of setting JULIA_PKGDIR it checks if IJulia is installed in the user package directory and if not it creates a ${USER}/.julia/v0.6 directory if it does not exist and then copy the contents of /root/.julia/v0.6 to the user package directory.

We went for option (2) although it still feels quite uncomfortable. Note that the problem will probably be solved once Pkg3 is there, because it will allow using different package directories at the same time (a few system-wide packages + user packages). To summarize, here is our setup, which can probab ly break in many ways, but it works for us:

content of kernel.json:

{
  "display_name": "Julia 0.6",
  "argv": [
    "/path_to_script/julia_0.6_bin.sh",
    "-i",
    "--startup-file=yes",
    "--color=yes",
    "/usr/local/share/jupyter/kernels/julia-0.61/kernel.jl",
    "{connection_file}"
  ],
  "language": "julia"
}

content of julia_0.6_bin.sh

#!/bin/bash --login

export JULIA_PKGDIR="/home/${USER}/.julia"

# Check if IJulia is already installed by the user
if [ ! -d "${JULIA_PKGDIR}/v0.6/IJulia" ]; then
  mkdir -p "${JULIA_PKGDIR}/v0.6"
  cp -r /root/.julia/v0.6/* "${JULIA_PKGDIR}/v0.6"
fi

julia $*
1 Like
4

Hi!
Sorry for replying to an old thread.
I am also using jupyterhub and am trying to install julia-0.6.4 kernel on it. This is what I did:

  1. sudo su -
  2. I added Julia’s path to ~/.bashrc: export PATH="/home/ubuntu/julia-9d11f62bcb/bin:$PATH"
  3. I added IJulia package: Pkg.add("IJulia")
  4. I carried out @sylvaticus steps. I copied all the files form /root/.local/share/jupyter/kernels/julia-0.6/ to /usr/local/share/jupyter/kernels/julia-0.6
    I changed the kernel.json file:
{
  "display_name": "Julia 0.6.4",
  "argv": [
    "/home/ubuntu/julia-9d11f62bcb/bin/julia",
    "-i",
    "--startup-file=yes",
    "--color=yes",
    "/usr/local/share/jupyter/kernels/julia-0.6/kernel.jl",
    "{connection_file}"
  ],
  "language": "julia"
}

On running jupyter-kernelspec list:
julia-0.6 /usr/local/share/jupyter/kernels/julia-0.6
5. After this, I carry out the steps mentioned by @fabiangans, I revised my kernel.json file:

{
  "display_name": "Julia 0.6.4",
  "argv": [
    "/usr/local/share/jupyter/kernels/julia-0.6/julia_0.6.4_bin.sh",
    "-i",
    "--startup-file=yes",
    "--color=yes",
    "/usr/local/share/jupyter/kernels/julia-0.6/kernel.jl",
    "{connection_file}"
  ],
  "language": "julia"
}

And this is what my /root/julia_0.6.4_bin.sh says:

#!/bin/bash --login

export JULIA_PKGDIR="/home/${USER}/.julia"

# Check if IJulia is already installed by the user
if [ ! -d "${JULIA_PKGDIR}/v0.6/IJulia" ]; then
  mkdir -p "${JULIA_PKGDIR}/v0.6"
  cp -r /home/ubuntu/.julia/v0.6/* "${JULIA_PKGDIR}/v0.6"
fi

julia $*

I have added 5 users and when I run jupyterhub from root, the kernel Julia-0.6 is displayed in the dropdown list. When I open a file using that kernel, this is what the log says:

ERROR: LoadError: ZMQ.StateError("Address already in use")
Stacktrace:
 [1] [I 2018-07-26 03:53:50.942 SingleUserNotebookApp log:158] 200 GET /user/vishwesh/api/contents/Untitled.ipynb?content=0&_=1532577109578 (vishwesh@::ffff:103.242.62.49) 1.77ms
bind(::ZMQ.Socket, ::String) at /home/vishwesh/.julia/v0.6/ZMQ/src/ZMQ.jl:288
 [2] init(::Array{String,1}) at /home/vishwesh/.julia/v0.6/IJulia/src/init.jl:99
 [3] include_from_node1(::String) at ./loading.jl:576
 [4] include(::String) at ./sysimg.jl:14
 [5] process_options(::Base.JLOptions) at ./client.jl:305
 [6] _start() at ./client.jl:371
while loading /usr/local/share/jupyter/kernels/julia-0.6/kernel.jl, in expression starting on line 9

Can someone please help me out with this?

5

I ended up adding this to /usr/local/sbin/adduser.local :

su $1 -c "julia /usr/bin/initJuliaRepository.jl"

Where initJuliaRepository.jl is just:

Pkg.update()
Pkg.add("IJulia")

In this way, when I create a new user with adduser MYUSER, this automatically initialises an user level julia repository and adds IJulia to it.

6

Will the packages built manually or modified or installed in just one installation be also copied directly for the new user in this way?

7

What I done is simply let each user automatically initialise its own private julia repository and install IJulia on it.
If you mean system-wide installation, I don’t think this is possible with the current Pkg2, although there are some workarounds and Pkg3 should go on that direction:

8

It is possible, you just need to set up the paths (LOAD_PATH and Base.LOAD_CACHE_PATH) correctly.

9

I’ve just started exploring Julia since it has published 1.0. After reading various posts, I’ve figure out a way to setup Julia in JupyterHub at dclong/jupyterhub-julia. The setup is relative simple.

FROM dclong/jupyterhub:18.10

RUN apt-get update \
    && apt-get install -y julia \
    && apt-get autoremove -y \
    && apt-get autoclean -y

# install IJulia  
ENV JUPYTER=/usr/local/bin/jupyter
RUN julia -e 'empty!(DEPOT_PATH); push!(DEPOT_PATH, "/usr/share/julia"); using Pkg; Pkg.add("IJulia")' \
    && cp -r /root/.local/share/jupyter/kernels/julia-* /usr/local/share/jupyter/kernels/ \
    && chmod -R +rx /usr/share/julia/ \
    && chmod -R +rx /usr/local/share/jupyter/kernels/julia-*/
10

@stevengj @legendu,
Hi ,
I am facing issue in julia kernel for AD user.
If I am adding a user to host then it work fine but for add user kernel is breaking.
If you know how to resolve such issue please guide me.

Thanksss…

11

I have a working JupyterHub installation, forcing IJulia to install on every user creation. However it’s a bit uncomfortable. Does anyone knows how I can use the system-wide installation in this case? I have both 1.0 and 1.4 versions installed. Probably I’ll update 1.4 to 1.5 soon.
Thanks!