I’ve been trying some more stuff. With the command above, it looked like sudo was interfering with the Base.DEPOT_PATH variable getting set. What worked was this command:
$ sudo env JULIA_DEPOT_PATH=/opt/julia-1.4.2/share/julia julia
This allowed me to install IJulia to the global depot. However, after that I had to run:
$ sudo mv ~/.local/share/jupyter/kernels/julia-1.4/ /usr/share/jupyter/kernels
so the IJulia kernel could be used globally on the system. Now the kernel works systemwide on my JupyterHub server