Launch Jupyter kernel with -e "using SomePackage" flag

General Usage
,
1

Hi,

I’m trying to setup Jupyter kernel with some automatically loaded packages as a matter of convenience. Lets say I want to have Julia Jupyter kernels for: data (Statistics, DataFrames), plotting(UnicodePlots, PyPlot), symbolic math (SymPy, PyCall).

What I would like to do is to create separate Jupyter kernels for each of these tasks.

What I’m trying to do is to create new kernel using IJulia in the following way:

julia> installkernel("sympy-jl", "-e \"using SymPy\" ")
[ Info: Installing sympy-jl kernelspec in /Users/Tikej/Library/Jupyter/kernels/sympy-jl-1.5
"/Users/Tikej/Library/Jupyter/kernels/sympy-jl-1.5"

Then I try to launch the server using:

Tikej$ jupyter-kernel --debug --kernel sympy-jl-1.5


[KernelApp] Searching ['/Users/Tikej', '/Users/Tikej/.jupyter', '/usr/local/Cellar/python@3.8/3.8.5/Frameworks/Python.framework/Versions/3.8/etc/jupyter', '/usr/local/etc/jupyter', '/etc/jupyter'] for config files
[KernelApp] Looking for jupyter_config in /etc/jupyter
[KernelApp] Looking for jupyter_config in /usr/local/etc/jupyter
[KernelApp] Looking for jupyter_config in /usr/local/Cellar/python@3.8/3.8.5/Frameworks/Python.framework/Versions/3.8/etc/jupyter
[KernelApp] Looking for jupyter_config in /Users/Tikej/.jupyter
[KernelApp] Looking for jupyter_config in /Users/Tikej
[KernelApp] Starting kernel 'sympy-jl-1.5'
[KernelApp] Starting kernel: ['/Applications/Julia-1.5.app/Contents/Resources/julia/bin/julia', '-i', '--startup-file=yes', '--color=yes', '-e using SymPy', '/Users/Tikej/.julia/packages/IJulia/rWZ9e/src/kernel.jl', '/Users/Tikej/Library/Jupyter/runtime/kernel-e1141e24-b343-4a12-a8aa-7ef7badba64f.json']
[KernelApp] Connecting to: tcp://127.0.0.1:63792
[KernelApp] Connection file: /Users/Tikej/Library/Jupyter/runtime/kernel-e1141e24-b343-4a12-a8aa-7ef7badba64f.json
[KernelApp] To connect a client: --existing kernel-e1141e24-b343-4a12-a8aa-7ef7badba64f.json

However, when I connect the client, the kernel is dead:

jupyter-console --debug --existing


[ZMQTerminalIPythonApp] Searching ['/Users/Tikej', '/Users/Tikej/.jupyter', '/usr/local/Cellar/python@3.8/3.8.5/Frameworks/Python.framework/Versions/3.8/etc/jupyter', '/usr/local/etc/jupyter', '/etc/jupyter'] for config files
[ZMQTerminalIPythonApp] Looking for jupyter_config in /etc/jupyter
[ZMQTerminalIPythonApp] Looking for jupyter_config in /usr/local/etc/jupyter
[ZMQTerminalIPythonApp] Looking for jupyter_config in /usr/local/Cellar/python@3.8/3.8.5/Frameworks/Python.framework/Versions/3.8/etc/jupyter
[ZMQTerminalIPythonApp] Looking for jupyter_config in /Users/Tikej/.jupyter
[ZMQTerminalIPythonApp] Looking for jupyter_config in /Users/Tikej
[ZMQTerminalIPythonApp] Looking for jupyter_console_config in /etc/jupyter
[ZMQTerminalIPythonApp] Looking for jupyter_console_config in /usr/local/etc/jupyter
[ZMQTerminalIPythonApp] Looking for jupyter_console_config in /usr/local/Cellar/python@3.8/3.8.5/Frameworks/Python.framework/Versions/3.8/etc/jupyter
[ZMQTerminalIPythonApp] Looking for jupyter_console_config in /Users/Tikej/.jupyter
[ZMQTerminalIPythonApp] Looking for jupyter_console_config in /Users/Tikej
[ZMQTerminalIPythonApp] Connecting to existing kernel: /Users/Tikej/Library/Jupyter/runtime/kernel-e1141e24-b343-4a12-a8aa-7ef7badba64f.json
[ZMQTerminalIPythonApp] Loading connection file /Users/Tikej/Library/Jupyter/runtime/kernel-e1141e24-b343-4a12-a8aa-7ef7badba64f.json
[ZMQTerminalIPythonApp] connecting shell channel to tcp://127.0.0.1:63789
[ZMQTerminalIPythonApp] Connecting to: tcp://127.0.0.1:63789
[ZMQTerminalIPythonApp] connecting iopub channel to tcp://127.0.0.1:63790
[ZMQTerminalIPythonApp] Connecting to: tcp://127.0.0.1:63790
[ZMQTerminalIPythonApp] connecting stdin channel to tcp://127.0.0.1:63791
[ZMQTerminalIPythonApp] Connecting to: tcp://127.0.0.1:63791
[ZMQTerminalIPythonApp] connecting heartbeat channel to tcp://127.0.0.1:63793
[ZMQTerminalIPythonApp] connecting control channel to tcp://127.0.0.1:63792
[ZMQTerminalIPythonApp] Connecting to: tcp://127.0.0.1:63792

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/jupyter_console/ptshell.py", line 433, in init_kernel_info
    reply = self.client.get_shell_msg(timeout=1)
  File "/usr/local/lib/python3.8/site-packages/jupyter_client/client.py", line 78, in get_shell_msg
    return self.shell_channel.get_msg(*args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/jupyter_client/blocking/channels.py", line 54, in get_msg
    raise Empty
_queue.Empty

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/bin/jupyter-console", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.8/site-packages/jupyter_core/application.py", line 270, in launch_instance
    return super(JupyterApp, cls).launch_instance(argv=argv, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/traitlets/config/application.py", line 836, in launch_instance
    app.initialize(argv)
  File "/usr/local/lib/python3.8/site-packages/traitlets/config/application.py", line 86, in inner
    return method(app, *args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/jupyter_console/app.py", line 137, in initialize
    self.init_shell()
  File "/usr/local/lib/python3.8/site-packages/jupyter_console/app.py", line 107, in init_shell
    self.shell = ZMQTerminalInteractiveShell.instance(parent=self,
  File "/usr/local/lib/python3.8/site-packages/traitlets/config/configurable.py", line 510, in instance
    inst = cls(*args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/jupyter_console/ptshell.py", line 351, in __init__
    self.init_kernel_info()
  File "/usr/local/lib/python3.8/site-packages/jupyter_console/ptshell.py", line 436, in init_kernel_info
    raise RuntimeError("Kernel didn't respond to kernel_info_request") from e
RuntimeError: Kernel didn't respond to kernel_info_request

I’m pretty sure it’s not matter of timeout because of compilation/loading times as I tested with builtin modules (Statistics for example) and they should load instantly (they don’t work either).

Regular kernels, without ‘-e’ flag work great (for example using -O3 flag from installkernel example).

Is this a bug or am I doing something wrong?

PS

julia> installkernel("sympy-jl", "-e using SymPy")

doesn’t work either.

2

Jupyter launches IJulia with julia [...options...] /path/to/IJulia/src/kernel.jl. Unfortunately, this is incompatible with -e.

The reason is that, if you run julia -e foo bar.jl, Julia executes the -e foo but ignores the bar.jl — -e supersedes the file input. So, if you try to pass -e during IJulia launch, it won’t execute IJulia itself.

(I’m not sure why julia behaves this way; it would make more sense to me to execute both the -e command and the file. Maybe this could be changed in a future release.)

1 Like