Networked REPL?

Internals & Design
7

Ok, just because I don’t have anywhere else public to make notes, this is just a quick rundown of potential issues I see with adopting IJulia as a REPL server. I am by no means an expert, and this is quite literally the first look I’ve had at the IJulia code, so please feel free to correct mistakes if I’ve made any, and I consider all points open for debate!

Pros

Cons

  • History appears to be tracked by the server (https://github.com/JuliaLang/IJulia.jl/blob/master/src/IJulia.jl#L227). History should really be a client concern for two reasons
    • Two users connected to the same REPL server shouldn’t have their histories interleaved
    • Keeping history client-local is useful for replaying commands across servers. e.g. Many times with Clojure I’ve connected to a staging REPL, evaluated some code then, after confirming the code does what I want, reconnect to a production REPL and re-eval the same code from my local history.
    • That said, having history on the server does not preclude client history, though managing two histories could be confusing.
  • Mode switching has a few problems
  • Handling of shell (;) and help (?) lines is based on the first character of the code string to be evaluated (https://github.com/JuliaLang/IJulia.jl/blob/master/src/execute_request.jl#L139), and these characters are hard-coded.
  • Modes are not extensible the way they are in the Julia REPL (e.g. Keno’s C++ mode).
  • The current IJulia implementation of Jupyter messaging is a tad ad-hoc (https://github.com/JuliaLang/IJulia.jl/blob/master/src/msg.jl#L69)
    • Would love to see this split into its own module for easier development if/when a new version of messaging is released.
    • A separate module would also facilitate middleware development.
  • The methods in InlineDisplay (https://github.com/JuliaLang/IJulia.jl/blob/master/src/inline.jl) seem like they should be client concerns (though I haven’t had time to dig into the corresponding Jupyter side of this bit).

Overall, I think IJulia could definitely form a solid base for a Julia REPL server. What I would suggest, on a high level, is the following:

  • Modularize. Off the top of my head:
    • Core
      • IO
      • MessageProtocol (decode & encode for client & server)
      • Evaluation
    • Middleware
      • ModeSwitch
      • History
      • InlineDisplay
  • Make ModeSwitch pluggable, provide default plugins for Help and Shell
  • Implement IJulia server as Core + all Middleware
  • Implement JuliaREPLServer as just Core + ModeSwitch

So far the biggest potential hurdle I see is that the Jupyter message protocol is too corse-grained. For example, it would be nice to be able to implement code completion as a server-side plugin, since the server will have full context for what symbols are in scope. However, this would necessitate the ability to send “action” events (e.g. user pressed <TAB> requesting a completion). Another example is mode switching. The server should contain information about what modes are available, but switching between modes (e.g. changing the REPL prompt to help> when the ? key is pressed) would require messages to be sent on each key press. (An alternative implementation would have the ModeSwitch middleware implemented both client and server-side, but this would still require some means of communicating non-code messages client-to-server, e.g. to establish which ModeSwitches were currently installed.)

That’s all I have time to write up for the moment. If this seems like a good start on a rough outline, I’d be happy to turn some of these points into GitHub tickets (just wanted to be able to have some discussion around the issue first).

1 Like