Setting the module for REPLCompletions would also make https://github.com/JuliaLang/IJulia.jl/pull/651 better.
The original motivation for julia-repl was to punt to Julia’s REPL implementation for all of the extra features (documentation, Pkg, shell) by pretending to be a terminal. Unfortunately, that design has proven to be a dead end, because Emacs’s terminal support is buggy.
A major reconsideration of this approach is necessary, which will probably involve implementing an interface for quite a bit of functionality. While this is a burden, at the same time it can probably be made more friendly (eg make docstrings appear in a new window, render them from Markdown or HTML, add a menu-driven interface for Pkg3 functionality, etc) — my ideal is something like SLIME. I will probably also involve some PRs to packages and Base separating functionality and interface.
Jumping to source code locations should be trivial, just filtering for a regex. I will look into making it work with your code.
1 Like
A major reconsideration of this approach is necessary, which will probably involve implementing an interface for quite a bit of functionality.
That’s the approach followed by ESS, which works pretty well. If you’re considering moving away from the terminal approach (which I think is a good idea, terminal support in emacs is crap), please give it a go: it would be very nice to reduce the fragmentation of the emacs-julia ecosystem.
We have similar issues with the new Juno REPL (not quite as bad though). We can fix ours by changing this function and I’m mildly confident you might be able to fix the emacs issue as well (at least I recall seeing the behaviour from that issue when I made a mistake while trying to patch refresh_multi_line).
1 Like
ESS was one of the first things I looked at to contribute to when I started using Julia in Emacs. Unfortunately, I find the code difficult to understand, I think this comes from it being mostly adapted to R/S/S-plus, which was then modified slightly for other languages.
That’s true, but ESS is a pretty stable and established piece of software, and so you get a few goodies for free from that integration (ie the “feature to julia-specific LOC” ratio is pretty good).
My goals aren’t really aligned with ESS - I want to support multiple editors (it looks like NeoVim/Emacs/Sublime are the main choices from my colleagues) and multiple languages (which are all implemented in Julia). So I’m trying to do as much of the work (parsing, display, debugger interface) as possible on the Julia side and keep the editor-specific code to a minimum.
I just saw that IJulia supports changing modules and they use the same REPLCompletions code that I do. If it’s possible to send new cells to Jupyter from the backend then I could use that as my display.
1 Like
I am not sure I agree. ESS is based on comint, and uses a lot of tricks to capture/redirect output. These can be very fragile and get very tedious to maintain after a while. I think that a client-server approach, as suggested eg by @jamii here, is the best way to go.
It seems like you are converging towards re-inventing Jupyter. Note there are already plugins for Atom (hydrogen), neovim (nvim-ipy), vscode, and emacs (EIN) to talk to Jupyter kernels, as well as a bitrotted plugin for sublime. Why not work on these rather than re-inventing the wheel?
1 Like
I was not aware of EIN, thanks for the reference. Apparently the version you link was abandonned then taken up again.
I prefer the buffer-REPL interaction approach to a single linear notebook, but maybe I can use parts of this.
If I am just looking for a protocol that sends stuff to a Julia process and gets output back from it, what’s the current standard I should consider, IJulia, LanguageServer.jl, or something else? My main concern is
- dealing with queries by various interfaces (eg
Pkg3usesTerminalMenus, is there a solution/workaround for that?) - other REPL modes: how to make documentation, shell, and pkg modes transparent to the user; as if using the terminal,
- how to deal with the debugger (eventually).
I am comfortable with implementing frontends, but it would be nice to share work on the Julia side for the relevant hooks. I am not sure what is emerging as a standard.
2 Likes
It could certainly be made to work in the Jupyter notebook and JupyterLab via Javascript widgets (potentially with some modifications to TerminalMenus to support other backends), but I don’t think the Jupyter protocol has defined a frontend-independent way to display a menu. However, the Jupyter messaging protocol includes a catch-all metadata field, and you could use this to define Julia/frontend-specific extensions. This allows you to re-use the “standard” Jupyter protocol for as much as possible while defining small extensions only where they are really needed.
The Jupyter folks are already working on debugger support, so again it seems better to join forces than to re-invent the wheel. In general, they are quite open to contributions.
1 Like
You mean like what JupyterLab already allows? Don’t confuse the notebook front-end with the messaging protocol, which can be used for many different styles of front-end.
1 Like
The problem with using Jupyter for this is that there’s a convention in some
Julia packages to provide functionality by tying directly into the REPL. The
most obvious example here is shell commands with ; and help commands with ?.
IJulia provides both of those, but it’s currently just explicitly
looking for lines that start with those characters. Packages like
Gallium/ASTInterpreter2 won’t work at all in IJulia. Nor will things like the
C++ shell in Cxx.jl or LispREPL.jl.
I spent some time digging through IJulia and generally Julia’s REPL code to see
if there was a way to allow this stuff to work more generically, but I got more
than a bit lost and ran out of time to work on the problem.
For anyone wondering, outside of emacs I use IJulia console as my main REPL if I
don’t need something like Gallium, and it works very well. I do this since I
haven’t figured out how to get vi-style keybindings working with the default
REPL yet.
Yes, but that approach needs to be lifted to a more generic backend mechanism before any real client-server functionality can happen…
They can work. I assume @Keno had already abandoned ASTInterpreter.jl at that point, so the PR got lost, but presumably it could be ported to ASTInterpreter2.jl
1 Like
This is why I think that for a workflow that involves
- working in a buffer then
- sending parts to the REPL
a terminal emulation is the cheapest option.
I’m very interested in these developments – they could be used to build something in the line of Erlang’s remote debugger. Which would be amazing, considering Julia’s distributed computing capabilities.