Improve debugger GUI to help adoption?

Recently I’m promoting using Julia for our group’s next project. For context, we are choosing between Python and Julia for numerical simulations and prototype modeling. I advertised Julia’s speed, and did casual demos for some language features as well as the Juno IDE.

One interesting observation is, I’m mostly worried about resistance to adoption coming from the following directions:
(1) JIT compilation is slow leading to slow 1st time execution (the famous “time to first plot” issue).
(2) Some packages in Python are not available or not mature/stable in Julia yet. In particular, Julia native visualization packages are not as mature as matplotlib.
(3) The Juno IDE is heavy in resource use and a bit slow, while other IDE choices do not have some key features.
(4) Not easy and not always possible to distribute a binary executable.

However, I’m surprised when I got the feedback from two colleagues who played with Julia + Juno a little bit: they mainly complained about the debugger GUI in Juno!

Here I have to mention this: we are an engineering company, not a software company; except the software team, most of our engineers use Windows and write code in Matlab, Python, or VB.NET. Therefore, they are all used to the convenient debugger GUI in IDEs such as Matlab editor, PyCharm, and Visual Studio. Most have never used a command line debugger before.

In my view the debugger GUI in Juno is already much more convenient than the command line debugger, but there are still some flaws. The main ones they complained about are:

(1) Has to “Juno.@enter” a function. Cannot just push a button and start debugging the whole script.

(2) Must switch back and forth between compilation mode vs. interpreted mode. Interpreted mode is slow, and will crash at external C calls; compilation mode can avoid these issues but may not hit a break point, and WILL NOT hit a break point set inside a function B that is called inside the function A under debug.
The trick I showed them is to also set a break point inside function A where function B is called, and use compilation mode + continue to reach there quickly without crashing due to a C call e.g. in PyPlot. Once there, I can switch to interpreted mode, and continue. It will be a bit slower, but the break point inside function B will be hit.
However, this trick is a bit too complicated for them. They also ask what if function B is wrapped in a big loop inside function A, and they want to observe the break point in function B being hit again and again and see what changes. If they have to switch back and forth between compilation and interpreted mode, it is rather tedious; forgetting to switch could lead to missing a break point, a very long wait, or even crashing the debugger.

(3) Debugger may enter Julia’s internal code if one wants to step over macros like @time and @profiler. Also debugger support for multi-threading does not seem to be ready?

(4) (This one is minor) No watch list for variables. Watching a single variable is OK but not watching several at the same time.

(5) In general, the debugging experience doe not feel responsive and solid. If the debugger hangs, one always wonders if he/she did something wrong, or if it’s just because debugger is still busy executing code in the interpreted mode.

I personally feel my debugging experience is acceptable, but my colleagues think there is no way they can debug something moderately large (e.g. thousands to tens of thousands lines of code with several modules) that’s written by somebody else. And some are willing to choose Python solely based on the current debugger situation. For them, visual debugger is the main tool they employ to understand other engineers’ code, since for engineers, unlike software developers, documentations for code are almost always lacking, and the original author(s) often have left for another company (silicon valley life). So reading others’ code is often like detective work, and the visual debugger is the most powerful tool - use real data as input to observe how it calculates, which logic branch it enters, and what values change, etc.

I want to focus on a language’s future potential in our applications, so I’d pick Julia any day. However, it seems to me that at least in some industries, debugger GUI could become the major obstacle to adoption, and this might be hard to understand for those who write code alone, or for software guys to whom Linux style command line operation is familiar, or where code architecture is clean and documentations are good. However, as Julia is competing with Matlab, I think we also need to pay attention to the need of those engineers who are used to Windows and Matlab. :slight_smile:

15 Likes

Agreed that interpreter speed is a major problem. It’s not that we don’t know or care about this issue, it’s that Kristoffer and I both do a ton of other stuff in the Julia ecosystem and haven’t had time to put real work into this issue. It would be great if a few more people jumped into this area.

Probably the most promising route to improving performance would be to come up with an optimized byte-code, an effort started at https://github.com/JuliaDebug/JuliaInterpreter.jl/pull/309.

16 Likes

Hi Tim, first got to say thank you and other Julia developers so much for the great packages! I really enjoy working in Julia.

Speaking for myself, I’d love to help with things where I can… Unfortunately my programming skills are limited to the user level - meaning that I can learn a high-level language quickly and write code to run simulations in my area of work, but I know little to nothing about compilers / interpreters, language/symbol servers, IDE plugins, Electron GUI design, etc. So my ways of participation are mostly limited to creating bug reports, posting discussions like this, and promoting Julia to friends and colleagues. :slight_smile:

Now I’d risk to say majority of Julia users are probably like me - used Julia and other languages enough to comment on things, but do not possess enough knowledge to develop Julia and its core ecosystem itself. Not sure if there is a way for those Julia users to contribute a little voluntarily without the requirement of technical skills in such things like programming language design and compilers?

You may want to post your ideas of what can be done in the frontend to improve Juno’s debugger here, like debugging watcher or debugging toplevel scripts directly, etc.

Well the most of user complaints are the speed, which should be solved at upstream level as @tim.holy says

What is the current thought on Infiltrator integration into Juno?

1 Like

Your intuition is right, there’s little one can do to help fix the interpreter performance without taking a deep dive. But if there’s someone who wants to invest the time, it’s a learnable skill; I certainly was never formally trained in any of this (the internet is an amazing thing :smile:), and my impression is that Kristoffer also just figured it out as he dove in. But it’s also understandable if you don’t want to go there and instead focus on things that you’re already efficient in doing; it’s just that then you have to be prepared to wait for someone else to take that plunge or otherwise find the time to invest in it.

Bottom line is that there’s no harm in bringing up a topic like this (everything you said is true), but FYI it also doesn’t do much to advance matters either. The issue reports and unfinished pull requests over at JuliaInterpreter already indicate abundant awareness that this is an important area. Fixing it will require resources, pure and simple, with no way to predict when those will become available.

14 Likes

Thank you for the suggestion. I will do that.

I think there is a lot of promise in the:
“Compile things until we need to debug them” approach benchmarked here:
https://github.com/JuliaDebug/JuliaInterpreter.jl/issues/306#issuecomment-536196825
and prototyped in https://github.com/oxinabox/MixedModeDebugger.jl

It does need some time spent to sit down with Debugger.jl folk and workout what (if anything) we want to do with this.

6 Likes

Call me impressed. I didn’t know one can pick up those skills (designing compiler/interpreter for a new language) by reading off the internet. :slightly_smiling_face: Maybe I should try to learn in the near future.

I don’t have complaints myself for the IDE or the debugger - I’m actually pretty happy with how the IDE and debugger progressed in just 6-12 months. What I wanted to share is this new info: I thought some other things like JIT compiler speed is more important to address for wider adoption, but it turned out the debugger GUI is more important - at least for the colleagues I heard from.

This is a small surprise to me, and may be new info for guys who do not use debugger GUI (my brother being one - he swears by vim and thinks printf is the best debugging tool :joy:). I don’t think anybody is “right” or “wrong”, just that Julia users come from many different backgrounds and have quite different perspectives, so it’s helpful to understand the different perspectives in order to promote Julia to a wider adoption. :slight_smile:

1 Like

Hi @oxinabox , thanks for sharing this. I don’t fully understand the post, but is this idea essentially doing the debugging mode switching automatically? Use compiled mode until a break point, then automatically switch to interpreted mode? When I hit “continue”, does it get switched back to compiled mode automatically, and switch to interpreted mode automatically again when a break point is hit?

Yes, it roughly does you point 2 automatically.
If I have understood correctly.

Yes, if you hit continue it will switch back to compiled once it returns from the function with the breakpoint, so functions called after you continue but before you return will be Interpretted still.
If we went further down this path we could do something about that.

Right now it’s a working proof of concept

1 Like

I also have the same question.

Either

or

are good candidates to solve the debugging issue.
They just need to be integrated to Juno.

2 Likes

As far as I understand Inflitrator: everything is fast except you you can’t “step into” functions. And it effectively would need to recompile the function each time you add a breakpoint.

If so, I think that is a reasonable tradeoff for the GUI. Tell people that “step-into” isn’t yet supported in the Infiltrator debugger so they should set breakpoints in any functions they want to debug. It isn’t perfect, but it gives a reasonable workaround and an otherwise solid experience? Or am I missing another downside to Infiltrator?

1 Like

Hi @jlperla, I tried Infiltrator and liked it a lot. It’s equivalent to Matlab’s “keyboard” which I am familiar with.

The scenario we do need a full visual debugger is, again, not to debug one’s own code, but to understand others’ code when there is no good documentation or clear architecture. When you have a spaghetti ball of function calls and you have to follow the execution to understand what the hell is going on, and when re-architecturing the whole thing is not an option, a visual debugger is the convenient detective tool one can apply to follow through at least a few typical cases and try to understand how the code works.

This happens a lot with some big projects that evolved over years. Of course a new Julia project wouldn’t be in such a sorry state, but I can understand people still want the tool to be there. :slight_smile:

5 Likes

I am 100% with you on the end-goal of a matlab debugger. It is outstanding and I live in it with matlab. But it will only happen towards that slowly without a huge external invsetment.

My goal is to figure out what is the best solution given the currently available tradeoffs. My feeling is that if Infiltrator was put into the GUI - for matlab style users - it would be a better set of tradeoffs (i.e. no stepping into functions or especially dynamic walking through a function, but running at a reasonable speed) than the current interpreter based approach which have different tradeoffs.

But I am not sure…

4 Likes

I’m not going to promise any timeline, but we still plan to integrate the debugger into the VS Code extension at some point. We would automatically get a very good UI that way that we don’t have to maintain.

11 Likes

@davidanthoff Looking forward to it!

I guess any improvement to JuliaInterpreter speed, as well as automatic switching between compiled mode vs. interpreted mode, etc. would still be useful in the VS Code extension, right?

Yes, whenever we get to work on this, we would just be using whatever debugger infrastructure exists at that point.

What we need to do to enable this scenario in VS Code is write a debug adapter. We’ve started a little bit already, but this is not a small project, and right now we are focusing on polishing some other things in the extension.

For reference this is. It normally a problem.

That is generally cheap enough.
It’s only voiding (part of) compile cache.
Extreme worse case scenario is it takes as long as first run time.

Any action you take during to user IO only needs to fast vs human reaction time.

You don’t want to recompile in your hot loop. But for setting breakpoints? No worries.

MixedModeDebugger trigger a similar amount of recompilation when you set a breakpoint.

1 Like

@pfitzseb things a approach similar to MixedModeDebugger’s could be added to Infiltrator to allow this.
(I’m not sure I see how to do that, exactly. But it feels right.)