How do you use debuggers?

[ScottPJones]

@glemieux

I guess this might be related to different modes of learning/mastering a particular subject?

When I first learned both how to type, and to program, making a mistake was incredibly costly (having to retype a whole page, or at the very least to stop, use white-out, wait for that to dry, and then retype over that. In a program, where you had to type everything onto punch cards, then submit them as a batch job, and wait (sometimes hours) to get the results from your job (because of all the other students queued up).
No debuggers available. In both cases, you learn to avoid mistakes at all costs. Not until about 7 years later, when I got to MIT, I was exposed to REPLs (in MacLisp, Scheme, then BASIC), and later a form of IDE (the Lisp Machines), but I already had been trained to spend most all of my time in design, and not in debugging.
(Measure 3 times, cut once )

I’ve learned to use debuggers as some people have mentioned as code (and data) inspectors, which is nice, but mostly I rely on logging & debug printing that I can switch on an a per module / function / condition basis (since that works even for when dealing with multiple processes / services / machines, while a debugger/IDE is generally limited to the process you are running it on [although some let you attach to an already running process, that’s usually limited to processes on the same machine]).

For me, the most interesting things to have would be the ability to inspect running Julia processes, and to be able to turn on/off the ability to more fully inspect what’s going on in a process on a per-method basis (so that things don’t become too slow to test), with some form of conditional breakpoint/watchpoint functionality, and for learning how the code is actually working, the ability to walk down through the code (I’d like to use ASTInterpreter2, but that’s still not working on master, AFAIK).

[Tamas_Papp]

@glemieux

being able to ‘inspect’ the code through a debugger is extremely helpful

I don’t disagree, but I have come to recognize the need for a debugger as a sign of code smell in Julia. Ideally, well-written Julia code consists of small functions that do one thing, and can be unit tested as building blocks of a larger solution.

I agree that one does not always control the quality of other’s code, and a debugger is occasionally useful. But I also think that scientists without a lot of programming experience

1. have a bias towards large monolithic functions,
2. tend not to be aware of the existence of unit tests,
3. instead, the approach is “when it errors, we use a debugger”.

So when I see something that would require a debugger to, er, debug, I have an itch to refactor the whole thing. Or rewrite it from scratch. Otherwise, I can’t trust the quality of that piece of code, so it is worse than useless.

That said, what I would find really useful is as an interim solution is the ability to inspect function arguments in a stack trace.

[ScottPJones]

@Tamas_Papp

That said, what I would find really useful is as an interim solution is the ability to inspect function arguments in a stack trace.

I agree totally - most of my “debugging” time is spent adding formatting output to functions (which is why I wrote StringUtils.jl, and then expanded it to StringLiterals.jl, the *printf functions/macros or doing things like print("0x$(string(value,base=16))") are too verbose, p"\%x(value)" is much more to my liking), and that’s all because I can’t inspect the function arguments in a stack trace.

Besides better ways of inspecting code (often, I’m not really debugging, the code works, passes unit tests, etc. but rather trying to figure out just what methods are actually being called - something not always easy when dealing with complex type specifications and parameterization and occasional dynamic dispatch), one thing that really interests me these days is having a recording debugger, that would help you be able to reproduce the state where the error occurred.

I’m also wanting to use Traceur.jl, hopefully it will be work soon “out-of-the-box” on master (problem is with packages it depends on, don’t know if they’ve been updated yet).

[randyzwitch]

@ScottPJones

You seemed to be making it out as if the problem would be too difficult do be tackled without loads of money and people. My point is that it might not be such a difficult task as you have presented.

If this were the case, why are so many people talking about it, but no one doing it?

[ScottPJones]

@randyzwitch

If this were the case, why are so many people talking about it, but no one doing it?

Possibly because people are thinking that the problem is a lot more difficult than it is, and hence, decide to spend their time on other projects where they feel there is more likelihood of success.

I’m not saying that creating a good debugger isn’t difficult, but I think that claims that it would cost multiple millions of dollars and tons of people is probably FUD.

[StefanKarpinski]

That is not what I claimed. I said that most debuggers of this kind that exist have cost that much and taken that much time. How much do you think Google has spent on the V8 debugger? How much has Apple spent on the WebKit debugger? At the same time, I think that an additional six months of Keno or Jeff or Jameson’s time would do it. Unfortunately, we can’t afford that right now since there are other things they need to do more urgently, like finishing 1.0 and keeping Julia Computing going. If we could hire another five people with that kind of skill set, sure, we could get this done quickly, but that would cost multiple millions of dollars. Moreover, we couldn’t necessarily do that even if we had the money and that was the only choice to spend it on because those kind of people aren’t generally for hire. They come to a project because they want to. Unless you’re Google/Apple/Facebook and you can afford to pay them a small fortune to do the work.

All this is to say (again) that you have three options regarding a debugger for Julia 1.0:

• spend a few million on a compiler team
• do it yourself
• wait

Further explanations of how useful debuggers are or how easy they are to implement are not helpful.

[jonathanBieler]

I wonder if we could use some macros or types to help debugging. For example when I have an issue in a function I can’t understand by just reading the code I go through the calling chain by hand and define all the variables used in the function in Main so I can check the values and test the function calls. One could probably make a macro that does that for you, e.g.

@bring_to_main test(x)
    y = sin(x)
end 

julia>test(0)
julia>y
0.0

[jlperla]

Everytime debuggers are brought up, there is inevitable lecturing from advanced programmers about how “nobody needs a debugger, it is indicative of a poor design, all I need are better stack traces, etc…”.

Whether those comments are true or not, it is a good way to separate the different types of users. These (primarily skilled package developers) are the the exact opposite of the people the debugger is intended to support. With all due respect, the easiest way to simplify a debugger is to split out those requirements into something completely different (i.e gdb style “debugger”, and utterly useless for the vast majority of people who say they want a debugger).

So if the main users of a debugger are scientists and students writing scripts, who find computing somewhat of an annoyance, then getting the GUI right is essential. Doing something half-assed and targeted at package developers is useless or worse. If a matlab or low-tech python user tries out “the debugger” and it is all about stack traces and macros, then they are likely to conclude that Julia is intended as a hardcore systems language, and not towards their needs.

So better to wait until the right resources are in place, do it right, and attempt to make it a matlab style code-stepping utility. My feeling is that all systems debugger style features can be entirely ignored or patched on as minifeatures (as suggested here).

When the time is right and the resources are in place, the big requirement that can slide is about speed. Obviously, the debugger can’t run all of Julia in interpreted mode, but it can be pretty slow (e.g two orders of magnitude is perfectly reasonable in code that is actually be debugged). A full Visual Studio style debugger which runs pretty fast and still manages to allow debugging anywhere in the codebase is not necessary at first.

I am under the assumption that the approach is basically to have an “AST interpreted” mode for functions that are being debugged, where as many other functions are compiled as possible. I imagine that the issue of having nested functions support this is an issue due to intensive inlining/etc. If so, then let me propose a simplifying assumption:

• The only functions which can run interpreted are ones that have a breakpoint set in them. i.e. hitting F9 in the GUI or right clicking to create a breakpoint. Note I am saying the GUI, because that is what the users actually want.
• The idea is that the breakpoint would flag the method as being something to run interpreted.
• Now, how would this work nested deep inside a bunch of calls where it may have been inlined? It wouldn’t! Methods run interpreted only if their caller was running interpreted…
• At the top level in Atom (note I am not even saying REPL, because the GUI is the key) lets say I have a “debug mode” setting.
• If that setting is on, then it would check whether the top-level method is flagged to be interpreted or not, and would run the method in the debugger it it was.
• Next, when the interpreter is running a method in interpreted mode, it can check to see if methods it is calling are in interpreted mode before deciding if it wants to enter them or not.
• This way, the only way to be in interpreted mode is if the entire call stack to that point was interpreted (and had breakpoints).
• Down the road, things could be setup so the “Step Into” (i.e. F11 or whatever) automatically flags a function it is about to call in interpreted mode, so that they wouldn’t have to actually set a breakpoint in each function.

Would this be confusing in practice? Not really, because they majority of people using this sort of debugging would be in only a few functions deep. Now, I have no idea if this helps simplify much… I know that even a slow interpreter is still a major undertaking. But this could get people who really want a debugger 80% of the way there.

[ScottPJones]

@StefanKarpinski

Further explanations of how useful debuggers are or how easy they are to implement are not helpful.

That’s rather a misrepresentation. First off, the topic of this is how people actually use debuggers (and many use them for code inspection / exploration, rather than trying to debug an error that occurs). The responses can be helpful when it comes time to prioritize just which functionality people really are looking for.
Secondly, nobody has said writing a debugger is particularly easy - I just said that some of the things you seem to feel make things very hard would not in fact be that difficult to deal with, and tried to present solutions to those issues. You also seem to be changing what was said. Now you are saying that the multiple millions of dollars cost would be to get it done quickly (which was not being discussed before), and that it could be done in 6 months by Keno, Jeff, or Jameson [much as they might deserve it, I doubt any of them is paid multiple millions of dollars a year, let alone for 6 months]. I also understand that there are other priorities before v1.0 is finished, but again, that was not was being discussed. The discussion was about how people use debuggers, and which features they really felt were useful. I’d already said that I personally don’t use debuggers much at all, because of my background, because I prefer to rely on unit tests and thorough code reviews, and because I rely on having good logging and formatted output facilities instead since I’m dealing with code running spread across multiple services.

[pfitzseb]

Just wondering whether you’ve tried Juno’s debugger/stepper recently – it works pretty well imho, but doesn’t implement the breakpoint stuff you describe above yet.

[jlperla]

Thanks, I will take a look as I haven’t played with it for a while. My feeling is that this Juno stuff is exactly on the right track. A souped up version of this that with more GUI would be what the people clammering “we need a debugger” would want.

[Tamas_Papp]

@jlperla

inevitable lecturing from advanced programmers about how “nobody needs a debugger, it is indicative of a poor design, all I need are better stack traces, etc…”

Apologies if I came across as lecturing, it was not intended. There are situations where debuggers are very useful, I did not mean to suggest otherwise.

@jlperla

If a matlab or low-tech python user tries out “the debugger” and it is all about stack traces and macros, then they are likely to conclude that Julia is intended as a hardcore systems language, and not towards their needs.

There is nothing that can protect people from erroneous conclusions, but I would hope that users would also base their assessment of Julia on other things, eg the language design (elegant, interactive yet fast), the manual, the package ecosystem, and the great examples of code out there on just about anything (in the future ).

@jlperla

When the time is right and the resources are in place, the big requirement that can slide is about speed.

This I can agree with, but one of the things users need to debug frequently in Julia is performance regressions, which can also be the most confusing for newcomers who are still adapting to Julia’s performance model.

[foobar_lv2]

So, to give my 2 cents on where I use debuggers and how a good debug experience looks like.
The in-browser javascript debuggers, both on chromium and firefox, are ridiculously user-friendly.

I use them for

1. “normal debugging”: My code is not doing what it is supposed to do; why? This is especially important if you use javascript/html5 as output format, e.g. interactive plots (because your problem can either be in your julia/python/… code, in the framework for interactive plots, or in your hand-written string-interpolated javascript code).
2. Understanding package code / normal API discovery: How is this supposed to work (especially if the documentation is insufficient)? Well, let’s step through it and inspect what is happening, i.e. what type this function actually expects in practice (type in the informal sense; javascript’s typesystem is too poor to properly describe composites).
3. Adversarial API discovery / reverse engineering / security testing. Combine burp suite and the inspector to figure out how an API (both web and local in-browser) is actually working, and see whether you e.g. access some surprising data, or whether the crazy third-party analytics javascript manages to transform some reflected data into an XSS.

I don’t think (3) is relevant for julia. While a debugger is nice for (1), I can live without one, using the dreaded “printf debuggging”.

But for (2), a good debugger with breakpoints and stack / local variable inspection is essential. Especially in julia, where one can have a single function which calls different methods of itself, which do genuinely different things, and it is also common to skip explicit type annotations if they can be inferred. This combines implicit types “this is supposed to be an array of tuples containing these pointers and that counters…”, which make javascript/python code hard to understand, with multiple dispatch, which makes it even harder to figure out which method actually gets called. This gets even worse if the code is macro-heavy. For example, the profiling code in Base is not entirely obvious to understand; stepping through it even once would have saved me quite some time for working with the raw output.

StefanKarpiski: I try not to whine about missing features. You explanation of current priorities was appreciated, and I am neither inclined nor capable to write a debugger myself. I’m looking forward to getting one eventually, but feel no need to get impatient.

[piever]

This as been mentioned already but I wanted to emphasize that Juno stepper functionality is already quite good. It does not have breakpoints yet but the “step to line” and “step into function” features allow to simulate it even though in a slightly more laborious way.

[ExpandingMan]

I haven’t read all of the responses here, but I wanted to mention that I’ve been using Revise.jl a lot more lately, and, while it is certainly a different tool than a debugger I have found that (at least in a vague sense) it enables some of the “tinkering” that I had enjoyed with the Python debugger. Revise.jl doesn’t just give you a way of reloading entire modules without exiting the REPL, it actually allows you to edit individual functions at your leasure without reloading, something you cannot do in most languages without a debugger of some kind. So, if you are still feeling aggravated about the lack of a debugger in Julia right now, even though Revise.jl is not an actual debugger, I strongly suggest you try it out. It doesn’t fill the role of a debugger in the “C++ sense” but it might in the “Python sense”.

[Gnimuc]

just found this on Twitter

[SurferJeff]

I use a debugger every day. It’s how I verify the code I write is behaving as I expect. I also use it to step through a new code base that I need to understand. I mostly build web applications in C#, C++, Python, and PHP, and use Powershell for scripts. I use debuggers in all languages. I’ve even simultaneously attached two debuggers to a single application written in two languages. The absence of a good debugger is why I won’t touch Ruby. The difficulty of using a debugger is why I don’t like Java.

In short, mastering the debugging tools has been the secret to my modest success. While another programmer has to recompile their application after adding a printf(), I can craft a quick command at the gdb prompt that behaves just like a printf and save myself minutes of recompilation time.

I’m checking out Julia, so I Googled “Julia debugger” and arrived at this page. Julia has so many other features I love, but I’m worried I won’t be able to understand the language without a debugger, preferably integrated into Visual Studio Code, but even a simple command line debugger would help a lot.

[Tamas_Papp]

GitHub - timholy/Rebugger.jl: An expression-level debugger for Julia with a...

[ExpandingMan]

I haven’t used it too much yet, but so far so good with Rebugger.jl.

It’s definitely possible to do all sorts of crazy things with Rebugger.jl + Revise.jl, so that’s fun. Some of the “trigger specific methods within existing packages to recompile” stuff that you can do in Julia really gets to be mind-blowing (e.g. it’s amazing that something like Rebugger is even possible since it doesn’t even work like a normal debugger, as far as I understand at least).

On a closely related note, type piracy might not be a good idea as a general practice but it can be damn awesome when you are just screwing around. The pirate’s life for me.

[Maxandre]

Hi,

Thanks for making Julia, a lot of features are really good, and I was about to switch from MATLAB to Julia for my PhD project, until I saw that there wasn’t any debugger available with breakpoints and stepping. For me this is a make-or-break feature to have, so I wish you good luck designing and implementing it! It could be a bit clearer that there isn’t any effective debugger available, it would have saved me some time today…

Best,
Max