How do you use debuggers?

Community
61

They developers do care about people who want debuggers, and it is said to be in the works. The constraint is developer time, not perceived/assumed lack of need for a debugger. See

4 Likes
62

I see. Thanks for the clarification.

I checked the Gallium repo often and didn’t see much updates and thought no development going on. I was wrong.

Still hoping to get more attention/effort on debugger, and many thanks to those who contribute!

63

I’m reminded of Rust. Every now and again someone who doesn’t understand what kinds of problems I work on says, “You should use Rust! You can guarantee that your code won’t have X, Y, or Z errors.” The problem is that, in my code, failures arise from the complex interactions of the algorithms with an ever more complex reality. My code doesn’t fail because it divides by zero or because an integer overflows. It fails because making a plane fly through a hurricane is a hard, open-ended problem, and it takes a lot of investigation to see what’s going on in any given algorithm. You can unit test the components, but that doesn’t (and can’t) tell you how the complex system responds. (E.g., you can unit test an algorithm to move a piece on a Go board, but that doesn’t tell you if your “move” algorithm will win when up against any given opponent.) Problems like these are far, far easier to investigate with a good debugger.

My favorite debugger is MATLAB’s. I think it’s one of that platform’s major strengths and shouldn’t be dismissed lightly. I find it useful for designing algorithms because I can, e.g., go into a function after it’s been operating on its state with tons of inputs for a while, look around, maybe run imagesc(big_matrix_1), and take a look at what’s going on. Such a workflow is critical for many people.

I would argue that “debugger” is really the wrong name for such a tool. “Inspector” might be closer to the mark, but debuggers can also be interactive (e.g., in MATLAB, you can call nested functions, change state, etc.). Maybe “investigator”?

Reasoning through code is unquestionably the first thing I’d expect a good engineer or scientist to do. Unit tests help too, but can’t tell you everything about the interaction of components (even when those components are perfectly understood) when part of a complex system (Go, Game of Life, etc.). Beyond that, a debugger is an excellent tool for designing algorithms where the answers are open-ended.

Maybe I should go donate some money earmarked for Gallium!

18 Likes
64

ASTInterpreter2.jl basically does all that, and it also seemed to work pretty well in the couple of hours I toyed around with it.
I do intend to spend some time working on integrating it into Juno so you’ll get the same experience you had with 0.5 (minus breakpoints, at least until Keno or someone else updates Gallium).

4 Likes
65

Very cool! I’ll definitely check out ASTInterpreter2.jl in time. For now, my work in Julia is in fact “just programming” and has nothing to do with the interaction of complex systems. :slight_smile:

(Just to be clear, I did not mean to compare MATLAB’s debugger to any effort under way for Julia; I was just answering the topic’s question about how and why I use a debugger.)

66

EDIT: @DNF has already answered (thanks) all of my question below. Only “the” may have been unclear: I suppose you @deltakam define your function: “the post-mortem analysis function”.

It seems you want a “watchpoint”, not just a breakpoint, for an arbitrary function of values (just not one). Is that possible (even with non-GUI one)? It may be what @tuckermcclure is referring to in MATLAB, just not sure if the “Inspector” is only to run such functions interactively after a breakpoint.

[Such watchpoints, should maybe rather be in your code as “asserts” (or “contracts”), at least be migrated there; and you seem even be checking for stuff that should never happen(?)]

Of course, the basic functionality should come first, and I believe it’s available, just not in GUI form (even with the docs saying otherwise… [for Juno]).

[I’m not up to speed on [Julia] debuggers, I believe that wasn’t possible back when I used one [eventually]… When I was first studying/programming, debuggers (and bounds checkers) weren’t even taught, as too pragmatic I guess, just told about, after submitting code assignment… (bounds checkers cost money back then, and the teacher had one, and could have told us of these concepts/tools beforehand…).]

@tuckermcclure “Maybe I should go donate6 some money earmarked for Gallium!”

Thanks for the tip, I just did (I kind of allowed to spend money for other uses, do not want to slow down Julia 1.0 out, yes both are important and if manpower for both, then the debugger is also important.

[I got “success” on second try, I guess I can trust it to be true: I believe I “changed” to the exact same info… I.e. nothing. Not sure why [Icelandic] Postal code needed… or useful, should it be a required field? I first thought non-US ZIP code the problem when I got:

“This transaction has failed. Please try another payment method. If this problem persists please contact the organization or your financial institution to complete the transaction. Thank you!”]

67

Both Matlab and Python debuggers have these ‘conditional breakpoints’. If I’m not misremembering, Gallium had them too.

Yes, eventually they should. But during development, you may not understand why those situations cause errors, or how they occur in the first place. Being able to set conditional breakpoints helps you track down the how and the why.

At least in Matlab, you can set the debugger to ‘catch’ an error and launch you into the debugger whenever and wherever an exception is raised. Perhaps that’s what is meant.

1 Like
68

This isn’t quite what you want, but you might still find TraceCalls.jl useful for this kind of scenario. For instance,

using TraceCalls
trace = @trace MyModule ...your code...
filter_lineage(tr->any(obj isa Matrix && !isposdef(obj) for obj in objects_in(tr)), trace)

This will highlight all function calls with any argument or return value that is a non-positive-definite matrix.

3 Likes
69

@Palli The post-mortem analysis function allows you to debug without setting up a breakpoint ahead of time. For its functionality, please search “pm()” form
26.2. pdb — The Python Debugger — Python 2.7.18 documentation and then find the example; this will explain what it does better.

By the way, I am not sure if this pm() is technically trivial to implement, and I am mostly fine without it. My main concern is just a robust debugger that lets me set breakpoints and allowing me to go up and down in the stack.

@Palli On the “watchpoint”: I don’t want to check the eigenvalues every time I do something with a matrix with assert statements as it will slow down a lot. This is another thing with the numerical computing; checking conditions often take nontrivial time.

Again, I want to emphasize my usage of Julia to see “early failures” of my ideas quickly; it’s not like I have a solid method that I understand well and I just need to implement it. Inserting assert at every suspicious place takes time. Running it with time-consuming asserts also takes time. In this context, I just write codes that just work for the main case, not the edge cases. When it hits edge cases, it will fail, and I need to quickly see what is going on. The best way is pm() function. If not, I can set a break point at a suspicious place, and trace back where things have gone wrong. What’s more, It may not be the place where the error occurs as numerical errors usually get propagated until serious error happens; e.g., by the time it spits out an error, matrix entries are like NaNs and Infs and so on, but the root cause is usually somewhere else.

Thanks @DNF for providing answers!

1 Like
70

Sorry for the long post, but I wanted to give a complete answer to How do you use debuggers?

I think this post by @tuckermcclure really captures the confusion that people have when talking about the “debugger”. Package developers from a linux environment may be used to seeing about GDB, Intel Parallel Debugger, and other “true” debuggers. In general, these are awful and used only in peculiar situations. When writing a package you tend to design the unit tests, etc. early and rarely need any “debugger” unless there is a crash you can’t figure out. Package developers coming from a windows environment may be used to the superb Visual Studio C++ “debugger”, but will also appreciate that they use for something fundamentally different than they would use GDB. Forget entirely about the use cases of package developers, who don’t really need and frequently don’t even want debuggers.

Now, imagine you are a library user working on a new algorithm or doing some exploratory data work. Even if you know about unit tests (they generally don’t), you are writing a small script using a large number of libraries, and throw away most of the code in the end.

Since you are a scientist (or social scientist) you are doing something completely new and : (1)are not sure if you have thought the right algorithm correctly; (2) not even sure what they output should be because it is exploratory; (3) don’t know exactly what the libraries you are using return; and (4) want to step through the algorithm to understand it better and see intermediate values as they are computed. This is the “inspector” usage pattern that @tuckermcclure is talking about What you may do is write the code and only step through it is the result looks “suspicious” - so it is a bug in some sense - but it may be that you thought through the algorithm or even the science wrong and it has very little to do with “debugging” in the sense of a package developer

Almost all of the activities of people using the “inspector” usage pattern are simple, so Gallium/Juno doesn’t need to add in fancy features or things focused on true debugging (e.g. stacktrace connections on an assert, etc.). So to answer “How do you use debuggers?”, the following is essentially it:

  • Set a breakpoint to stop execution on a line (with a mouse!). I don’t think that conditional breakpoints are really necessary in the first iteration, as they tend to be more useful for true debugging than inspecting. Few matlab users know about them.
  • Step to the next line, step into a function, or just continue the script (with F keys)
  • Easily Inspect the value of all of the variables in scope at the breakpoint. At first it is totally reasonable to have them type display(XXX) or whatever into the REPL, but also nice to have mouse hovering and/or a window in the IDE with variables you are watching.
  • Run small bits of code in the REPL given the state: For example:
    • Might be worried the matrix A might be singular, so run det(A). See if a vector has any NaN in it…
    • Change a value in the REPL to see what would happen to the algorithm. For example, you worry that the algorithm works for a certain value, and want to see if it fails for a larger one. Or you realize you calculated something wrong previously, and want to change the value to see if the rest of the code works.
    • Run a single-line piece of code to see if you get a failure or not. This is great because in that code can use all variables in the local scope at the breakpoint, and you can then copy/paste it into the script itself.
    • Crucial that the program you are inspecting doesn’t abort just because you type something invalid into the REPL.
    • If you have any room for other features, then having a window in the IDE which shows current values of watched variables in it is the next major feature to add.

This really is it. Keep thinking of a subset of the Matlab or Visual Studio approach, and as far away in spirit from that godawful mess GDB as possible. Minimal features, maximal connection to the IDE and embedded REPL.

In fact, I see no reason that it shouldn’t be directly coupled to Juno(of course, the ASTInterpreter could be used in other IDEs). The linux Vim crew and package developers don’t really need a debugger at all, and if they did it would have a different set of features. If you made Gallium only accessible from setting breakpoints and using buttons in Juno, none of its users would shed a tear.

14 Likes
71

I switched from Windows to Linux 3 to 4 years ago. I do not regret it. BUT, the ONE thing I miss the most, is debugging a program in Visual Studio. The experience is very satisfying. I could repair bugs faster than I could produce them!

If we ever get a debugger experience like that with Julia in Linux … wow.

1 Like
72

You can get around that with a fast function like rand (or some other deterministic):

if rand(1:100) == 1 && slow_real_check() … # line runs every time but really does the slow check less often since && short-circuits.

74

You can get around that with a fast function like rand (or some other deterministic):
if rand(1:100) == 1 && slow_real_check() … # line runs every time but really does the slow check less often since && short-circuits.

Yes, that would do an efficient check. However, it will catch something that happens frequently enough only, by definition…

75

People always speak of Visual Studio, but also qtcreator c++ debugger is fabulous (gdb), with the possibility to inspect the state of variables on each scope of the call stack, or set conditional breakpoints, even by pass count…

While having the possibility of running code interactivelly reduces somehow in my opinion the need for debugging, still for large models this would be very appreciated…

1 Like
76

Sneak preview for interactive stepping in Juno (leveraging ASTInterpreter2.jl): https://i.imgur.com/gCWT0aD.gif (1.6MB)

Would be great to know what you folks think about that; specifically, what you think is missing (except for breakpoints obviously).

22 Likes
77

I love it.

I’m particularly happy that there is a fixed debug menu, not just the floating one that follows the current line. It was quite frustrating to hunt for that all the time (and sometimes it fell off the screen.)

Some questions:

  • I presume that you can assign keyboard shortcuts as well?
  • Do you have to debug in a special ‘debug console’ (like in Python), or can you launch the debugger inside the current console, keeping the current state of the workspace (like in Matlab)?
  • When will rr debugging be supported? (Just joking.)

Missing:

  • Breakpoints (of course)
  • Conditional breakpoints
  • Launch debugger on raised exception
  • Launch debugger on captured exception
2 Likes
78

I presume that you can assign keyboard shortcuts as well?

Sure. For now I’ve gone with Matlab’s F10, F11, and Shift-F11, but you can of course customize those.

Do you have to debug in a special ‘debug console’ (like in Python), or can you launch the debugger inside the current console, keeping the current state of the workspace (like in Matlab)?

Debugging works in the current console, so no, no special debug console needed.

Breakpoints (of course)
Conditional breakpoints
Launch debugger on raised exception
Launch debugger on captured exception

Those are all tied together and will have to wait for Gallium2.jl (or whatever it’ll be called).

2 Likes
79

Can we get out of a for loop once inside it?

80

No. A Run to n-th line command would be convenient though.

4 Likes
81

Dear juliohm
You asked “how do you use debuggers?” -
Although the thread has long been abandoned I still feel the need to add my 50 cents - maybe it will help some understand how a breakpoint is a massive time and effort saver.

I run my program - it crashed somewhere.

If I have a breakpoint - I rerun (mostly a simple rerun is easy enough) and stop just before the crash. In most cases it is easy to see what has crashed and why it has crashed. In some cases I go a level up in the stack if some variables are not as expected and the bug is easy enough to trace.
Sometimes I need another rerun(s) with another breakpoint(s) to trace the issue further back.
No recompilation / reloading.

In comparison, without a breakpoint - I need to add printfs / write to files / other output that is not always easy to get / breaks timing / other issue that prevented me from having good debug data at that stage to begin with.
Let’s say I do manage to get the debug data from the outputs. The problem could have happened long before the actual crash, but the code was “safe enough” and the issue just propagated onwards and onwards until the code was no longer safe. So I need to backtrace the issue - adding more and more printfs / files and so on. Each time I add debug data here, I need to possibly recompile or reload stuff, or rerun something and in some cases a single setup is very tricky.
Eventually I need to remove all I have added, retest the code and timings and so on.
And that was for a single bug.
Multiply it by 10 or more, and you know why a breakpoint is essential.

Consider how bad things get when stress-testing the project / long simulations.

A good debugger and breakpoint worth their code in gold

8 Likes