[ANN] Rebugger: interactive debugging for Julia 0.7/1.0

Community
1

After several weeks of development, I’m pleased to officially announce Rebugger. While the package is still young and has obvious bugs, it’s already become an essential part of my Julia experience. So I decided it’s time to share.

Rebugger’s paradigm evolved considerably while I was developing it, and the “end result” (hah!) is interesting and feels different from anything I’ve encountered previously. With usage I’ve personally come to think that the differences are mostly improvements. To paraphrase from Rebugger’s fairly extensive documentation:

With most debuggers, you enter some special mode that lets the user “dive into the code,” but what you are allowed to do while in this special mode may be limited. In contrast, Rebugger brings the code along with its input arguments to the user, presenting them both for inspection, analysis, and editing in the (mostly) normal Julia interactive command line.

As a consequence, you can:

  • test different modifications to the code or arguments as many times as you want; you are never forced to exit “debug mode” and save your file
  • run the same chosen block of code repeatedly (perhaps trying out different ways of fixing a bug) without needing to repeat any of the “setup” work that might have been necessary to get to some deeply nested method in the original call stack. In other words, Rebugger brings “internal” methods to the surface.
  • run any desired command that helps you understand the nature of a bug.

For example, if you’ve already loaded MyFavoritePlottingPackage in your session, then when debugging you can (transiently) add Main.MyFavoritePlottingPackage.plot(x, y) as a line of the method-body you are currently analyzing, and you should see a plot of the requested variables.

Rebugger exploits the Julia REPL’s history capabilities to simulate the stacktrace-navigation features of graphical debuggers. Thus Rebugger offers a command-line experience that is more closely aligned with graphical debuggers than the traditional s, n, up, c commands of a console debugger.

Internally, Rebugger is a “low tech” debugger that is essentially orthogonal to more ambitious strategies like ASTInterpreter2. Indeed, once that package comes to full fruition Rebugger may switch to using it for its internal operations. In my estimation, the possibly-enduring contribution of Rebugger is the workflow and user interface, not its debugging internals. One of my personal hopes is that Rebugger will make it easier for Keno to continue work on ASTInterpreter2; it’s very hard to write a debugger, and doubly-hard to write one that can debug itself. To me, having Rebugger available to help fix and improve ASTInterpreter2 seems like it could be a good thing.

That said, because Rebugger’s debugging strategy is relatively simple, it’s fairly easy to extend. It already supports capturing stacktraces from calls that throw an error, and breakpoints would presumably be relatively straightforward. Somewhat to my own surprise, I’ve not yet started serious work on breakpoints because I’ve not yet found a real need to have them—the fact that you interact with “living code” means that breakpoints are basically something you do while you’re playing with the method. That said, if users find compelling examples that illustrate how useful they might be and offer a path towards figuring out what the interface should look like, please file a detailed description as an issue.

Enjoy! And as always, please report any problems. I expect quite a few, and I will try to fix them as time permits.

79 Likes
2

Just want to compliment you on the brilliant wordplay in the name (not to mention the package itself, which appears to be yet another can’t-live-without-it contribution). There is also a fourth meaning to the name though: rebugging is the process of creating new bugs for every old one you fix. I’m somewhat of an expert at this - it’s been an integrated part of my development workflow for as long as I can remember.

7 Likes
3

Looks fantastic! I’m very excited to try it out!

Note that currently when I do dev https://github.com/timholy/Rebugger.jl I always get

ERROR: Unsatisfiable requirements detected for package HeaderREPLs [0c95fe84]:
 HeaderREPLs [0c95fe84] log:
 ├─HeaderREPLs [0c95fe84] has no known versions!
 └─restricted to versions 0.2.0-0.2 by Rebugger [6bca5d8e] — no versions left
   └─Rebugger [6bca5d8e] log:
     ├─possible versions are: 0.0.0 or uninstalled
     └─Rebugger [6bca5d8e] is fixed to version 0.0.0

whether or not I have HeaderREPLs installed (at version 0.2.0).

Is this a bug with the package manager? I’ve seen this happen before and I wasn’t able to resolve without deleting JULIA_HOME.

(Can work around only by doing dev HeaderREPLs.)

4

Yes, I’ve become aware it’s currently uninstallable. Oops. I’m working on seeing if I can fix it. The most puzzling aspect is that despite having been registered in METADATA.jl it’s not in JuliaRegistries/General. But the problems seem to go deeper than that. Clearly I’m a Pkg noob.

5

HeaderREPLs is, but Rebugger probably hasn’t made it through yet. That shouldn’t matter, however, since you should still be able to install it with dev.

I’m increasingly suspicious that the error I showed is a Pkg bug. (Like I said, I’ve had this problem before.)

6

OK, I can now verify that the following works:

pkg> dev https://github.com/timholy/HeaderREPLs.jl.git https://github.com/timholy/Rebugger.jl.git

It does not yet work with add. I will see what I can do to fix that, but for now hopefully this gets people going.

6 Likes
7

Well, that was an embarrassing start. I think it’s fixed now if you do

pkg> up
# stuff...

pkg> add Rebugger
# should install correctly

Sorry for the inconvenience, and many thanks to @kristoffer.carlsson for all his help!

4 Likes
8

There’s also now a report that the default keybindings may not work for Mac users. Is anyone else experiencing this? If so, please comment at Bad keybindings for Macs? · Issue #28 · timholy/Rebugger.jl · GitHub with confirmation and hopefully a suggested replacement, and we’ll see if we can come up with some kind of consensus about a default that works.

I’ve now changed the key bindings 3 times hoping to find something that works for all platforms, and every choice has caused a problem for someone. We may be getting to the point of having different defaults for different platforms.

9

Any plans to incorporate this into Juno?

2 Likes
10

https://github.com/timholy/Rebugger.jl/issues/18

Help wanted!

5 Likes
11

Do I understand this right that Rebugger does not change the source files, only the code loaded in the julia session?

12

In R, I could put a browser() into any where I want. It gives me a REPL so that inspection, changing variable values, etc. are possible. It makes debugging very easy and convenient.

Ideally, if we could do it in Julia, it would be like:

function xxx()
    """
    codes that are 100% fine
    """

    browser() # REPL for inspection and making arbitrary adhoc changes

    """
    codes that may have problems
    """
end

Is it possible in Julia?

Right now I’m using ASTIntepreter2.@enter(), which allows me to step into a function. However, everytime I have to start at the beginning of the function rather in the middle, or being conditional…

any suggestions? thanks.

13

Having to change the code by adding calls like browse() inside it doesn’t seem that convenient to me.

The ideal solution will be being able to set breakpoints. And I think that’s coming, with ASTInterpreter. It’s just not finished.

14

One cannot eval into a local scope in Julia, thus a REPL of a local scope is not possible.

1 Like
15

Right, no changes to the source file. Of course since you can fix a bug right there at the rebug> prompt, I’ve contemplated making Ctrl-s mean “save change to the source file.” It’s a little scary, though, it requires quite a lot of faith in making sure the line numbering is all working correctly. So for now you have to copy/paste any edits you want to keep back into the source file.

If you only “step in” (Meta-e), then it doesn’t even make changes to any of the methods in your running session. (It defines some hidden methods, but does not alter anything that’s already been compiled.) Capturing a stacktrace, however, does alter existing methods.

16

Editing source files this way really seems a little scary :slight_smile: , not only because of the line numbers, if someone doesn’t know about the short cut they might change their source files without realizing it. Ctrl-s is the standard short cut for saving in many text editors and I guess that it is part of the muscle memory of many people.

Maybe you could open a text editor with a buffer that contains the edited file or create a tempfile .filename.jlto make it easier to apply the changes.

17

Regarding editing files while rebugging: That seems like something an IDE integration could very naturally provide. :slight_smile:

Will take a look a the Juno integration once I find the time.

4 Likes
18

Actually, I’ve come to suspect that if you’re sufficiently devious it may not quite be true. I haven’t tested so I can’t promise this will work, but here’s a sketch of how I’ve imagined implementing breakpoints. For a “method” with 2 interior breakpoints, Rebugger would put code on your command-line that looks something like this (if you’ve not looked, the docs might help explain some of this):

rebug> @eval SomeModule let (vars..., label_at_exit) = getstored(uuid) begin
          @goto $(label_at_exit)
          # somecode
          setstored!(uuid, varnames, varvals, :label1)
          throw(StopException())
          @label label1
          # code just after the first breakpoint
          setstored!(uuid, varnames, varvals, :label2)
          throw(StopException())
          @label label2
          # code just after the second breakpoint
      end

So if you enter this at the top, it will store program state just above label1 and then exit. (The StopException would be caught by Rebugger…or maybe that should just be a return nothing? Not sure.) Because it stored the label at which it exited, and loads from that storage at the top, if you just re-run this same command it will begin at label1 and continue on to label2.

I’m sure this will need some modification, but this is the general gist.

3 Likes
19

Yes, I agree, this sounds safer if an editor knows what you’re up to.

20

seems like it’s inconsistent with:

which one is true?