How do you use debuggers?

[avik]

Thanks everyone for the thoughts in here. It is very interesting to see the variety of use-cases, particularly since the wide appeal of Julia is one of the the charms of this community. The debate on the merits of each approach is less so, people should be free to use whatever workflow they are comfortable in.

For the benefit of the onlookers, I should probably clarify that we do have a debugger. It works reasonably well on 0.5. Setting a breakpoint is sometimes dodgy from Juno, but its better from the command line, and stepping and variable inspection is quite good. So it’s not as if people have not put in huge amounts of effort in the debugger, lets not diminish that work. It’s not perfect, but its there.

Yes, it is unfortunate that it does not yet work on 0.6. But you will appreciate that the debugger is deeply tied to the internals of the language, and so will need substantial changes given the evolution of the language. Also, it needs skills both in language internals and OS internals, so the pool of people willing to work on it is smaller. That work will happen eventually, and the fact that its somewhat delayed is no comment on the importance of the debugging workflow.

Regards

Avik

[joa-quim]

An IDE without a debugger is just a DE.

[kevbonham]

@juliohm it seems you misunderstand me, or you misunderstand the sort of work that many biologists do. If one wants to calculate the mean of 5 numbers, it is not necessary to provide the code that was used to make the calculation reproducible. One can simply say they took the sum of the numbers and divided it by 5. Sure, if no mean () function existed, it might be nice to provide that for the community, and many scientists are in the business of writing tools.

But a lot of data processing work is idiosyncratic, and spending the time to write a well-tested suit of functions that no one else will ever use is a waste of time. Better to describe the analysis and let someone implement it the way that makes sense to them. Further, by far the majority of the code I write is exploratory and is not intended for publication.

@juliohm

However, if you care about educating other people, delivering reproducible science outside of your lab that works with other’s datasets, then there is no escape from test suites and a good amount of assertions in your code.

My science shouldn’t be reproducible because of my code, it should be reproducible because I design well controlled experiments. Again, I’m not a computer scientist - I hope to god no one is trying to learn from the code I write. I know you don’t intend to be insulting, since you said as much to someone else, but when you imply (as you do here) that the work someone is doing is less valuable than other work, some people might take offense.

@juliohm

Users of your science aren’t computer scientists either and you have to ask yourself if you want them to 1) spend time debugging your code or 2) learn what the input is supposed to be from the assertions you wrote.

Actually, I don’t want them to do either. My science typically isn’t my code, that’s exactly the point. Most of my science doesn’t have “users.” To the extent my work has value, it has value in that I’m designing good experiments, executing them properly and thoroughly describing the results. People that care about the biology won’t bother looking at the code that analyzed the data, and people that care about the data are almost certainly better off working their own code to analyze it.

I did write a piece of software that’s intended for others to use, and that’s documented, tested, and I’ve got CI running on it. But that package accounts for maybe 5% of the coding That I’ve done or will likely do in the future.

[ChrisRackauckas]

@kevbonham

@juliohm it seems you misunderstand me, or you misunderstand the sort of work that many biologists do. If one wants to calculate the mean of 5 numbers, it is not necessary to provide the code that was used to make the calculation reproducible. One can simply say they took the sum of the numbers and divided it by 5. Sure, if no mean () function existed, it might be nice to provide that for the community, and many scientists are in the business of writing tools.

But a lot of data processing work is idiosyncratic, and spending the time to write a well-tested suit of functions that no one else will ever use is a waste of time. Better to describe the analysis and let someone implement it the way that makes sense to them.

How do I know you didn’t miss any details? Trust you, because you’re an authority in the subject? You already typed everything down into the REPL, so copy pasting that into a script and putting a test on the end to make sure it actually outputs the result you think it does takes less than a minute. I don’t understand why someone wouldn’t want to make it easy for someone else to run their exact computational analyses to make it easy to reproduce every figure with 1 button and enough computing resources.

(I really applaud Fernando Perez was mentioning about the new tools to accommodate reproducible science and make this easier, https://www.youtube.com/watch?v=DUdE3M2nlDE&t=1s . I hope this becomes required.)

Linking back to the original purpose of this thread, test suite are the best debugger because humans are infallible and get things wrong. Tests get corrected and refined over time. Other people check them, ask questions, and make sure try to re-understand them (maybe you in the future!). What you type into the REPL is something you did once, and sometimes your recall of it is wrong. We’ve all typed things into the REPL and been like “huh? Oh… I renamed …” or something silly like that, because we are all fallible. We all write bad code, and we all write code incorrectly. Tests are a clear statement of what you’re looking at, and it allows others (or you in the future) to double check no only that you’re looking at the right thing, but also that you’re looking for the right thing. Hell, one instance of this was already documented earlier today:

Bigfloat lufact

I pulled an example out. In this case, it looks like lufact! is the only one correct on BigFloat: A = 92.317*eye(4) b = [0.0970454, 0.00944241, 0.167562, 0.08518] println("True solution") println(b/92.317) lufact!(A) println("lufact") println(A\b) A = 92.317*eye(4) qrfact!(A) println("qrfact") println(A\b) A = 92.317*eye(4) svdfact!(A) println("svdfact") println(A\b) Abig = big.(A) bbig = big.(b) lufact!(Abig) println("lufact bigfloat") println(Float64.(Abig\bbig)) Abig = big.(A) qrfac…

What I was throwing into the REPL was wrong because I’m silly and fallible, but my tests still led me to the right spot and uncovered a Base bug.

The only reason to have complete trust in what you do interactively is if you never make mistakes, but if that’s the case, why would you need a debugger?

[DNF]

@ChrisRackauckas

test suite are the best debugger because humans are infallible and get things wrong.
[…]
The only reason to have complete trust in what you do interactively is if you never make mistakes, but if that’s the case, why would you need a debugger?

I’ll take your word that test suites are the best way to discover bugs. But as I have said, and some others too, debuggers are not just about catching bugs, but about understanding code. And I have yet to see any better way to inspect and understand a piece of code, than to watch it do its work step by step, to be able to drill down into any call as needed, and watch how it transforms your variables in ‘real-time’.

Indirectly, this understanding helps you fix bugs, but, to me, it is first of all a code inspection tool, and, for some at least, nothing aids understanding quite like watching–seeing–the thing at work.

[tamasgal]

This thread is already quite long but I can’t resist to add my two cents.

I personally have also something like 20 years of coding experience, learned dozens of languages (of course only a few of them really well) and what I can say is a basically a general statement about tools and their usability:

I use them [debuggers] when they feel comfortable/natural to use, i.e. are easily accessible.

Eg. when I worked with Visual Studio (C++) or Eclipse (Java) a few years ago on a daily basis (I was forced to have some adventures on Windows), I used the debugger literally all the time, because it was easy to point and click on lines to set breakpoints to see when a piece of code is called, step through the code via the F-keys, inspect and manipulate variables on the go etc.

However, I normally code in VIM inside tmux sessions and I still have not developed a fluid workflow for using gdb, pdb, jdb, or whatever. They just feel awkward to me and are most of the time outperformed by very basic debugging techniques like printing or isolating in REPL/notebooks/etc.

For me, debugging simply depends on the editor/IDE I use. I also gave WebStorm, PyCharm, and Atom for Julia a try and loved their debugging features (including Gallium), however I can’t live without my VIM/tmux workflow, so all those IDEs are no replacements for me. I occasionally fire them up if I really need an actual debugger and don’t want to mess around with *db in the terminal.

Also having a good unit test coverage makes debugging a bit more obsolete imo.

[lobingera]

@avik

we do have a debugger. It works reasonably well on 0.5.

I’m currently trying to debug a few things and my experience using Gallium is somehow unsatifying, as in 2 of 2 cases (different platforms) Gallium isn’t even precompiling or installing. Overall these years i’m trying to use Gallium it seriously looks like a prototype / an experiment. Still there are two JuliaCon Videos (2015+2016) in which the primary author goes into lengths what you can do with it.

I understand that some people use it, i can’t (even in a v0.5 release)

Another thought. One of the usecases of a debugger is to understand other’s code and the interaction of code/packages which has quite an emphasis at this release to release updates of packages (or own code).

[oatlzzvztd]

@tamasgal

Also having a good unit test coverage makes debugging a bit more obsolete imo.

Sure, and this opinion is expressed (far more condescendingly and dismissively) above. A good debugger works very well, in my humble opinion, to bridge the gap between today’s test suite and tomorrow’s. Yes, even as an adjunct to proper test-driven development!

Printf debugging is rightfully considered harmful, and I avoid it wherever possible. The Julia REPL doesn’t do us any favors here as it lacks an effective module reload, ClobberingReload.jl notwithstanding.

[oatlzzvztd]

@lobingera

I’m currently trying to debug a few things and my experience using Gallium is somehow unsatifying, as in 2 of 2 cases (different platforms) Gallium isn’t even precompiling or installing.

Seconded. My experience with Gallium is that it works without complaint about 30% of the time. Half of the time it segfaults, and the other 20% of the time it hangs because the code it’s being asked to introspect is too complicated for it to analyze.

[lobingera]

Note: I’m not complaining about the performance of Gallium, as i’m aware how complicated its job is, marketing it as a working solution is (imho) far fetched…

[kevbonham]

I don’t want to belabor the point, but I actually do care really deeply about open and reproducible science. I try to write all of my code and version-control it in public repos, even though it’s mostly pretty embarrassing because I’m such a noob. If I’m really in the wrong, I want to be set right. Maybe it should be pulled out into a different thread? But for now:

@ChrisRackauckas

You already typed everything down into the REPL, so copy pasting that into a script and putting a test on the end to make sure it actually outputs the result you think it does takes less than a minute. I don’t understand why someone wouldn’t want to make it easy for someone else to run their exact computational analyses to make it easy to reproduce every figure with 1 button and enough computing resources

Sure enough - by the time I have everything ready for publication, my code is tidy and ordered, and it surely makes sense to make it into a script. Or in a jupyter notebook/executable markdown etc. OK, maybe I should put in a unit test then for any functions I have written, but for eg this work, there aren’t actually many functions/novel analysis, it’s just applying methods that exist, and the notebook documents how.

And in any case, my point is that this is the last 5% of my work at most. Up until I get to the point where I’m publishing/describing my work, I’m trying different things out, most of which don’t work, or don’t display the output in the way I want, etc etc. This code will not be used by others, and my results are not dependent on all the fuck-ups I make along the way. I can’t write tests for these ahead of time because I don’t know what I’m doing ahead of time. It’s like asking me to write the paper before I’ve done the experiments.

And what about my glue scripts that just run different analyses that were written by others? Or the ones for personal use that takes all the files from a collaborator of mine and renames them in a way that works better for myself? Doing all of these things better/faster would benefit from a debugger. I think.

[sambitdash]

Hi All,

I am a novice Julia developer. I have 2 projects actively been developed :

1. A PDF parser
2. Sample code solution for AI a modern approach

I definitely need a debugger for 1. Without single stepping debugging a parser is almost impossible. In a handwritten parser with direct and indirect recursions after a few tokens it becomes very hard to track progress without a debugger.

Sample code example is a very scientific computation kind of problem. I can write special testcases for misuse cases and ensure the algorithm is working as intended.

A single stepping debugger should be given priority for native Julia ASAP. It can save lots of time and efforts.

regards,

Sambit

[ExpandingMan]

@tamasgal

I use them [debuggers] when they feel comfortable/natural to use, i.e. are easily accessible.

While I agree that debuggers are not absolutely necessary, they certainly can be quite useful. I spent about 8 years coding mainly in C++ (and some Fortran) and then the last two years coding mainly in Julia (and some Python). My first experience with debuggers was with Visual Studio and I have to say that setup was really amazing (it says really bad things about Microsoft that I nevertheless find that not to be worth it). Even so, back then I was using the debugger for its intended purpose: to debug (this was C++ mind you). Therefore, debugging didn’t seem like much more than a convenient curiosity to me at the time.

When I started using Python I made much more extensive use of the debugger. I dare say that it seems likely that the ease and power of debuggers in interpreted languages is a major reason for the success of Python. In Python I used the debugger as a user interface: I would write command line tools that launched me into the Python debugger. While other people were using excel to view data, I was using my command datacat which quickly loaded a dataset into a pandas dataframe. I would write scripts that were part interactive (via the debugger), part static. Granted, much of this was to overcome what I found to be the shortcomings of Python, but me the debugger eliminated many uses of GUI’s. I also became somewhat accustomed to reasoning about code by inserting lots of break points, going in and messing around, and seeing what result I got. When I was using it, it still seemed like a fun new toy, so I have to imagine that as time passed I would have used it much less.

I have said many times before on this forum that the debugger is the one and only thing I miss about Python. Granted, Julia has a much better REPL than Python, and it also has pretty much the best stack traces I’ve ever seen (though sometimes tracebacks through macros are still ugly) so I think the situation we have right now is just about as good as it could be without a debugger. I still think we should have a really good one though.

[Tamas_Papp]

Minor but kind of related improvement in 0.6: after a stack trace, number + Ctrl-Q in the REPL takes you to the right function. Together with Revise.jl, and #265 fixed, I experienced large increases in productivity when debugging (so now I generate more subtle and insidious bugs).

[sambitdash]

Is there any timelines when the debugger will support 0.6 or 0.7?

As much we think we are 100% sure of the system we build there is one thing or the other in the wild that proves our limitation of knowledge. So debugger definitely helps than introducing debug prints all over the place and then remove them.

Please consider this as a high priority requirement.

regards,

Sambit

[DNF]

I have seen some comments to the effect that work on updating the debugger is ‘ongoing’, but it’s not so easy to see what’s going on. https://github.com/Keno/Gallium.jl doesn’t show any activity, though sometimes people work on ‘private branches’.

My uneducated, ill-informed guess is that the update may not come until after version 1.0 is out, since there is a tremendous amount of other work that is getting higher priority, breaking changes, etc. I don’t know this, but I’m getting that sense from lack of ‘chatter’, no mention of timelines, and that the most active contributors don’t seem to need a debugger that desperately.

[mzaffalon]

To back up on @DNF reply, see also this comment.

[Keno]

The work on the new version of ASTInterpreter is in GitHub - JuliaDebug/ASTInterpreter2.jl: Re-write of ASTInterpreter for julia 0.6+. It mostly works, but there’s a few things to take care of before I formally announce it. It doesn’t have breakingpointing, which is a pretty difficult feature to implement well and performantly, so that may have to wait until after 1.0, but at least the exploratory workflow should be well supported.

[chobbes]

In the last 7 months, I’ve been working on a project where Julia is not involved. Now I’m back to Julia, realizing that a lot has happened. At the time when I left, v0.6 was just rolled out. Now it is fully-fledged and bundled with JuliaPro, which is really nice.

What I care most is the debugger since I often need to understand large packages written by others (more than 30,000 lines of code per package on average). Stepping through using debugger is the only efficient way to approach for a quick digest, to be honest. I was excited when I checked the downloads page for JuliaPro, where my eyeballs were caught be this line:

It includes the Juno IDE, the Gallium debugger, and a number of packages for plotting, optimization, machine learning, databases and much more (requires registration).

I spent the whole day yesterday playing with JuliaPro and didn’t see any trace of the debugger functionality being in effect. My reading of some of the posts in this thread seems to have confirmed my understanding - debugger is not ready for v6.0, unfortunately.

Here is my question - If I connect Juno (that I obtained via installation of JuliaPro) with v5.0+, am I able to use debugger within Juno?

Two more questions - When will a mature debugger be in place for v0.6? May I have a rough estimate? I’m really curious to know. Are we going to have something close to the Matlab debugger - you set breakpoints by simply clicking the corresponding line?

Anyway, big thank you to Julia team and all contributors!

[deltakam]

If you are a developer and your daily life is coding and writing robust codes to be reused, yes you might really need to have a very good mental model of what is going on, and debugger is not needed.

TL;DR: as a scientific researcher who writes code just sometimes and throws them away in most cases, debugger is a huge time saver.

The use of debugger is of course to get fix it quickly without maintaining a good mental model of what is going on. As a researcher, I spend 90% of time on something else, but spend only 5% of my time coding. This involves testing my algorithmic idea, which ends up not interesting most times, and I just want to find out ‘early failure’. In this case, I just want to load my shitty data quickly (quickly in coding time, not run time), write the code that just works for the core case, and when I get an error / unexpected behavior, I just want to quickily set a breakpoint and starting printing values.

Sometimes, starting a script even involves loading my data and processing it and takes more than 10 seconds, and so on. In a long run when my shitty algorithm seems to work and I want to run a serious experiment, yes I should process the data well, save the processed data somewhere, optimize code, and so on. But, why would I want to spend my time on these things when my new algorithm is not interesting 90% of the time?

I also want to emphasize that scientific computing requires more need for the debugger since there are numerical issues you need to carefully examine but hard to expect ahead of time. I often find negative eigen values in (supposedly) positive semi-definite matrices, or the optimization method just does not find the right solution and so on. The best thing about using python here is that I don’t even need to setup a breakpoint ahead of time. When it fails, I just call the post-mortem analysis function, and it gets me to the place where I can examine eigen values and check optimality conditions and so on… And these are often rare cases that happens once every 1000 random runs… I’ve been there.

Anyway, I really would like to emphasize that having a debugger is really important in attracting many users into Julia. No matter how you argue well, they will still want a debugger.

I really like where Julia is going. But as I learn more about it, Julia is more like providing convenience for people who write library, but not that much for people who use it “sometimes” to check things quickly and produce code just to write a paper. In this use case, you mostly write scripts that won’t be reused.

Or, it could be that I just belong to a community that Julia does not need to care…