Understanding a Julia codebase, or, How do you navigate inter-package specializations?

I have a specific question, but I think it’s an instance of a general question. The general question is: suppose some funny behavior is happening when you use a library written in Julia, and that it’s not obvious where the behavior comes from. How do you, personally, figure out what is making that funny thing happen?

The specific example here is: suppose that when you use Plots.jl to plot data against a column of ZonedDateTime UTC timestamps, the x axis is given the suffix “(Z)”, no matter what you set the xlabel to. How would you go about finding the code responsible for the “(Z)” suffix, and how would go about figuring out what to do? Again, I’m not so much interested in this specific bump in the road as I am in learning how to deal with things like this in general.

When you suspect some unwanted behavior in your code might be coming from the interaction of specializations possibly defined across several different modules and you can’t figure out what is actually being invoked much less where it was defined, what do you do? What techniques do you use to navigate through the (beautiful!) space of open multimethods defined across packages that make up the Julia ecosystem? I’m comfortable with this kind of bug-oriented navigation through codebases written in other languages, but navigating Julia code is different. How do you go about it?

@edit f(a,b)
#or
@which f(a,b)

Sometimes the chain following from there is hard to track, particularly when a macro is involved. But that’s a start.

5 Likes

Thanks for the pointer to those tools!

Do you know of any similar tools that can look deeper into the transitive call graph of an expression? In the case of my Plots.jl mystery, I did manage to find relevant code in UnitfulExt.jl but I had to use grep to do it. It seems like it should be possible (overwhelming, maybe, but possible) to automatically trace downward from a top-level expression like plot(...) to internal specializations…

If anyone is willing to describe their workflow for this kind of debugging and navigation in more detail (or, if there’s a guide to Julia codebase navigation elsewhere, to point me toward it) I’d really appreciate that! This kind of hands-on knowledge often goes unwritten, so I think it could be very helpful for intermediate Julia users to see it recorded.

1 Like

In many cases, you can use Cthulhu.jl.

5 Likes

Cthulhu’s @descend or VSCode’s @enter (or Debugger.jl’s) are usually the way to go with this in cases where @edit is not enough to find the cause.

4 Likes

That was also one of the motivations for my TraceFuns.jl.
In your example of Plots, it’s probably not of much help though, unless you already know which functions (or at least modules) could be involved.

2 Likes

Thanks for the tips, everyone. I spent some time spelunking with Cthulhu.jl… it’s exactly what I was imagining, powerful navigation and overwhelming amount of information and all :slight_smile:. While Cthulhu is brilliant, the style of Plots’s recipe dispatch code seems to be dynamic enough that Cthulhu can’t locate the specific recipe associated with the type of a given series. You get to the point where it is looking up recipes, but you can follow that no further. I’ll keep trying to see if Cthulhu could be useful for this sort of detective work but in this case at lease it seems to run aground on sufficiently dynamic dispatch.

I did manage to figure out that ZonedDateTime.jl defines its own Plots.jl recipes and that this is the source of the text I’m seeing rather than the UnitfulExt functionality inside the Plots package itself. I was only able to figure this out by grepping through my package’s source cache for the relevant undesired output string. Honestly I still don’t understand how the code in question gets invoked or how to fix by problem, but I’ll chalk that up to ignorance of the recipe facility of Plots.jl.

Again, thanks for helping me out. Please feel free to continue offering advice or battle stories or your own accounts of how to figure stuff like this out. If I come up with a good workflow I’ll write it up here.

1 Like