Precompilation and tracking dependencies

New to Julia
,
1

I’m somehow bouncing back on this post about the time it takes for precompilation, and I agree with @mattsignorelli that it can scare new users about the practical performance of Julia. I’ve also been frustrated a lot because I sometimes had to restart Julia after a crash and everything had to recompile, without really understanding why it should recompile.

However, my frustration is not really that it takes time, but rather that it is not easy to understand what’s going on. Is there a way to get the tree of all dependencies? Maybe if there was another representation during precompilation of what is happening, or some output file that explains everything and why there was a new precompilation in a very user-friendly way (I’m thinking for example of Obsidian’s graph representation for taking notes), it would create more of a “Wow, Julia handles packages very well” effect.

Moreover, I always struggle to find the “method dispatch tree-chain” (I don’t know if it has a name) that is being called when I call a function with specific types. For example, @which sum(a) gives the implementation that dispatches to mapreduce, then @which mapreduce gives other methods and so on. It’s already hard to get to the final method from Base, and I’m speaking only of the sum function! There are many extensions in Julia and it’s very easy to get completely lost in comparison to other languages.Of course I don’t want to know that + or basic operators are called at each step, but at least I would like to know which “non-basic” methods are called.

3 Likes
2

I can’t possibly say, but I sometimes trigger precompilation in a restarted session because I altered or switched environments a lot in the prior sessions, which is exacerbated if environments were stuck with an already loaded package of an “unexpected” version. Julia’s precompile cache holds 10 versions of each package by default, so it’s not easy to switch versions often enough to outpace the cache.

PkgDependency.tree(packagename::String) does that job, but the entire tree is often cumbersomely large and probably most unnecessary for re-precompilation. We’d only need the dependencies that need to be recompiled as well as their dependents, don’t think subtree is the right word.

Cthulhu.jl lets us interactively step into callees’ methods, if statically known. Profiling-based flame graphs also represent trees but for other purposes. I don’t know of a way to just print a tree of methods, that might also just have been too big to be useful.