Investigating the bottleneck in a parallel program preventing it from scaling linearly like its C++ counterpart. The program includes I/O and uses CodeZlib (which has ccall). Trying to see how threads are spending time.
Maybe this is already a profiling package?
I don’t know of a Gantt chart library for plain multithreaded Julia code; that would require a thread-safe timing library, similar to TimerOutputs (which itself is not thread-safe), so that you can collect the relevant events into different groups for display.
I would like to point out that Dagger.jl has a built-in Gantt chart, which I’m in the process of overhauling to be more efficient (the PR will be available in the next few days). If you’re interested in porting your code to Dagger, I could help you setup Dagger’s Gantt chart and expose custom events to it.
is time() not thread-safe? someone tried some hacky stuff: https://github.com/aminnj/ThreadGantt.jl/blob/678093b4587408303020b36b4fccc044e7d5b280/src/main.jl#L3 and seems got qualitatively useful visualization
Sure, @time is technically thread-safe (all it’s doing is making two system calls and doing a subtraction to get the time), but collecting all @time-d information from every thread requires considerations to be realistically thread-safe. I can’t tell if the ThreadGantt.jl package is data race-free, so I can’t make any comments on whether using it will work for you.
Here’s something I played with a bit: Gallery · EventTracker
If you want conventional profiling, you can use the stdlib profiler in the nightly build. Better threading support was added a few days ago: https://github.com/JuliaLang/julia/pull/41742