Profile segfault

jstrube · August 15, 2018, 5:17am

I’m porting some particle physics code (Fox Wolfram moments) to Julia.
The first attempt had pretty awful performance, so I’m trying to figure out the bottle necks.
Unfortunately, profile barfs (trace below). This code is using an external library to read a 700 MB file, so a minimal working example would take me a bit to put together, but if you have seen this before, or can figure out if I’m doing something wrong from the trace, please let me know.
All I did was to try running @profile on my function (which iterates multiple times over a collection defined in C++).

Trace:

signal (11): Segmentation fault
in expression starting at no file:0
sig_match_simple at /buildworker/worker/package_linux64/build/src/typemap.c:125 [inlined]
jl_typemap_entry_assoc_exact at /buildworker/worker/package_linux64/build/src/typemap.c:780
jl_typemap_assoc_exact at /buildworker/worker/package_linux64/build/src/julia_internal.h:883 [inlined]
jl_typemap_level_assoc_exact at /buildworker/worker/package_linux64/build/src/typemap.c:833
jl_typemap_assoc_exact at /buildworker/worker/package_linux64/build/src/julia_internal.h:886 [inlined]
jl_typemap_level_assoc_exact at /buildworker/worker/package_linux64/build/src/typemap.c:833
jl_typemap_assoc_exact at /buildworker/worker/package_linux64/build/src/julia_internal.h:886 [inlined]
jl_lookup_generic_ at /buildworker/worker/package_linux64/build/src/gf.c:2133 [inlined]
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2179
lookup at ./stacktraces.jl:114
lookup at ./stacktraces.jl:119 [inlined]
#6 at ./none:0
iterate at ./generator.jl:47 [inlined]
Type at ./dict.jl:104
print at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Profile/src/Profile.jl:182
jl_fptr_trampoline at /buildworker/worker/package_linux64/build/src/gf.c:1829
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2182
do_call at /buildworker/worker/package_linux64/build/src/interpreter.c:324
eval_value at /buildworker/worker/package_linux64/build/src/interpreter.c:428
eval_stmt_value at /buildworker/worker/package_linux64/build/src/interpreter.c:363 [inlined]
eval_body at /buildworker/worker/package_linux64/build/src/interpreter.c:686
jl_interpret_toplevel_thunk_callback at /buildworker/worker/package_linux64/build/src/interpreter.c:799
unknown function (ip: 0xfffffffffffffffe)
unknown function (ip: 0x7fd13bcb798f)
unknown function (ip: (nil))
jl_interpret_toplevel_thunk at /buildworker/worker/package_linux64/build/src/interpreter.c:808
jl_toplevel_eval_flex at /buildworker/worker/package_linux64/build/src/toplevel.c:787
jl_toplevel_eval_in at /buildworker/worker/package_linux64/build/src/builtins.c:622
eval at ./boot.jl:319
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2182
eval_user_input at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/REPL/src/REPL.jl:85
macro expansion at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/REPL/src/REPL.jl:117 [inlined]
#28 at ./task.jl:259
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2182
jl_apply at /buildworker/worker/package_linux64/build/src/julia.h:1536 [inlined]
start_task at /buildworker/worker/package_linux64/build/src/task.c:268
unknown function (ip: 0xffffffffffffffff)
Allocations: 30864501 (Pool: 30857753; Big: 6748); GC: 66
Segmentation fault (core dumped)

tobias.knopp · August 15, 2018, 5:50am

Then you should file an issue in the Julia Github repo. Maybe this is https://github.com/JuliaLang/julia/issues/28648.

tk3369 · September 16, 2018, 1:12am

It is quite annoying that we cannot use the profiler. Does anyone have a workaround?

Ralph_Smith · September 16, 2018, 3:05am

For small problems, you can reduce the pointer count (as suggested in one of the issue discussions):

Profile.init(n=10^5)

This has worked for me in a few cases.

tk3369 · September 16, 2018, 4:22pm

Yes, I’ve tried that but the result is somewhat unpredictable and only works in small test cases. By small, I mean anything that takes less than a second to run would be fine but longer always seg faults. And, even if I make it work with a small test case, it may still seg faults after the 2nd or 3rd run.

bennedich · September 16, 2018, 9:12pm

I get the same thing. The workarounds I know of:

limit number of instruction pointers with the n argument
increase the delay between backtraces
profile in Julia 0.6 if you can
profile shorter functions instead of your whole script
use @time in an iterative way, by timing sections of code, or removing parts of code

Unfortunately, I’ve found that even when the profiler works in Julia 0.7, it can sometimes give inaccurate results, i.e. attribute time to the wrong lines. For this reason, I’ve mostly relied on option 5 in the list above.