Hello there!
I’ve been trying to embed julia into a digital content creation software named Houdini (I’ve asked questions about this about two years ago, finally got back at it )
So everything seems to work fine, but jl_atexit_hook(0) always crashes,
even if everything my plugin does is just this:
jl_init();
jl_atexit_hook(0);
the trace is always looking like this:
[551813] signal (11.1): Segmentation fault
in expression starting at none:0
je_arena_mapbitsp_read at /home/prisms/builder-new/WeeklyDevToolsHEAD/dev_tools/src/jemalloc/jemalloc-3.6.0/jemalloc-3.6.0/include/jemalloc/internal/arena.h:525 [inlined]
je_arena_mapbits_get at /home/prisms/builder-new/WeeklyDevToolsHEAD/dev_tools/src/jemalloc/jemalloc-3.6.0/jemalloc-3.6.0/include/jemalloc/internal/arena.h:532 [inlined]
je_arena_mapbits_binind_get at /home/prisms/builder-new/WeeklyDevToolsHEAD/dev_tools/src/jemalloc/jemalloc-3.6.0/jemalloc-3.6.0/include/jemalloc/internal/arena.h:573 [inlined]
je_arena_salloc at /home/prisms/builder-new/WeeklyDevToolsHEAD/dev_tools/src/jemalloc/jemalloc-3.6.0/jemalloc-3.6.0/include/jemalloc/internal/arena.h:991 [inlined]
je_isalloc at /home/prisms/builder-new/WeeklyDevToolsHEAD/dev_tools/src/jemalloc/jemalloc-3.6.0/jemalloc-3.6.0/include/jemalloc/internal/jemalloc_internal.h:849 [inlined]
ifree at /home/prisms/builder-new/WeeklyDevToolsHEAD/dev_tools/src/jemalloc/jemalloc-3.6.0/jemalloc-3.6.0/src/jemalloc.c:1228
uv__finish_close at /workspace/srcdir/libuv/src/unix/core.c:313
uv__run_closing_handles at /workspace/srcdir/libuv/src/unix/core.c:327
uv_run at /workspace/srcdir/libuv/src/unix/core.c:415
ijl_atexit_hook at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/src/init.c:336
execute_native_thread_routine at /lib/libstdc++.so.6 (unknown line)
start_thread at /lib/libc.so.6 (unknown line)
__GI___clone3 at /lib/libc.so.6 (unknown line)
Allocations: 2906 (Pool: 2897; Big: 9); GC: 0
my inexperienced eye tells me that this looks like somehow julia’s allocs get mixed up with main software’s jemalloc…
Does anyone have an insight of what is really going on here, or what can I do to determine the cause and find a solution?
I am completely inexperienced with this but from a read of the source code of jl_atexit_hook and your stacktrace, it looks it’s not julia per say but rather the underlying libuv which is calling your ifree function, and crashing as a result. Are you internally using libuv in your project? In that case, there is probably a conflict with the libuv julia ships with, which is a custom fork (as mentioned here).
unfortunately, all the problems come from the software i’m writing plugin for, and it is closed source. But from the look of it it does not use libuv, so it’s all on julia, however it does use jemalloc that seem to be causing the problems.
using gdb (which I’m completely inexperienced with) I got a but more detailed trace:
#0 0x00007f0651075413 in je_arena_mapbitsp_read (mapbitsp=0x8050b0)
at 0x00004413 (/nix/store/d0821saljv4xafnqyyxmh4rh914wfgzj-houdini-runtime-20.0.506/dsolib/libjemalloc.so.1) in include/jemalloc/internal/arena.h:525
#1 je_arena_mapbits_get (pageind=864, chunk=0x800000) at 0x00004413 (/nix/store/d0821saljv4xafnqyyxmh4rh914wfgzj-houdini-runtime-20.0.506/dsolib/libjemalloc.so.1) in include/jemalloc/internal/arena.h:532
#2 je_arena_mapbits_binind_get (pageind=864, chunk=0x800000)
at 0x00004413 (/nix/store/d0821saljv4xafnqyyxmh4rh914wfgzj-houdini-runtime-20.0.506/dsolib/libjemalloc.so.1) in include/jemalloc/internal/arena.h:573
#3 je_arena_salloc (demote=false, ptr=0xb60c80) at 0x00004413 (/nix/store/d0821saljv4xafnqyyxmh4rh914wfgzj-houdini-runtime-20.0.506/dsolib/libjemalloc.so.1) in include/jemalloc/internal/arena.h:991
#4 je_isalloc (demote=false, ptr=0xb60c80) at 0x00004413 (/nix/store/d0821saljv4xafnqyyxmh4rh914wfgzj-houdini-runtime-20.0.506/dsolib/libjemalloc.so.1) in include/jemalloc/internal/jemalloc_internal.h:849
#5 ifree (ptr=0xb60c80) at 0x00004413 (/nix/store/d0821saljv4xafnqyyxmh4rh914wfgzj-houdini-runtime-20.0.506/dsolib/libjemalloc.so.1) in src/jemalloc.c:1228
#6 0x00007f05d8343459 in uv__finish_close (handle=0xb60c80) at 0x0011c459 (/home/user/project/dso/julia/lib/julia/libjulia-internal.so.1.10) in src/unix/core.c:313
#7 0x00007f05d834349f in uv__run_closing_handles (loop=0x7f05d87ef2c0 <default_loop_struct>)
at 0x0011c49f (/home/user/project/dso/julia/lib/julia/libjulia-internal.so.1.10) in src/unix/core.c:327
#8 0x00007f05d8343759 in uv_run (loop=0x7f05d87ef2c0 <default_loop_struct>, mode=UV_RUN_DEFAULT)
at 0x0011c759 (/home/user/project/dso/julia/lib/julia/libjulia-internal.so.1.10) in src/unix/core.c:415
#9 0x00007f05d828f71e in ijl_atexit_hook (exitcode=<optimized out>)
at 0x0006871e (/home/user/project/dso/julia/lib/julia/libjulia-internal.so.1.10) in /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/src/init.c:336
#10 0x00007f06170a65c3 in execute_native_thread_routine () at 0x000e05c3 (/lib/libstdc++.so.6) in /lib/libstdc++.so.6
#11 0x00007f0616d66084 in start_thread () at 0x0008b084 (/lib/libc.so.6) in /lib/libc.so.6
#12 0x00007f0616de860c in clone3 () at 0x0010d60c (/lib/libc.so.6) in /lib/libc.so.6
I’m afraid solving the root issue is beyond my level of expertise. But if you do not do any kind of IO from the julia side, then I guess you should not care about libuv. In that case, I would suggest creating your own jl_atexit_hook_nolibuv function consisting of the definition of jl_atexit_hook with all the parts related to libuv removed. It looks like except these uv_ calls, all the other ones are jl_ function calls, which should be exported so accessible from within your code.
Or you could completely drop the call to jl_atexit_hook. That’s far from ideal if julia has stuff remaining to be done at this point, but otherwise I would assme that’s fine to do if your program exits immediately after anyway?
not calling jl_atexit_hook at all causes a different segfault by jealloc when thread is being destroyed (i use julia in a dedicated thread)
I’ll try to make an atexit variant that does not make any uv_ calls and see.
as a workaround to avoid crashes - i can just to std::quick_exit , but since it’s done in software’s exit hooks - i rob other plugins from being able to run exit callbacks.
Perhaps see this issue, and specifically this comment about a simple workaround to test. It worked for me to get rid of an at-exit memory-related error while trying to use Julia through Python in Blender
)