Calling C functions --- segfault when ccall inside a function

General Usage
1

This is really weird. I have this library that I’m trying to call from Julia. If I run this in Main (e.g. in the REPL or a script),

ccall( (:TTVFast, "path/to/library.so"), Cvoid,
    ( ...signature... ), ..., pointer(arr), ...
)

the call works well. Here arr is some array created in Julia. But if I put this inside a Julia function

function foo()
    ccall( (:TTVFast, "path/to/library.so"), Cvoid, ...)
end

I get a segmentation fault:

signal (11): Segmentation fault
in expression starting at /home/daniel/Science/P4-KepDichotomy/TTVFast/jl_version/TTVFast_demo.jl:49
gc_try_setmark at /buildworker/worker/package_linux64/build/src/gc.c:1422 [inlined]
gc_mark_scan_objarray at /buildworker/worker/package_linux64/build/src/gc.c:1516 [inlined]
gc_mark_loop at /buildworker/worker/package_linux64/build/src/gc.c:1804
_jl_gc_collect at /buildworker/worker/package_linux64/build/src/gc.c:2468
jl_gc_collect at /buildworker/worker/package_linux64/build/src/gc.c:2633
jl_gc_pool_alloc at /buildworker/worker/package_linux64/build/src/gc.c:954
jl_gc_alloc_ at /buildworker/worker/package_linux64/build/src/julia_internal.h:274 [inlined]
jl_gc_alloc at /buildworker/worker/package_linux64/build/src/gc.c:2668
_new_array_ at /buildworker/worker/package_linux64/build/src/array.c:100 [inlined]
_new_array at /buildworker/worker/package_linux64/build/src/array.c:158 [inlined]
jl_alloc_array_1d at /buildworker/worker/package_linux64/build/src/array.c:418
Type at ./boot.jl:394 [inlined]
Type at ./boot.jl:403 [inlined]
Type at ./boot.jl:411 [inlined]
similar at ./abstractarray.jl:618 [inlined]
similar at ./abstractarray.jl:617 [inlined]
Type at ./compiler/inferencestate.jl:57
Type at ./compiler/inferencestate.jl:120 [inlined]
abstract_call_method_with_const_args at ./compiler/abstractinterpretation.jl:212
abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:107
abstract_call at ./compiler/abstractinterpretation.jl:829
abstract_eval_call at ./compiler/abstractinterpretation.jl:858
abstract_eval at ./compiler/abstractinterpretation.jl:943
typeinf_local at ./compiler/abstractinterpretation.jl:1155
typeinf_nocycle at ./compiler/abstractinterpretation.jl:1225
typeinf at ./compiler/typeinfer.jl:15
abstract_call_method_with_const_args at ./compiler/abstractinterpretation.jl:216
abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:107
abstract_call at ./compiler/abstractinterpretation.jl:829
abstract_eval_call at ./compiler/abstractinterpretation.jl:858
abstract_eval at ./compiler/abstractinterpretation.jl:943
typeinf_local at ./compiler/abstractinterpretation.jl:1155
typeinf_nocycle at ./compiler/abstractinterpretation.jl:1225
typeinf at ./compiler/typeinfer.jl:15
abstract_call_method_with_const_args at ./compiler/abstractinterpretation.jl:216
abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:107
abstract_call at ./compiler/abstractinterpretation.jl:829
abstract_eval_call at ./compiler/abstractinterpretation.jl:858
abstract_eval at ./compiler/abstractinterpretation.jl:943
typeinf_local at ./compiler/abstractinterpretation.jl:1169
typeinf_nocycle at ./compiler/abstractinterpretation.jl:1225
typeinf at ./compiler/typeinfer.jl:15
typeinf_edge at ./compiler/typeinfer.jl:499 [inlined]
abstract_call_method at ./compiler/abstractinterpretation.jl:381
abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:85
abstract_call at ./compiler/abstractinterpretation.jl:829
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2184
abstract_apply at ./compiler/abstractinterpretation.jl:537
abstract_call at ./compiler/abstractinterpretation.jl:585
abstract_eval_call at ./compiler/abstractinterpretation.jl:858
abstract_eval at ./compiler/abstractinterpretation.jl:943
typeinf_local at ./compiler/abstractinterpretation.jl:1169
typeinf_nocycle at ./compiler/abstractinterpretation.jl:1225
typeinf at ./compiler/typeinfer.jl:15
typeinf_edge at ./compiler/typeinfer.jl:499 [inlined]
abstract_call_method at ./compiler/abstractinterpretation.jl:381
abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:85
abstract_call at ./compiler/abstractinterpretation.jl:829
abstract_eval_call at ./compiler/abstractinterpretation.jl:858
abstract_eval at ./compiler/abstractinterpretation.jl:943
typeinf_local at ./compiler/abstractinterpretation.jl:1155
typeinf_nocycle at ./compiler/abstractinterpretation.jl:1225
typeinf at ./compiler/typeinfer.jl:15
typeinf_edge at ./compiler/typeinfer.jl:499 [inlined]
abstract_call_method at ./compiler/abstractinterpretation.jl:381
abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:85
abstract_call at ./compiler/abstractinterpretation.jl:829
abstract_eval_call at ./compiler/abstractinterpretation.jl:858
abstract_eval at ./compiler/abstractinterpretation.jl:943
typeinf_local at ./compiler/abstractinterpretation.jl:1169
typeinf_nocycle at ./compiler/abstractinterpretation.jl:1225
typeinf at ./compiler/typeinfer.jl:15
typeinf_edge at ./compiler/typeinfer.jl:499 [inlined]
abstract_call_method at ./compiler/abstractinterpretation.jl:381
abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:85
abstract_call at ./compiler/abstractinterpretation.jl:829
abstract_eval_call at ./compiler/abstractinterpretation.jl:858
abstract_eval at ./compiler/abstractinterpretation.jl:943
typeinf_local at ./compiler/abstractinterpretation.jl:1169
typeinf_nocycle at ./compiler/abstractinterpretation.jl:1225
typeinf at ./compiler/typeinfer.jl:15
typeinf_edge at ./compiler/typeinfer.jl:499 [inlined]
abstract_call_method at ./compiler/abstractinterpretation.jl:381
abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:85
abstract_call at ./compiler/abstractinterpretation.jl:829
abstract_eval_call at ./compiler/abstractinterpretation.jl:858
abstract_eval at ./compiler/abstractinterpretation.jl:943
typeinf_local at ./compiler/abstractinterpretation.jl:1169
typeinf_nocycle at ./compiler/abstractinterpretation.jl:1225
typeinf at ./compiler/typeinfer.jl:15
typeinf_edge at ./compiler/typeinfer.jl:499 [inlined]
abstract_call_method at ./compiler/abstractinterpretation.jl:381
abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:85
abstract_call at ./compiler/abstractinterpretation.jl:829
abstract_eval_call at ./compiler/abstractinterpretation.jl:858
abstract_eval at ./compiler/abstractinterpretation.jl:943
typeinf_local at ./compiler/abstractinterpretation.jl:1169
typeinf_nocycle at ./compiler/abstractinterpretation.jl:1225
typeinf at ./compiler/typeinfer.jl:15
typeinf_ext at ./compiler/typeinfer.jl:574
typeinf_ext at ./compiler/typeinfer.jl:611
jfptr_typeinf_ext_1.clone_1 at /home/daniel/.local/share/julia-1.0.3/lib/julia/sys.so (unknown line)
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2184
jl_apply at /buildworker/worker/package_linux64/build/src/julia.h:1537 [inlined]
jl_apply_with_saved_exception_state at /buildworker/worker/package_linux64/build/src/rtutils.c:257
jl_type_infer at /buildworker/worker/package_linux64/build/src/gf.c:275
jl_compile_method_internal at /buildworker/worker/package_linux64/build/src/gf.c:1786 [inlined]
jl_fptr_trampoline at /buildworker/worker/package_linux64/build/src/gf.c:1830
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2184
logging_error at ./logging.jl:339
unknown function (ip: 0x7fe5f83a659a)
jl_fptr_trampoline at /buildworker/worker/package_linux64/build/src/gf.c:1831
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2184
do_call at /buildworker/worker/package_linux64/build/src/interpreter.c:324
eval_value at /buildworker/worker/package_linux64/build/src/interpreter.c:430
eval_stmt_value at /buildworker/worker/package_linux64/build/src/interpreter.c:363 [inlined]
eval_body at /buildworker/worker/package_linux64/build/src/interpreter.c:682
jl_interpret_toplevel_thunk_callback at /buildworker/worker/package_linux64/build/src/interpreter.c:806
unknown function (ip: 0xfffffffffffffffe)
unknown function (ip: 0x7fe60397e2bf)
unknown function (ip: 0x1f)
jl_interpret_toplevel_thunk at /buildworker/worker/package_linux64/build/src/interpreter.c:815
jl_toplevel_eval_flex at /buildworker/worker/package_linux64/build/src/toplevel.c:805
jl_parse_eval_all at /buildworker/worker/package_linux64/build/src/ast.c:838
jl_load at /buildworker/worker/package_linux64/build/src/toplevel.c:839
include at ./boot.jl:317 [inlined]
include_relative at ./loading.jl:1044
include at ./sysimg.jl:29
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2184
include at ./client.jl:392
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2184
do_call at /buildworker/worker/package_linux64/build/src/interpreter.c:324
eval_value at /buildworker/worker/package_linux64/build/src/interpreter.c:430
eval_stmt_value at /buildworker/worker/package_linux64/build/src/interpreter.c:363 [inlined]
eval_body at /buildworker/worker/package_linux64/build/src/interpreter.c:682
jl_interpret_toplevel_thunk_callback at /buildworker/worker/package_linux64/build/src/interpreter.c:806
unknown function (ip: 0xfffffffffffffffe)
unknown function (ip: 0x7fe6029300af)
unknown function (ip: 0xffffffffffffffff)
jl_interpret_toplevel_thunk at /buildworker/worker/package_linux64/build/src/interpreter.c:815
jl_toplevel_eval_flex at /buildworker/worker/package_linux64/build/src/toplevel.c:805
jl_toplevel_eval_in at /buildworker/worker/package_linux64/build/src/builtins.c:622
eval at ./boot.jl:319
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2184
eval_user_input at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/REPL/src/REPL.jl:85
macro expansion at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/REPL/src/REPL.jl:117 [inlined]
#28 at ./task.jl:259
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2184
jl_apply at /buildworker/worker/package_linux64/build/src/julia.h:1537 [inlined]
start_task at /buildworker/worker/package_linux64/build/src/task.c:268
unknown function (ip: 0xffffffffffffffff)
Allocations: 2183395 (Pool: 2182927; Big: 468); GC: 3
zsh: segmentation fault (core dumped)  julia

I see a lot of calls to the garbage collector. So that’s a hint. The C function modifies two arrays that were initially created in Julia. In a previous version of this code those values were allocated with Libc.calloc. If I switch back to using Libc.calloc to create the arrays, the function foo() no longer gives a segfault, but when I try to access arr I get a segfault.

Does anyone understand what’s happening? How can I read array values made by a C function?

Thanks for the help.

2

This is most likely wrong. Don’t call pointer in normal code (outside of unsafe_convert and properly GC.@preserved blocks)

That usually means you corrupsted the memory. The GC is the one place that touches a big portion of the heap and the compiler (seen in your bt up in the call stack) is the place where a lot of small allocations happens within well written program that can trigger the GC.

3

Thanks for the help. I’m looking at the documentation but I’m struggling. Can you help me figure out how to pass an array to C so that it is modified in C and I can read the result in Julia? There is a relevant example in the documentation:

result_array = Vector{Cdouble}(undef, nmax - nmin + 1)
errorcode = ccall(
    (:gsl_sf_bessel_Jn_array, :libgsl),
    Cint,
    (Cint, Cint, Cdouble, Ref{Cdouble}),
    nmin, nmax, x, result_array
)

But when I try to do something similar for a struct, I still get segmentation faults:

params = Vector{Cdouble}(undef, 2 + 7nplanets)

... fill params ...

struct RV_entry
  time::Cdouble
  RV::Cdouble
end

struct transit_entry
  planet::Cint
  epoch::Cint
  time::Cdouble
  rsky::Cdouble
  vsky::Cdouble
end

transit_workspace = Vector{transit_entry}(undef, nevents)
rv_workspace      = Vector{RV_entry}(undef, nrvs)

ccall( (:TTVFast, "path/to/libttvfast.so"), Cvoid, 
	(Ptr{Cvoid},Cdouble,Cdouble,Cdouble,Cint,Ptr{Cvoid},Ptr{Cvoid},Cint,Cint,Cint,),
	params,            # input -- array of doubles
	dt,tstart,tfinal,nplanets,
	transit_workspace, # output -- array of structs
	rv_workspace,      # output -- array of structs
	nrvs,nevents,input_flag
)

This works well as-is. But the moment I put the ccall inside a function I get a segmentation fault. Can you see what I’m still doing wrong?

Thanks for the help.

4

The usage as is seems fine. It’s also possible that the segfault might be caused by something else.

5

It turns out that it’s worse than I thought. I was wrong when I said that it works outside the function. It only seemed to work. When I added other (seemingly unrelated) code, I got a segfault again. :frowning:

6

I can’t see anything wrong with the code you’ve posted but having the symptoms occur a bit “randomly” depending on context is pretty standard if you’ve got memory corruption going on.

One simple thing you could try for debugging is over-allocating your input arrays and see whether (a) this stops the segfaults and (b) which parts of those arrays are written to unexpectedly.