I am trying to develop LSODA.jl and to overcome some malloc issues. In order to achieve this, I took inspiration from Sundials.jl but I cannot solve the test issue in test1.jl. Basically, calls to lsoda_free give
julia(7820,0x7fffc9d7a3c0) malloc: *** error for object 0x7fec0a0a22d0: pointer being freed was not allocated
*** set a breakpoint in malloc_error_break to debug
signal (6): Abort trap: 6
while loading no file, in expression starting on line 0
__pthread_kill at /usr/lib/system/libsystem_kernel.dylib (unknown line)
Allocations: 1935656 (Pool: 1934692; Big: 964); GC: 0
I am a bit new to finalizers mechanisms and I would appreciate some input. Also, it might be useful to know that a Julia structure is passed to a C library which will fill in the internals. When freeing this might cause a problem. In Sundials, everything is done on the C side.
This almost certainly means you are allocating in the library and freeing with Julia (or vice-versa). I didn’t see any obvious problem, but the lsoda_common_t struct looks suspicious (alloc_mem is hard to follow, at least at a quick glance).
Yes you are right. I allocate lsoda_context_t() and lsoda_opt_t() with julia and send it to the C-library which further allocates some subfields. Am I not allowed to do that?
From what I understand, the library expects a pointer to an existing struct. It then allocates memory for the common struct, and fills that in the ->common field. Likewise lsoda_free only touches the common field.
This should be fine, but:
typically struct (formerly immutable) is recommended for C layout compatibility
there’s no need call Ref( ... ) to pass a reference. Just set the argument type as Ref{ ... } and pass the variable by name (referring to the code with Ref(ctx) in the types file).
I tried to reproduce the issue from these examples but ran in to some errors. A minimal demo might help with debugging if you are still stuck.
immutability doesn’t increase C compatibility. It’s fine to use mutable struct as long as the C library doesn’t write a reference of a heap reference in another object. (i.e. C library doing a.b = c is not allowed if a.b is not a pointerfree field). It is also perferred to use mutable type if the C library is expected to modify it.
This has been a soft rule-of-thumb for a long time, because struct/immutable will basically just work like a C struct (that was even one of the minor arguments for the immutable → struct rename), but mutable struct requires more thought. There are various issues like: a mutable struct can’t be stored inline even if the fields are otherwise pointer free, unsafe_store! isn’t available, and they can’t be passed by-val.
That said, I don’t think it’s the problem here unless with with_kw macro is doing something really unexpected.
For layout, kind of, but in general, it depends. Since immutable are, well, not mutable, it is actually preferable to use mutable struct in this case so that the C library can easily mutate the struct.
using LSODA
function rhs!(t, x, ydot, data)
ydot[1]=1.0E4 * x[2] * x[3] - .04E0 * x[1]
ydot[3]=3.0E7 * x[2] * x[2]
ydot[2]=-ydot[1] - ydot[3]
nothing
end
rhs!(t, x, ydot) = rhs!(t, x, ydot, nothing)
y0 = [1.,0.,0.]
# case with a vector
println("\n####################################\n--> Use of a vector of times where output is required")
tspan = [4.*10.0^k for k=-1:10]
ctx, res = @time LSODA.lsoda(rhs!, y0, tspan, reltol= 1e-4,abstol = Vector([1.e-6,1.e-10,1.e-6]))
lsoda_free(ctx)
returns
unhandled error message:
julia(21398,0x7fff7860e000) malloc: *** error for object 0x110698dd0: pointer being freed was not allocated
*** set a breakpoint in malloc_error_break to debug
signal (6): Abort trap: 6
while loading /Users/rveltz/Downloads/crash-lsoda.jl, in expression starting on line 18
__pthread_kill at /usr/lib/system/libsystem_kernel.dylib (unknown line)
Allocations: 2748719 (Pool: 2747721; Big: 998); GC: 2
where L18 refers to lsoda_free(ctx) so somehow my function lsoda_free is buggy.
Ok, the problem is (1) taking and storing a pointer to Julia allocated memory in the ctx->error field and (2) trying to free it when lsoda_free sees that the pointer is valid. This small patch fixes the malloc error on Julia 0.5:
Thank you! It seems to fix the malloc issue. This week-end, I found that using error::Cstring = Cstring(C_NULL) did not fix the malloc issue and then I embarked in writing a wrapper where allocation is done in the C-library side. I cannot reproduce the errors I was finding though.
@ihnorton How do you find the origin bug? Did you use a debugger like gdb?
I did run under lldb at first, but ended up finding the bug with a simpler observation. I commented out the lsoda_free call, and printed the context struct:
Then called lsoda_free directly, and noticed that the pointer being freed (0x11536acf0) was the one stored in the error field:
julia> lsoda_free(ctx)
unhandled error message:
jl5(31726,0x7fffa98ef3c0) malloc: *** error for object 0x11536acf0: pointer being freed was not allocated
*** set a breakpoint in malloc_error_break to debug