Pointer being freed was not allocated


#1

Hi,

I am trying to develop LSODA.jl and to overcome some malloc issues. In order to achieve this, I took inspiration from Sundials.jl but I cannot solve the test issue in test1.jl. Basically, calls to lsoda_free give

julia(7820,0x7fffc9d7a3c0) malloc: *** error for object 0x7fec0a0a22d0: pointer being freed was not allocated
*** set a breakpoint in malloc_error_break to debug

signal (6): Abort trap: 6
while loading no file, in expression starting on line 0
__pthread_kill at /usr/lib/system/libsystem_kernel.dylib (unknown line)
Allocations: 1935656 (Pool: 1934692; Big: 964); GC: 0

I am a bit new to finalizers mechanisms and I would appreciate some input. Also, it might be useful to know that a Julia structure is passed to a C library which will fill in the internals. When freeing this might cause a problem. In Sundials, everything is done on the C side.

Thank you for your help,


#2

This almost certainly means you are allocating in the library and freeing with Julia (or vice-versa). I didn’t see any obvious problem, but the lsoda_common_t struct looks suspicious (alloc_mem is hard to follow, at least at a quick glance).


#3

Yes you are right. I allocate lsoda_context_t() and lsoda_opt_t() with julia and send it to the C-library which further allocates some subfields. Am I not allowed to do that?


#4

From what I understand, the library expects a pointer to an existing struct. It then allocates memory for the common struct, and fills that in the ->common field. Likewise lsoda_free only touches the common field.

This should be fine, but:

  • typically struct (formerly immutable) is recommended for C layout compatibility
  • there’s no need call Ref( ... ) to pass a reference. Just set the argument type as Ref{ ... } and pass the variable by name (referring to the code with Ref(ctx) in the types file).

I tried to reproduce the issue from these examples but ran in to some errors. A minimal demo might help with debugging if you are still stuck.


#5

immutability doesn’t increase C compatibility. It’s fine to use mutable struct as long as the C library doesn’t write a reference of a heap reference in another object. (i.e. C library doing a.b = c is not allowed if a.b is not a pointerfree field). It is also perferred to use mutable type if the C library is expected to modify it.


#6

This has been a soft rule-of-thumb for a long time, because struct/immutable will basically just work like a C struct (that was even one of the minor arguments for the immutable -> struct rename), but mutable struct requires more thought. There are various issues like: a mutable struct can’t be stored inline even if the fields are otherwise pointer free, unsafe_store! isn’t available, and they can’t be passed by-val.

That said, I don’t think it’s the problem here unless with with_kw macro is doing something really unexpected.


#7

For layout, kind of, but in general, it depends. Since immutable are, well, not mutable, it is actually preferable to use mutable struct in this case so that the C library can easily mutate the struct.


#8

Thank you all for your answers.

Hopefully, I won’t have to change the C-library. As suggested, a solution might be to do all allocations on C-side.

@ihnorton Here is a copy from these examples which generates the malloc issue.

using LSODA

function rhs!(t, x, ydot, data)
	ydot[1]=1.0E4 * x[2] * x[3] - .04E0 * x[1]
	ydot[3]=3.0E7 * x[2] * x[2]
	ydot[2]=-ydot[1] - ydot[3]
  nothing
end

rhs!(t, x, ydot) =  rhs!(t, x, ydot, nothing)

y0 = [1.,0.,0.]
# case with a vector
println("\n####################################\n--> Use of a vector of times where output is required")
tspan = [4.*10.0^k for k=-1:10]
ctx, res = @time LSODA.lsoda(rhs!, y0, tspan, reltol= 1e-4,abstol = Vector([1.e-6,1.e-10,1.e-6]))
lsoda_free(ctx)

returns

unhandled error message: 
julia(21398,0x7fff7860e000) malloc: *** error for object 0x110698dd0: pointer being freed was not allocated
*** set a breakpoint in malloc_error_break to debug

signal (6): Abort trap: 6
while loading /Users/rveltz/Downloads/crash-lsoda.jl, in expression starting on line 18
__pthread_kill at /usr/lib/system/libsystem_kernel.dylib (unknown line)
Allocations: 2748719 (Pool: 2747721; Big: 998); GC: 2

where L18 refers to lsoda_free(ctx) so somehow my function lsoda_free is buggy.


DifferentialEquations Pkg won't be added
#9

Ok, the problem is (1) taking and storing a pointer to Julia allocated memory in the ctx->error field and (2) trying to free it when lsoda_free sees that the pointer is valid. This small patch fixes the malloc error on Julia 0.5:

diff --git a/src/types_and_consts.jl b/src/types_and_consts.jl
index 419f556..7aa130d 100644
--- a/src/types_and_consts.jl
+++ b/src/types_and_consts.jl
@@ -66,7 +66,7 @@ typealias _lsoda_f Ptr{Void}
     data::Ptr{Void} = C_NULL ##
     neq::Cint = 0
     state::Cint = 0
-    error::Cstring = pointer("")
+    error::Cstring = C_NULL
     common::Ptr{lsoda_common_t} = C_NULL
     opt::Ptr{lsoda_opt_t} = C_NULL
 end

But, interestingly, the code runs fine on 0.6-dev (which is why I could not reproduce initially).


#10

Thank you! It seems to fix the malloc issue. This week-end, I found that using error::Cstring = Cstring(C_NULL) did not fix the malloc issue and then I embarked in writing a wrapper where allocation is done in the C-library side. I cannot reproduce the errors I was finding though.

@ihnorton How do you find the origin bug? Did you use a debugger like gdb?


#11

I did run under lldb at first, but ended up finding the bug with a simpler observation. I commented out the lsoda_free call, and printed the context struct:

(LSODA.lsoda_context_t
  function_: Ptr{Void} Ptr{Void} @0x0000000314136490
  data: Ptr{Void} Ptr{Void} @0x0000000114dc5c10
  neq: Int32 3
  state: Int32 2
  error: Cstring Cstring(0x000000011536acf0)
  common: Ptr{LSODA.lsoda_common_t} Ptr{LSODA.lsoda_common_t} @0x00007fdf2a459d70
  opt: Ptr{LSODA.lsoda_opt_t} Ptr{LSODA.lsoda_opt_t} @0x0000000114dcf6d0
,
[1.0 0.0 0.0; 0.91175 2.31366e-5 0.0882271; … ; 4.65893e-7 1.86357e-12 1.0; 1.42368e-8 5.69474e-14 1.0])

Then called lsoda_free directly, and noticed that the pointer being freed (0x11536acf0) was the one stored in the error field:

julia> lsoda_free(ctx)
unhandled error message:
jl5(31726,0x7fffa98ef3c0) malloc: *** error for object 0x11536acf0: pointer being freed was not allocated
*** set a breakpoint in malloc_error_break to debug

HTH!


#12

Nice! Thank you.