# Memory management issue (malloc) on macBook Pro M2

**URL:** <https://discourse.julialang.org/t/memory-management-issue-malloc-on-macbook-pro-m2/97277>\
**Category:** General Usage\
**Tags:** memory, c\
**Created:** [April 9, 2023, 11:01am UTC](https://discourse.julialang.org/t/memory-management-issue-malloc-on-macbook-pro-m2/97277 "2023-04-09T11:01:34Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![mmv](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mmv/32/26323_2.png) [@mmv](https://discourse.julialang.org/u/mmv)\
**Post date:** [April 9, 2023, 11:01am UTC](https://discourse.julialang.org/t/memory-management-issue-malloc-on-macbook-pro-m2/97277/1 "2023-04-09T11:01:34Z")

</div>

Hi!

I’m developing Julia wrappers to the NASA C library `Mtk` used to manipulate remote sensing data files, available from [GitHub - nasa/MISR-Toolkit: an API facilitating the access of MISR standard product files](https://github.com/nasa/MISR-Toolkit). Here is an example:

```julia
function jMtkLatLonToPathList(latitude, longitude)
    path_cnt = Ref{Cint}(0)
    path_list = Ref{Ptr{Cint}}()
    status = ccall((:MtkLatLonToPathList, mtklib),
        Cint,
        (Cdouble, Cdouble, Ref{Cint}, Ref{Ptr{Cint}}),
        latitude, longitude, path_cnt, path_list)
    if status != 0
        error("MTK error $status: ", jMtkErrorMessage(status))
    end
    julia_path_list = [unsafe_load(path_list[], i) for i in 1:path_cnt[]]
    status = @ccall mtklib.MtkStringListFree(path_cnt[]::Ref{Cint}, path_list::Ref{Ptr{Cint}})::Cint
    if status != 0
        error("jMtkLatLonToPathList status = ", status,
            ", error message = ", jMtkErrorMessage(status))
    end
    return path_cnt[], julia_path_list
end

```

This function works fine on an Intel x86 MacBook Pro (2017) with Julia 1.8.2, and I know the result is correct because it is identical to the output of an IDL implementation on the same platform:

```julia
julia> using JMtk15
[Info: Precompiling JMtk15 [600a0e46-2e83-4a57-a945-993fe116ce31]

julia> latitude = 66.121646
66.121646

julia> longitude = 89.263022
89.263022

julia> path_cnt, path_list = jMtkLatLonToPathList(latitude, longitude)
(17, Int32[7, 8, 9, 10, 11, 12, 13, 14, 146, 147, 148, 149, 150, 151, 152, 153, 154])

```

I am now porting those wrapper functions to an Apple M2 MacBook Pro (2023). Since the `Mtk` library is currently available only for x86 platforms, I am using the x86 version of Julia 1.8.5 under `Rosetta 2`. This works fine for dozens of other functions (not involving memory management), but this particular function causes the following issue:

If I comment out the `@ccall mtklib.MtkStringListFree` statement and the following `if status` block, the output looks correct, but the memory assigned to `path_list` may not be properly disposed of (as recommended in the original C source code):

```julia
julia> using JMtk15
[Info: Precompiling JMtk15 [6c71635d-e68b-418f-880a-c8128ca9dc0a]

julia> latitude = 66.121646
66.121646

julia> longitude = 89.263022
89.263022

julia> path_cnt, path_list = jMtkLatLonToPathList(latitude, longitude)
(17, Int32[7, 8, 9, 10, 11, 12, 13, 14, 146, 147, 148, 149, 150, 151, 152, 153, 154])

```

However, if I re-issue the same command after uncommenting those statements, I run into a malloc error linked to that memory management function:

```julia
julia> path_cnt, path_list = jMtkLatLonToPathList(latitude, longitude)
julia(31292,0x2006d02c0) malloc: *** error for object 0x800000007: pointer being freed was not allocated
julia(31292,0x2006d02c0) malloc: *** set a breakpoint in malloc_error_break to debug

signal (6): Abort trap: 6
in expression starting at REPL[4]:1
__pthread_kill at /usr/lib/system/libsystem_kernel.dylib (unknown line)
pthread_kill at /usr/lib/system/libsystem_pthread.dylib (unknown line)
abort at /usr/lib/system/libsystem_c.dylib (unknown line)
malloc_vreport at /usr/lib/system/libsystem_malloc.dylib (unknown line)
malloc_report at /usr/lib/system/libsystem_malloc.dylib (unknown line)
MtkStringListFree at /Applications/Mtk-1.5.0/lib/libMisrToolkit.so.1.5.0 (unknown line)
jMtkLatLonToPathList at /Users/michel/Projects/MISR/MISR_Toolkit.1.5/JMtk15/src/jMtkLatLonToPathList.jl:64
unknown function (ip: 0x10c0292cd)
ijl_apply_generic at /Users/michel/.julia/juliaup/julia-1.8.5+0.x64.apple.darwin14/lib/julia/libjulia-internal.1.8.dylib (unknown line)
do_call at /Users/michel/.julia/juliaup/julia-1.8.5+0.x64.apple.darwin14/lib/julia/libjulia-internal.1.8.dylib (unknown line)
eval_body at /Users/michel/.julia/juliaup/julia-1.8.5+0.x64.apple.darwin14/lib/julia/libjulia-internal.1.8.dylib (unknown line)
jl_interpret_toplevel_thunk at /Users/michel/.julia/juliaup/julia-1.8.5+0.x64.apple.darwin14/lib/julia/libjulia-internal.1.8.dylib (unknown line)
jl_toplevel_eval_flex at /Users/michel/.julia/juliaup/julia-1.8.5+0.x64.apple.darwin14/lib/julia/libjulia-internal.1.8.dylib (unknown line)
jl_toplevel_eval_flex at /Users/michel/.julia/juliaup/julia-1.8.5+0.x64.apple.darwin14/lib/julia/libjulia-internal.1.8.dylib (unknown line)
jl_toplevel_eval_flex at /Users/michel/.julia/juliaup/julia-1.8.5+0.x64.apple.darwin14/lib/julia/libjulia-internal.1.8.dylib (unknown line)
jl_toplevel_eval_flex at /Users/michel/.julia/juliaup/julia-1.8.5+0.x64.apple.darwin14/lib/julia/libjulia-internal.1.8.dylib (unknown line)
ijl_toplevel_eval_in at /Users/michel/.julia/juliaup/julia-1.8.5+0.x64.apple.darwin14/lib/julia/libjulia-internal.1.8.dylib (unknown line)
eval at ./boot.jl:368 [inlined]
eval_user_input at /Users/julia/.julia/scratchspaces/a66863c6-20e8-4ff4-8a62-49f30b1f605e/agent-cache/default-macmini-x64-5.0/build/default-macmini-x64-5-0/julialang/julia-release-1-dot-8/usr/share/julia/stdlib/v1.8/REPL/src/REPL.jl:151
repl_backend_loop at /Users/julia/.julia/scratchspaces/a66863c6-20e8-4ff4-8a62-49f30b1f605e/agent-cache/default-macmini-x64-5.0/build/default-macmini-x64-5-0/julialang/julia-release-1-dot-8/usr/share/julia/stdlib/v1.8/REPL/src/REPL.jl:247
start_repl_backend at /Users/julia/.julia/scratchspaces/a66863c6-20e8-4ff4-8a62-49f30b1f605e/agent-cache/default-macmini-x64-5.0/build/default-macmini-x64-5-0/julialang/julia-release-1-dot-8/usr/share/julia/stdlib/v1.8/REPL/src/REPL.jl:232
#run_repl#47 at /Users/julia/.julia/scratchspaces/a66863c6-20e8-4ff4-8a62-49f30b1f605e/agent-cache/default-macmini-x64-5.0/build/default-macmini-x64-5-0/julialang/julia-release-1-dot-8/usr/share/julia/stdlib/v1.8/REPL/src/REPL.jl:369
run_repl at /Users/julia/.julia/scratchspaces/a66863c6-20e8-4ff4-8a62-49f30b1f605e/agent-cache/default-macmini-x64-5.0/build/default-macmini-x64-5-0/julialang/julia-release-1-dot-8/usr/share/julia/stdlib/v1.8/REPL/src/REPL.jl:355
jfptr_run_repl_63566 at /Users/michel/.julia/juliaup/julia-1.8.5+0.x64.apple.darwin14/lib/julia/sys.dylib (unknown line)
ijl_apply_generic at /Users/michel/.julia/juliaup/julia-1.8.5+0.x64.apple.darwin14/lib/julia/libjulia-internal.1.8.dylib (unknown line)
#967 at ./client.jl:419
jfptr_YY.967_56999 at /Users/michel/.julia/juliaup/julia-1.8.5+0.x64.apple.darwin14/lib/julia/sys.dylib (unknown line)
ijl_apply_generic at /Users/michel/.julia/juliaup/julia-1.8.5+0.x64.apple.darwin14/lib/julia/libjulia-internal.1.8.dylib (unknown line)
jl_f__call_latest at /Users/michel/.julia/juliaup/julia-1.8.5+0.x64.apple.darwin14/lib/julia/libjulia-internal.1.8.dylib (unknown line)
#invokelatest#2 at ./essentials.jl:729 [inlined]
invokelatest at ./essentials.jl:726 [inlined]
run_main_repl at ./client.jl:404
exec_options at ./client.jl:318
_start at ./client.jl:522
jfptr__start_57423 at /Users/michel/.julia/juliaup/julia-1.8.5+0.x64.apple.darwin14/lib/julia/sys.dylib (unknown line)
ijl_apply_generic at /Users/michel/.julia/juliaup/julia-1.8.5+0.x64.apple.darwin14/lib/julia/libjulia-internal.1.8.dylib (unknown line)
true_main at /Users/michel/.julia/juliaup/julia-1.8.5+0.x64.apple.darwin14/lib/julia/libjulia-internal.1.8.dylib (unknown line)
jl_repl_entrypoint at /Users/michel/.julia/juliaup/julia-1.8.5+0.x64.apple.darwin14/lib/julia/libjulia-internal.1.8.dylib (unknown line)
Allocations: 4713645 (Pool: 4710179; Big: 3466); GC: 2
MicMac3 ~/Projects/MISR % 

```

Questions:

1. Is there anything wrong with the syntax of the Julia function at the top of this message?

2. Is there anything peculiar to watch for when running Julia under `Rosetta 2`?

3. Is it in fact necessary to call the C function `MtkStringListFree` at all, as Julia’s garbage collector may take care of those memory allocations once they are not needed anymore?

Thanks for any comment or suggestion on this matter.

---

<div class="post-metadata">

**Author:** ![mkitti](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mkitti/32/12459_2.png) [@mkitti](https://discourse.julialang.org/u/mkitti)\
**Post date:** [April 9, 2023, 6:47pm UTC](https://discourse.julialang.org/t/memory-management-issue-malloc-on-macbook-pro-m2/97277/2 "2023-04-09T18:47:23Z")

</div>

Why is the first argument of `MtkStringListFree` a `Ref{Cint}`? It shoud just be `Cint`.

[https://nasa.github.io/MISR-Toolkit/html/\_mtk\_string\_list\_free\_8c.html](https://nasa.github.io/MISR-Toolkit/html/_mtk_string_list_free_8c.html)

---

<div class="post-metadata">

**Author:** ![mkitti](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mkitti/32/12459_2.png) [@mkitti](https://discourse.julialang.org/u/mkitti)\
**Post date:** [April 9, 2023, 7:16pm UTC](https://discourse.julialang.org/t/memory-management-issue-malloc-on-macbook-pro-m2/97277/3 "2023-04-09T19:16:12Z")

</div>

Also it is not clear to me if you are using the appropriate function to free the pointer.

[https://nasa.github.io/MISR-Toolkit/html/\_mtk\_lat\_lon\_to\_path\_list\_8c.html](https://nasa.github.io/MISR-Toolkit/html/_mtk_lat_lon_to_path_list_8c.html)

`path_list` is of type `int **`. It is not an array of `char** `.

Looking into the C source code, I see a single `malloc` call is used to allocate the memory. Thus I would use a single `free` call to free the pointer:

```julia
Libc.free(path_list[])

```

---

<div class="post-metadata">

**Author:** ![mmv](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mmv/32/26323_2.png) [@mmv](https://discourse.julialang.org/u/mmv)\
**Post date:** [April 10, 2023, 11:57pm UTC](https://discourse.julialang.org/t/memory-management-issue-malloc-on-macbook-pro-m2/97277/4 "2023-04-10T23:57:12Z")

</div>

Hi @mkitti,

Thanks for taking the time to look into this issue. Here are some further findings:

- The core of the Julia wrapping to the C function was generated automatically with the `CLang` package, as suggested earlier by @Gnimuc, and that came out as

```julia
function MtkLatLonToPathList(lat_dd, lon_dd, pathcnt, pathlist)
    ccall((:MtkLatLonToPathList, libMisrToolkit), MTKt_status, (Cdouble, Cdouble, Ptr{Cint}, Ptr{Ptr{Cint}}), lat_dd, lon_dd, pathcnt, pathlist)
end

```

- Looking at those automatic translations for all C functions of `Mtk`, it appears that **input** arguments are declared explicitly as types (Cint, Float, etc), while **output** arguments are always coded as _pointers_ to variables of the appropriate types.

- I am not very familiar with the C language, but I see that the first 5 lines of the function `MtkLatLonToPathList.c` are as follows:

```plaintext
MTKt_status MtkLatLonToPathList(
  double lat_dd, /**< [IN] Latitude */
  double lon_dd, /**< [IN] Longitude */
  int *pathcnt, /**< [OUT] Path Count */
  int **pathlist /** < [OUT] Path List */ )

```

so why would referring to `pathcnt` as `Ref{Cint}` in the Julia function not be legitimate?

- I also note that this function works well on my older computer: that does not mean the syntax is correct, but at least it does not generate an error, let alone a malloc error that crashes Julia completely.

- On the other hand, I see that `pathlist` is defined as a pointer to an array of integers, and that the statement meant to clear that pointer calls a function applicable to strings. That is very likely a problem. However, I have tried to replace it with `@ccall Libc.free(path_list::Ref{Ptr{Cint}})::Cint`, or variations thereof, and that also crashes Julia completely, or complains about a missing library. What should the syntax of this whole statement be? And how can I check whether I do have such a `free` function installed (I did download `Xcode` and `gcc` on both Macs, so C should be available).

- In the Julia documentation ([C Standard Library · The Julia Language](https://docs.julialang.org/en/v1/base/libc/)) for `Base.Libc.free`, I saw the statement `Call free from the C standard library. Only use this on memory obtained from malloc, not on pointers retrieved from other C libraries. Ptr objects obtained from C libraries should be freed by the free functions defined in that library, to avoid assertion failures if multiple libc libraries exist on the system.` I did not find a function within `Mtk` to clear a pointer to an array of integers, though.

- Lastly, I checked the source file of `MtkLatLonToPathList_test.c`, which is available from `https://github.com/nasa/MISR-Toolkit/blob/master/OrbitPath/src/MtkLatLonToPathList_test.c`, and saw that it does indeed call `free` directly, but without specifying a library name…

So I am still wondering (1) why this worked at all on the old MacBook and (2) how to write a valid statement to clear that pointer.

Thanks again for your inputs.

---

<div class="post-metadata">

**Author:** ![mkitti](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mkitti/32/12459_2.png) [@mkitti](https://discourse.julialang.org/u/mkitti)\
**Post date:** [April 11, 2023, 12:00am UTC](https://discourse.julialang.org/t/memory-management-issue-malloc-on-macbook-pro-m2/97277/5 "2023-04-11T00:00:59Z")

</div>

> [@mmv](#):
>
> - On the other hand, I see that `pathlist` is defined as a pointer to an array of integers, and that the statement meant to clear that pointer calls a function applicable to strings. That is very likely a problem. However, I have tried to replace it with `@ccall Libc.free(path_list::Ref{Ptr{Cint}})::Cint`, or variations thereof, and that also crashes Julia completely, or complains about a missing library. What should the syntax of this whole statement be? And how can I check whether I do have such a `free` function installed (I did download `Xcode` and `gcc` on both Macs, so C should be available).

`Libc.free` is a Julia function. Do not invoke it via `@ccall`.

```julia
help?> Libc.free
  free(addr::Ptr)

  Call free from the C standard library. Only use this on memory obtained from malloc, not on pointers retrieved from
  other C libraries. Ptr objects obtained from C libraries should be freed by the free functions defined in that
  library, to avoid assertion failures if multiple libc libraries exist on the system.

```

---

<div class="post-metadata">

**Author:** ![mkitti](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mkitti/32/12459_2.png) [@mkitti](https://discourse.julialang.org/u/mkitti)\
**Post date:** [April 11, 2023, 12:03am UTC](https://discourse.julialang.org/t/memory-management-issue-malloc-on-macbook-pro-m2/97277/6 "2023-04-11T00:03:55Z")

</div>

> [@mmv](#):
>
> so why would referring to `pathcnt` as `Ref{Cint}` in the Julia function not be legitimate?

The problem is not with the call to `MtkLatLonToPathList`. The problem is the call to `MtkStringListFree`. If this were the correct function to call, it should be as follows. However, this is not the correct function to call.

```julia
status = @ccall mtklib.MtkStringListFree(path_cnt[]::Cint, path_list::Ref{Ptr{Cint}})::Cint

```

---

<div class="post-metadata">

**Author:** ![mkitti](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mkitti/32/12459_2.png) [@mkitti](https://discourse.julialang.org/u/mkitti)\
**Post date:** [April 11, 2023, 12:11am UTC](https://discourse.julialang.org/t/memory-management-issue-malloc-on-macbook-pro-m2/97277/7 "2023-04-11T00:11:37Z")

</div>

> [@mmv](#):
>
> - In the Julia documentation ([C Standard Library · The Julia Language](https://docs.julialang.org/en/v1/base/libc/)) for `Base.Libc.free`, I saw the statement `Call free from the C standard library. Only use this on memory obtained from malloc, not on pointers retrieved from other C libraries. Ptr objects obtained from C libraries should be freed by the free functions defined in that library, to avoid assertion failures if multiple libc libraries exist on the system.` I did not find a function within `Mtk` to clear a pointer to an array of integers, though.

This is true. However, Mtk does not provide a `free`. Thus, you must use `Libc.free` in Julia. We then have to hope that we are referring to the same `Libc`. The problem here is with `Mtk`.

To be clear, your function should be as follows.

```julia
function jMtkLatLonToPathList(latitude, longitude)
    path_cnt = Ref{Cint}(0)
    path_list = Ref{Ptr{Cint}}()
    status = ccall((:MtkLatLonToPathList, mtklib),
        Cint,
        (Cdouble, Cdouble, Ref{Cint}, Ref{Ptr{Cint}}),
        latitude, longitude, path_cnt, path_list)
    if status != 0
        error("MTK error $status: ", jMtkErrorMessage(status))
    end
    julia_path_list = [unsafe_load(path_list[], i) for i in 1:path_cnt[]]
    Libc.free(path_list[])
    return path_cnt[], julia_path_list
end

```

---

<div class="post-metadata">

**Author:** ![mmv](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mmv/32/26323_2.png) [@mmv](https://discourse.julialang.org/u/mmv)\
**Post date:** [April 11, 2023, 12:19am UTC](https://discourse.julialang.org/t/memory-management-issue-malloc-on-macbook-pro-m2/97277/8 "2023-04-11T00:19:07Z")

</div>

Hi @mkitti,

That works great indeed:

```julia
julia> using JMtk15

julia> latitude = 66.121646
66.121646

julia> longitude = 89.263022
89.263022

julia> path_cnt, path_list = jMtkLatLonToPathList(latitude, longitude)
(17, Int32[7, 8, 9, 10, 11, 12, 13, 14, 146, 147, 148, 149, 150, 151, 152, 153, 154])

```

Thanks a lot for clarifying and solving this issue!