# Ccall works on global scope but not within function

**URL:** <https://discourse.julialang.org/t/ccall-works-on-global-scope-but-not-within-function/72119>\
**Category:** General Usage\
**Tags:** ccall, c\
**Created:** [November 26, 2021, 2:55pm UTC](https://discourse.julialang.org/t/ccall-works-on-global-scope-but-not-within-function/72119 "2021-11-26T14:55:23Z")\
**Posts on this page:** 14\
**Page:** 1

<div class="post-metadata">

**Author:** ![xor0110](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/xor0110/32/7926_2.png) [@xor0110](https://discourse.julialang.org/u/xor0110)\
**Post date:** [November 26, 2021, 2:55pm UTC](https://discourse.julialang.org/t/ccall-works-on-global-scope-but-not-within-function/72119/1 "2021-11-26T14:55:24Z")

</div>

I’m trying to create bindings to a C library. I generated some Julia code using `Clang.jl`, and I was able to successfully make some function calls reproducing some C-code program. It’s not very reliable, though. Right now I have a piece of code that seems to work often if I run it in the global scope, but if I create a function with that code inside, and call that function instead, it does not work. The only kind of problem I imagine might happen would be related to invalid memory access, but I never get a segfault or something like that, it only hangs, or the function fails to work as it should (detecting some hardware). What’s the best way to even approach debugging something like that?

Something I suspect I could be doing wrong is that the function expects to receive an array where it can fill in some data. It does not allocate this memory, it only writes there. I think in C this is supposed to be just some array of a struct, and the function returns the number of positions used.

I’m creating a Vector{MyStruct} in Julia, and passing that to ccall. Could this be a problem? Are there situations where one should call malloc from within Julia to allocate memory instead?

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [November 26, 2021, 10:01pm UTC](https://discourse.julialang.org/t/ccall-works-on-global-scope-but-not-within-function/72119/2 "2021-11-26T22:01:08Z")

</div>

Could be a rooting problem, where your Julia array is getting garbage-collected before the C routine accesses it?

In particular, given a `v::Vector{MyStruct}`, you should be doing `ccall` with a `Ptr{MyStruct}` argument and passing `v`, _not_ `pointer(v)` (which is not rooted unless you explicitly use `GC.@preserve`).

Hard to say more without example code.

---

<div class="post-metadata">

**Author:** ![Gnimuc](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gnimuc/32/2194_2.png) [@Gnimuc](https://discourse.julialang.org/u/Gnimuc)\
**Post date:** [November 27, 2021, 1:34am UTC](https://discourse.julialang.org/t/ccall-works-on-global-scope-but-not-within-function/72119/3 "2021-11-27T01:34:54Z")

</div>

> [@xor0110](#):
>
> Right now I have a piece of code that seems to work often if I run it in the global scope, but if I create a function with that code inside, and call that function instead, it does not work.

This looks like a GC issue. You need to take a closer look to the C API to guarantee that the lifetime of the resources allocated on the Julia side extends that on the C side. Sometimes it’s evitable to use global variables and reference counting.

> [@xor0110](#):
>
> but I never get a segfault or something like that, it only hangs, or the function fails to work as it should (detecting some hardware).

This is probably due to you didn’t correctly use the C API to release the resources or reset the device.

For example, in one Julia session, you created a handle `devicePtr` to detect the hardware, then somehow you leaked the `devicePtr`. Then, the next time you try to detect the hardware, it just hangs.

To debug this kinda problem, you may need to do a “hard reset” by plug-unplug the device.

---

<div class="post-metadata">

**Author:** ![xor0110](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/xor0110/32/7926_2.png) [@xor0110](https://discourse.julialang.org/u/xor0110)\
**Post date:** [November 27, 2021, 12:25pm UTC](https://discourse.julialang.org/t/ccall-works-on-global-scope-but-not-within-function/72119/4 "2021-11-27T12:25:29Z")

</div>

Thanks, I did make the mistake with `pointer` before, but I fixed that after I started using Clang.jl to generate the bindings.

Here’s a slightly involved example that supposedly reproduces how this library works. I’m not sure I reproduced all the bugs, but it does illustrate one of the issues I’m having. I’m using [this way](https://discourse.julialang.org/t/interpret-ntuple-1024-uint8-as-a-c-string/23906) to transform `NTuple{N, UInt8}` to strings, and _that_ fails when I move the code to within a function. There seems to be some gotcha related to references. What’s the proper way to encapsulate that code, creating a handy `getstr` or something similar?

Here’s my C code

```julia
#include<stdint.h>
#include<stdio.h>

struct MyStruct {
  char name[16];
  uint8_t ipaddress[4];
  double xyz;
};

int writedata(struct MyStruct * buffer, int * buffersize) {
  *buffersize = *buffersize >> 1;
  for (int i = 0 ; i< *buffersize; i++) {
    int j = 0;
    for (; j<3; j++)
      buffer[i].name[j] = 'a' + j + (j+1)*i;
    buffer[i].name[j] = 0;
    buffer[i].ipaddress[0] = 192;
    buffer[i].ipaddress[1] = 168;
    buffer[i].ipaddress[2] = 0;
    buffer[i].ipaddress[3] = 128 + i;
    buffer[i].xyz = i*i*3.1415;
  }
  return (*buffersize > 0) ? 0 : 1;
}

int main(int argc, char** argv) {
  struct MyStruct mydata[8];
  int buffersize = 8;
  writedata(mydata, &buffersize);
  for (int i; i < buffersize; i++) {
    printf("%s - %d.%d.%d.%d - %f\n",
	   mydata[i].name,
	   mydata[i].ipaddress[0], mydata[i].ipaddress[1], mydata[i].ipaddress[2], mydata[i].ipaddress[3],
	   mydata[i].xyz);
  }
}

/* gcc testapi.c -o testapi.so -shared -fPIC */

```

And the Julia code

```julia
struct MyStruct
    name ::NTuple{16, UInt8}
    ipaddress ::NTuple{4, UInt8}
    xyz ::Float64
end

# getstr(x) = unsafe_string(Ptr{UInt8}(pointer_from_objref(Ref(x))))
getstr(x) = unsafe_string(Ptr{UInt8}(pointer_from_objref(x)))
# getstr(x) = Ptr{UInt8}(pointer_from_objref(x))
# getstr(x) = x

function api_writedata(buffer, buffersize)
    ccall(
        (:writedata, "testapi.so"),
        Cint,
        (Ptr{MyStruct}, Ptr{UInt32}),
        buffer, buffersize
    )
end 

mydata = Vector{MyStruct}(undef, 8)
buffersize = Ref(0x00000008)

api_writedata(mydata, buffersize)
for data in mydata[1:buffersize[]]
    # println("$(data.name) - $(join(data.ipaddress,'.')) - $(data.xyz)")
    println("$(getstr(Ref(data.name))) - $(join(data.ipaddress,'.')) - $(data.xyz)")
end
println("***")

function process()
    mydata = Vector{MyStruct}(undef, 8)
    buffersize = Ref(0x00000008)

    api_writedata(mydata, buffersize)
    for data in mydata[1:buffersize[]]
        # println("$(data.name) - $(join(data.ipaddress,'.')) - $(data.xyz)")
        println("$(getstr(Ref(data.name))) - $(join(data.ipaddress,'.')) - $(data.xyz)")
    end
    
end

process()

```

The resulting output:

```julia
abc - 192.168.0.128 - 0.0
bdf - 192.168.0.129 - 3.1415
cfi - 192.168.0.130 - 12.566
dhl - 192.168.0.131 - 28.273500000000002
***
 - 192.168.0.128 - 0.0
 - 192.168.0.129 - 3.1415
 - 192.168.0.130 - 12.566
 - 192.168.0.131 - 28.273500000000002

```

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [November 27, 2021, 1:23pm UTC](https://discourse.julialang.org/t/ccall-works-on-global-scope-but-not-within-function/72119/5 "2021-11-27T13:23:34Z")

</div>

> [@xor0110](#):
>
> `getstr(x) = unsafe_string(Ptr{UInt8}(pointer_from_objref(x)))`

This isn’t safe, because `x` isn’t rooted — it could get garbage-collected while `unsafe_string` is running. You need `getstr(x) = GC.@preserve x unsafe_string(Ptr{UInt8}(pointer_from_objref(x)))`

However, I would tend to avoid `pointer_from_objref` entirely, and just use something like `String(collect(data.name))`.

---

<div class="post-metadata">

**Author:** ![xor0110](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/xor0110/32/7926_2.png) [@xor0110](https://discourse.julialang.org/u/xor0110)\
**Post date:** [November 27, 2021, 3:07pm UTC](https://discourse.julialang.org/t/ccall-works-on-global-scope-but-not-within-function/72119/6 "2021-11-27T15:07:29Z")

</div>

Great, thanks! I think the point of `unsafe_string` is that it expects zero-terminated data. I suppose an alternative could be `String(collect(Iterators.takewhile(x->x!= '\0', Char.(aa))))`?

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [November 27, 2021, 4:12pm UTC](https://discourse.julialang.org/t/ccall-works-on-global-scope-but-not-within-function/72119/7 "2021-11-27T16:12:07Z")

</div>

> [@xor0110](#):
>
> Great, thanks! I think the point of `unsafe_string` is that it expects zero-terminated data. I suppose an alternative could be `String(collect(Iterators.takewhile(x->x!= '\0', Char.(aa))))` ?

Don’t convert to `Char`, which is a 32-bit quantity (representing a Unicode codepoint) — you want to just look at bytes for a NUL-terminated C string.

You could do `String(collect(Iterators.takewhile(!iszero, data.name)))`.

---

<div class="post-metadata">

**Author:** ![xor0110](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/xor0110/32/7926_2.png) [@xor0110](https://discourse.julialang.org/u/xor0110)\
**Post date:** [November 27, 2021, 5:37pm UTC](https://discourse.julialang.org/t/ccall-works-on-global-scope-but-not-within-function/72119/8 "2021-11-27T17:37:10Z")

</div>

Alright, but creating a `String` will still implicitly convert it al to `Char`, right?

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [November 27, 2021, 9:48pm UTC](https://discourse.julialang.org/t/ccall-works-on-global-scope-but-not-within-function/72119/9 "2021-11-27T21:48:21Z")

</div>

> [@xor0110](#):
>
> Alright, but creating a `String` will still implicitly convert it al to `Char` , right?

Nope. A `String` is stored as a UTF-8 encoded sequence of bytes, which is _not_ the same as an array of `Char`. (When you iterate over a `String`, it _decodes_ these bytes into a sequence of `Char` values for you, but that’s not the format in which it is stored.)

---

<div class="post-metadata">

**Author:** ![xor0110](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/xor0110/32/7926_2.png) [@xor0110](https://discourse.julialang.org/u/xor0110)\
**Post date:** [November 29, 2021, 9:16am UTC](https://discourse.julialang.org/t/ccall-works-on-global-scope-but-not-within-function/72119/10 "2021-11-29T09:16:50Z")

</div>

TIL! Still, converting something to Char and then into a String, or give the array straight to the String constructor, we should expect the UTF-8 encoding process to finish with the same result, no? Sorry for prolonging this topic, you’re saying something very insightful and I’m trying to tap into this sea of knowledge 🙂

---

<div class="post-metadata">

**Author:** ![xor0110](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/xor0110/32/7926_2.png) [@xor0110](https://discourse.julialang.org/u/xor0110)\
**Post date:** [November 29, 2021, 9:22am UTC](https://discourse.julialang.org/t/ccall-works-on-global-scope-but-not-within-function/72119/11 "2021-11-29T09:22:53Z")

</div>

OK everyone, I think I got it working now. Other than the string issue, it seems the problem I had was only related to how the C function works. There’s a second array I was treating as an output, but apparently it’s an input, so I was feeding it garbage and causing havoc. I just followed the C examples, with a C\_NULL instead of an array, and now the bindings seem reliable. Nice start for a Monday. Thanks so much for the help!

---

<div class="post-metadata">

**Author:** ![kristoffer.carlsson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kristoffer.carlsson/32/22_2.png) [@kristoffer.carlsson](https://discourse.julialang.org/u/kristoffer.carlsson)\
**Post date:** [November 29, 2021, 9:24am UTC](https://discourse.julialang.org/t/ccall-works-on-global-scope-but-not-within-function/72119/12 "2021-11-29T09:24:43Z")

</div>

> [@stevengj](#):
>
> You could do `String(collect(Iterators.takewhile(!iszero, data.name)))` .

That will make a redundant copy of the created array in the call to `String`. You need to allocate the array with `Base.StringVector(n)` (which allocates space for a null terminator) to ensure that a no-copy version of `String` is used.

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [November 29, 2021, 12:30pm UTC](https://discourse.julialang.org/t/ccall-works-on-global-scope-but-not-within-function/72119/13 "2021-11-29T12:30:09Z")

</div>

> [@kristoffer.carlsson](#):
>
> That will make a redundant copy of the created array in the call to `String` . You need to allocate the array with `Base.StringVector(n)` (which allocates space for a null terminator) to ensure that a no-copy version of `String` is used.

Yes, but it’s not clear that the use-case here is performance-sensitive enough for the extra code (and reliance on undocumented internals) to be worth the trouble.

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [November 29, 2021, 12:56pm UTC](https://discourse.julialang.org/t/ccall-works-on-global-scope-but-not-within-function/72119/14 "2021-11-29T12:56:37Z")

</div>

> [@xor0110](#):
>
> Still, converting something to Char and then into a String, or give the array straight to the String constructor, we should expect the UTF-8 encoding process to finish with the same result, no?

Yes, there is a `String(::AbstractVector{Char})` constructor (which seems to be undocumented?) that will construct the string by UTF-8 encoding the characters.