Clarification regarding `Ref` documentation

The documentation for Ref says

  When passed as a ccall argument (either as a Ptr or Ref type), a Ref object will be converted to a native pointer to the data it
  references. For most T, or when converted to a Ptr{Cvoid}, this is a pointer to the object data. When T is an isbits type, this
  value may be safely mutated, otherwise mutation is strictly undefined behavior.

does this mean one should not pass a mutable struct to a ccall function ?

@ccall "libfoo".baz( a1 :: Ref{MutableStruct}) ::Cvoid

if void baz( Cstruct *a1) does something like a1->f++ inside it ?

does this also mean in general its unsafe to pass mutable structs to shared libraries until one is 100% sure of the function implementation and the structs dont get mutated by the library ?

1 Like

Well, no one else is responding so I’ll take a crack…

You have to think of what the C code will be doing. It needs some way of knowing what memory to actually change. For example, the C code might go 80 bytes from the pointer, interpret the following 4 bytes as an unsigned int, then increment it by 1.

From the Julia side, this means that the unsigned int — or UInt4 UInt32 on our side — needs to be stored reliably in that location. This works for structs with isbits types, which are

essentially, things like numbers and bools that are stored like C types

On the other hand, if the Julia struct is some complicated thing hidden behind pointers or with variable-width things, it’s impossible to say where that unsigned int will actually be. For example, BigFloat numbers can change size depending on if and how setprecision was called, so that’s not an isbits type. So if your Julia struct has a BigFloat in front of the unsigned int, there’s no telling how far it will be from the pointer, so we can’t make any promises about what happens if you pass that thing to your C code.

I don’t know enough about how Julia actually stores structs to say for sure, but I’d bet there are more cases where things go wrong, so there’s just no way to make any promises unless you’re storing a bunch of stuff that just translates nicely to C memory layout.

Did you mean UInt32?

1 Like

Maybe you are looking for

# corrected in next chunk
# @ccall "libfoo".baz( a1 :: Ref(thestruct)) ::Cvoid

corrected:

@ccall "libfoo".baz( a1 :: Ref(MutableStruct)) ::Cvoid

A mutable struct will be passed as a pointer which would be what the library expects.

And prefixing with GC.@preserve:

GC.@preserve thestruct @ccall ...

to avoid deallocation for any reason during the call.

Even if MutableStruct is composed entirely of isbits fields it doesn’t translate nicely to a C structs layout ?

i’m confused Ref(thestruct) , normal parenthesis ?

I meant, Ref{MutableStruct}… which is what was in the OP. So haven’t contributed much.

@jameson wrote that part of the Ref docstring, so maybe he could clarify what exactly he meant.

I read it as saying that it’s okay to send a Ref{MutableStruct}, to ccall, but it’s not okay to mutate it.

However, that doesn’t really mesh well with my understanding of how these things work. My understanding was that it’s not okay to send a Ref{MutableStruct} to ccall at all, since the ccall would end up sending a Ref-pointer to a julia pointer that doesn’t necessarily have a guaranteed layout or stable implementation details. So hopefully we can get an answer from Jameson or someone else knowledgeable about these things.

2 Likes

Yes, of course. Thanks for pointing that out.