Are Julia Arrays double pointers?

When you have a immutable struct

struct A
a::Array{Float64, 1}
end
b = A([1,2,3])
then you can do A.a[2] = 5. Ok, I understand that this is possible as you just mutate the object which A.a is pointing to, you do not change it. You can’t do A.a = [1,2,3,4]. I also get that as A is immutable you can’t just change the pointer A.a.
But you can do push!(A.a, 4). push! of course doesn’t allocate new memory and change A.a every time it is called, but it sometimes will.

So you can change the pointer A.a.

Another experiment someone on slack did is this:
array = ; println(pointer_from_objref(array))
for i in 1:1000000 push!(array, 5) end
println(pointer_from_objref(array)) # same memory address

This now leads to the conclusion that Julia Arrays are in fact pointers to pointers on the heap, and these pointers actually point to the content?

ptr = pointer_from_objref(v::Array{Cint}) can be used as as int** ptr with v[i] stored at ptr[0][i-1], and this is what you should pass to C-functions taking int**. For convenience, there is the function pointer(v), returning ptr[0].

Wrapping another mutable struct A around it makes it a triple pointer. Wrapping an immutable struct A around it makes it a compiler choice whether to allocate a heap-object (triple pointer) or just store the contents (double pointer) on the stack / in register.

In your immutable struct wrap case, pointer_from_objref(b.a) will remain forever unchanged, but pointer(b.a) may change when you push!/resize!.

The julia array is defined here https://github.com/JuliaLang/julia/blob/391f2dd2b5ac67da4a136673b1923d966b606439/src/julia.h#L166-L185.

pointer_from_objref gives you a pointer to the Julia object (constant) while pointer gives you a pointer to the data (can change when e.g. resizing).

Yes it is, but no you must not do this and pointer is by all mean not “for convenience”. In fact, you shouldn’t even use pointer in almost all cases when passing an array to C.

1 Like

Thanks, it appears I yet again underestimated the assumptions julia makes. Apart from the GC-preserve, what is the bad thing about passing pointer_from_objref or pointer to C?

The fact that the C-function might mistakenly believe that it is OK to write to pointer_from_objref, i.e. the jl_array? (which you nicely showed me is never OK for multidimensional arrays, thanks again!)

You can pass these values just fine but in almost no case should you get these values by calling these functions directly. You should use the appropriate argument type in ccall to do this automatically. There’s certainly ways to call these directly and correctly but that’s something you shouldn’t just recommend to everyone.

And the fact that the first element of the array struct is the pointer is implementation detail that you have no reason to rely on.

Thank you very much for the very clear answer, that already helps. Do I understand it right now:
(1) In general an Array here is a int**, so when I do a = zeros(N) I actually have to allocate Nint64 somewhere and also some space for the Array header that points to the Nint64.
(2) A mutable struct will always have pointers to all of its fields and they are all heap-allocated
(3) An immutable struct can also be as the above but if I for example create an immutable struct inside a function and it sees that nobody else will need access to that field and the immutable struct goes out of scope soon anyways, Julia may optimize it away and store it directly to the stack

No.

Yes, just not necessarily Int64

No. There’s no layout difference between mutable struct and immutable ones with the same field types.

There’s very few case where it’ll actually be on the stack. In most cases the struct will just be completely gone and it’s not an optimization that has anything to do with the mutability of the type.