Hi Guys,
I have some nested struct like this, all the field is either another struct, or Float64.
struct A
A1::Float64
A2::Float64
end
Struct B
B1::A
B2::Float64
end
Now, I would like to convert them into an array of Float64, is there any quick method I can do that. In C++, I can simply “cast” the memory.
I know there are some difference between mutable and immutable struct. If the thing I wanna do is not achievable with immutable struct, I can do that with mutable struct if there is a solution.
Thanks
If you have an array of such structs, you can use reinterpret:
julia> struct A
A1::Float64
A2::Float64
end
julia> struct B
B1::A
B2::Float64
end
julia> x = [B(A(3,4),1.2), B(A(6.7,8.9),5)]
2-element Array{B,1}:
B(A(3.0, 4.0), 1.2)
B(A(6.7, 8.9), 5.0)
julia> reinterpret(Float64, x)
6-element reinterpret(Float64, ::Array{B,1}):
3.0
4.0
1.2
6.7
8.9
5.0
4 Likes
oh right when it is in an array it will has an address…
thanks
Well everything stored in memory has an “address” that’s how the code references it in memory. The trouble is immutable structures are usually stored on the stack, so that memory location is only valid while the function is executing.
If you are dealing with an array of the objects then @stevengj probably gave you the answer you want. If you only want to work with a single instance you probably want to look into pointer_from_objref.
If B is mutable then you could do:
julia> ob = B(A(3,4),1.2)
julia> pointer_from_objref(ob)
Ptr{Nothing} @0x00007f351e4923d0
Which gives you a pointer, pass it to C++ and do whatever you want on it. If B is immutable you are probably going to have issues…You might be able to do some trickery like:
julia> pointer_from_objref(Ref(B(A(3,4),1.2)))
Ptr{Nothing} @0x00007f351e7ad4b0
But I’m not sure if that gives you a pointer to the Ref or a pointer to the object it’s referencing.
This is true but irrelevant. Objects in julia does not necessarily have an address, stack or otherwise.
1 Like
I’m probably going to be sorry for asking this. Does that mean that there are objects in Julia you cannot create a Ref() from? Or does the fact that you create a Ref() cause them to have an address?
Neither. Ref doesn’t necessarily have an address. You can force a mutable object to have an stable address but without doing it the object doesn’t necessarily have one.
1 Like
(For example, an object could be stored entirely in registers. Or only partially stored, if the compiler can determine that some portion of the object is not needed.)
2 Likes
I totally agree…But this is a compiler optimization, if I wanted the pointer to that object (and the language allowed me to get a pointer to that object) then the compiler would have to NOT optimize it to register and give me a pointer to the object.
Unless you are saying I can declare an object and require that the compiler only allow it to exist in registers in which case yes, there is no pointer to that object. But I don’t think that is in the Julia language.
Saying that an object might not have an address and IF I need to pass it by pointer to a C library, means I cannot use that object (in this situation). In which case when those objects are declared or used there should be some way for me to tell that it does not have or will not have an address and therefor cannot be passed by pointer to a C library.
Stable address or unstable address makes no difference it has an address. I should be able to pass that address to an external library without issue. The only caveat would be that another thread couldn’t move that object. I could see a compacting garbage collector doing this, but I don’t believe Julia has a compacting garbage collector. I’m also not aware of any Julia commands that would move another thread’s data around in memory. (Yes, I’m aware that growing an array could cause the DATA to to moved to another address…however the address of the array OBJECT would not move.)
And yes I’m sorry I started this, I feel we’re hijacking the original thread.
If you need to pass a pointer to a C library, you declare the argument as Ref{Foo} for ccall. In constructing the Ref{Foo}(foo_object) from a foo_object parameter, the Julia compiler will indeed ensure that it has a memory address that remains valid for the duration of the ccall.
1 Like
No, the fact that an object have an address is a compiler implementation detail.
No that’s not the correct logic. A stable address is something that you have ask for explicitly, which is why I’m saying that’s “not the default”. It’s not part of the normal julia code semantics that you can get an address of it.
No. It means that you need to ask for an address.
The correct way to say this is: The object doesn’t have an address, but you should be able to ask for one if you do it correctly. GC has absolutely nothing to do with any of these.
Put it in another way, there are obviously ways to get an address in either interpretation. However, the distinction here is what the default mode is and what the compiler is meant to hide from you.
In C, in some sense, the object always has an address. Removal of the address is indeed a pure optimization and is meant to be completely transparent for the user. In another word, unless the compiler can proof that you are not using the address in a way that can observe the optimization, the compiler can not remove the address.
This is not the case in julia. Unlike C, normal julia code doesn’t interact with external code by default. That’s why by default non objects has an address you can rely on and that’s why you have to explicitly ask for the address and maintain it. The compiler is meant to hide that address from you. You can get an address with pointer_from_objref but the compiler is making no attempt to figure out how you are using that pointer (in fact, it’ll assume you are NOT using that pointer). You are required to explicitly tell the compiler how you are using the address (GC.@preserve or ccall).
And that’s also why this is “not even wrong”. In julia, for the user, the address is not how you refer to a object, the abstract notion of object reference is, which can be implemented as an address or it might not. Immutable structures are not much more likely to be stored on the stack than mutable ones and pointer_from_objref can very often give you a stack address.
3 Likes
Okay it sounds like we’re getting into semantics. You sound like compiler people (sorry if I’m mistaken) and I’ve had a discussion go this way, and walked away because it wasn’t worth it. The original statement I was commenting on was:
While while technically correct (assuming the compiler implementation can’t doesn’t turn arrays into non addressable entities) I felt could lead people to the wrong conclusions. i.e. people start putting objects in arrays in order to get an address they can pass to C libraries.
So I made the naive statement:
I guess I should have said “If you need an address for an object then the object will have an address.”
I’m not entire sure that Julia programmers need to be aware that an object might not have an address. Or that it didn’t have an address before you asked for an address. I mean is that something I need to consider while programming in Julia, it sounds like no? If I was working on the compiler, then yes, I guess I would need to know that.
Sorry the compiler’s reference was really that sometimes I feel compiler writers go off into the internals of the implementation/optimizations that a user of the language doesn’t really care about or need to know…and knowing just makes them paranoid about what the compiler is doing underneath the covers.
For a lot of things in Julia, it is not that the user doesn’t care or need to know, but sometimes the very concept is undefined by default, allowing the compiler more freedom to optimize/rearrange things.
The compiler can’t ATM but it is allowed to. The array address is also only valid when explicited maintained.
Yes, that’s more or less correct. I didn’t like the original claim that array made the address available either but haven’t got around to reply yet. If I did I would have said sth very similar to this.
It’s sth you need to consider when interacting with C or writing unsafe code in general. That whole area is basically exactly about how to get an address when there’s generally none. Of course when you follow the rules listed in the document you generally dont need to consider much more.
However, if you then take the unsafe code document and trying to apply it to normal code you can easily get some wrong or pretty confusing conclusions about how things should work. That had happened over the years. In another word, what I want to make sure is that you don’t take address, or the unsafe code document in general, as how normal julia works. Normal julia code gives the compiler much more freedom than, say, C compiler in this regard.
The point is “address” itself is a pretty advanced concept in julia (i.e. it only makes sense in unsafe code) and most users should’t need to worry about it. That is actually a detail that the user doesn’t need to know or care. The fact that object may not have an address doesn’t even show up when address itself isn’t something that the user need to know about writing normal code.
2 Likes