So, say the library simply adds 1 to each field passed (or to each number field of the struct passed). So, x1 and x2 would go in as 10 and 12 and come back as 11 and 13. For the whole struct - the results would be [11, 13, 15, 17].
@Frank_Applin, I think you have not read the links @yuyichao has shared with you.
There is a very useful piece of information there, which goes (verbatim from @yuyichao) :
Even though it is not called so in C, obj.field produces a lvalue reference which is why & on it gives you the address inside obj. OTOH, obj.field always produce a rvalue in julia, or in another word, local variable assignment is never significant, you can insert as many of them as you want in the code and they’ll never make any difference in semantics. This means that pointer_from_objref(m.handle) corresponds to
in your example,
mutable struct points
x1::Int32
y1::Int32
x2::Int32
y2::Int32
end
function increment(obj::points)
x1, y1, x2, y2 = obj.x1, obj.y1, obj.x2, obj.y2
ccall((:incfunc, "shapes.so"), Void, (Ref{CInt}, Ref{CInt}, Ref{CInt}, Ref{CInt}), x1, y1, x2, y2)
obj.x1 = x1
obj.y1 = y1
obj.x2 = x2
obj.y2 = y2
end
is totally fine. What is not OK is to use
ccall((:incfunc, "shapes.so"), Void, (Ref{CInt}, Ref{CInt}, Ref{CInt}, Ref{CInt}), obj.x1, obj.y1, obj.x2, obj.y2)
since this is not doing what you want it to for the exact reason @yuyichao has mentioned before. When you call obj.field, you simply create a copy and change that copied variable’s value, which has no effect on the obj.field’s actual value. This is the exact same reason why you see this behaviour:
julia> mutable struct MyType
x::Int32
y::Float64
end
julia> obj = MyType(1, 2.)
MyType(1, 2.0)
julia> pointer_from_objref(obj.x)
Ptr{Void} @0x00007f3c39160080
julia> pointer_from_objref(obj.y)
Ptr{Void} @0x00007f3c2d1d4f50
julia> pointer_from_objref(obj.x)
Ptr{Void} @0x00007f3c39160080
julia> pointer_from_objref(obj.y) # different address, as this time the copy of obj.y is allocated at a different place in memory
Ptr{Void} @0x00007f3c2d1d7320
julia> pointer_from_objref(obj)
Ptr{Void} @0x00007f3c2ccaac90
julia> pointer_from_objref(obj) + fieldoffset(MyType, 1)
Ptr{Void} @0x00007f3c2ccaac90
julia> pointer_from_objref(obj) + fieldoffset(MyType, 2)
Ptr{Void} @0x00007f3c2ccaac98
As you can see, the copied variables’ addresses are not correct at all!
Sorry, @yuyichao, if I am being rude to interfere, or if I am writing nonsense (which I hope I am not, after that discussion).
OK. Thanks. I’m just trying to get a grip on what retains it value and what doesn’t. Now, I see that by your example that the local variables gets changed by the ccall’s to the external function (which is suppose to increment each field by one) and you just reassign the values from the local variables back to the obj fields.
@Frank_Applin, the point in that example was to show you (since I had the very same problem in trying to pass opaque Ptr{Void} handles to a library to manage resources that are not used anymore, with the help of the finalizer feature of the garbage collector) that obj.field notation in Julia creates a copy. This is already mentioned by @yuyichao, though. I am trying to keep in mind that obj.field is equivalent to getfield(obj, :field) which is simply a function call that returns a value of type typeof(obj.field). The value is the same as obj.field, but they are not necessarily the same objects.
And in that long discussion with @yuyichao, I was also trying to understand the mechanism. By all these types of precautions that he is trying to teach us, we are either making the compiler to keep the object live until ccall finishes its job or doing what we really want to do.
Apologies to both of you if I interrupted…
Both
And
are valid code it’s just that you shouldn’t expect any of them to affect the value of obj, x1, y1, x2, y2. The Ref argument type has a special semantic that does a copy and pass by reference so only that (inaccessible) copy will be modified. If you want an object/field to be modified, you have to do that explicitly.
Note that this will give you the same value since we cache the boxed value. It still has no relation with the address of obj or the x field though.
No. This is the thing you have to do explicitly.
Keeping the object alive is another story, which is related by was what I was trying to do after making sure the right pointer is passed in the other thread.
Before worrying about GC safety, the main piece that’s missing here is to make the output parameter working. The only thing that needs to be done is to translate &obj->x1 or &x1 from C. Copying/modifying from the example in the other thread, you can do these with
x1 = Ref(obj.x1)
ccall(..., x1)
x1[]
and
ccall(..., pointer_from_objref(obj) + fieldoffset(typeof(obj), 1)))
Then see Properly using `finalizer`, `ccall`, `cconvert` and `unsafe_convert` - #10 by yuyichao for how to make the latter GC safe.
Other than special syntax from macros, whenever you have f(a) where f can be anything including ccall etc, it is impossible for it to modify the binding of a. It can only mutate a if a is mutable.
Just to persuade myself, I implemented those now, and I have one question:
# C Code
#
# void incfunc(int* x1, int* y1, int* x2, int* y2) {
# *x1 += 1;
# *y1 += 1;
# *x2 += 1;
# *y2 += 1;
# }
const libhandle = Libdl.dlopen(joinpath(pwd(), "libshapes.so"))
const funchandle = Libdl.dlsym(libhandle, :incfunc)
type Points
x1::Int32
y1::Int32
x2::Int32
y2::Int32
end
obj = Points(1,2,3,4)
function wronginc!(obj::Points)
ccall(funchandle, Void, (Ref{Cint}, Ref{Cint}, Ref{Cint}, Ref{Cint}),
obj.x1, obj.y1, obj.x2, obj.y2)
return obj
end
res = wronginc!(obj)
# Works fine...
function correctinc1!(obj::Points)
x1 = Ref(obj.x1)
x2 = Ref(obj.x2)
y1 = Ref(obj.y1)
y2 = Ref(obj.y2)
ccall(funchandle, Void, (Ref{Cint}, Ref{Cint}, Ref{Cint}, Ref{Cint}),
x1, x2, y1, y2)
obj.x1 = x1[]
obj.y1 = y1[]
obj.x2 = x2[]
obj.y2 = y2[]
return obj
end
res = correctinc1!(obj)
# Below is working fine
function attempt2!(obj::Points)
# Below, using Ref{Cint} results in `InexactError()`
ccall(funchandle, Void, (Ptr{Cint}, Ptr{Cint}, Ptr{Cint}, Ptr{Cint}),
pointer_from_objref(obj) + fieldoffset(Points, 1),
pointer_from_objref(obj) + fieldoffset(Points, 2),
pointer_from_objref(obj) + fieldoffset(Points, 3),
pointer_from_objref(obj) + fieldoffset(Points, 4))
return obj
end
res = attempt2!(obj)
# Julia crashes horribly below
# WARNING: An error occurred during inference. Type inference is now partially
# disabled.
# BoundsError(a=Array{Any, 1}[
# Core.Inference.VarState(typ=Core.Inference.Const(val=typeof(Base.haskey)(),
# actual=false), undef=false),
# Core.Inference.VarState(typ=Base.Dict{Int64, Juno.LazyTree}, undef=false),
# Core.Inference.VarState(typ=Int64, undef=false)], i=(4,))
function attempt3!(obj::Points)
x1, y1, x2, y2 = obj.x1, obj.y1, obj.x2, obj.y2
ccall(funchandle, Void, (Ptr{Cint}, Ptr{Cint}, Ptr{Cint}, Ptr{Cint}),
pointer_from_objref(x1), pointer_from_objref(y1), pointer_from_objref(x2),
pointer_from_objref(y2))
obj.x1, obj.y1, obj.x2, obj.y2 = x1, y1, x2, y2
return obj
end
res = attempt3!(obj::Points)
What is wrong in attempt3? I also tried using
x1::Int32, y1::Int32, x2::Int32, y2::Int32 = obj.x1, obj.y1, obj.x2, obj.y2
when declaring the local variables. Now, I can’t see why it fails.
And why do we have InexactError() in attempt2!? Is it because of the conversion from Ptr{Void} to Ref{Cint}, which are of different types?
So, how about actually passing an instance of the Type/struct to the C function instead of individual fields? That would cover both examples. Thanks for taking the time to do this. I think that’s what we need is some simple to more complex examples - completely worked through.
If you cast that pointer to Ptr{Cint} it’ll be fine. The backtrace should tell you that it is trying to convert a Ptr{Void} to a Cint. This is the same issue why you need to define cconvert to get Ref{Ptr{Void}} working in the other thread IIRC.
This is “working” except that the GC can free obj. Of course that’s the next level of problem so this is fine for demonstrating how to pass the correct pointer.
If the same error happens when funchandle is C_NULL (i.e. so the ccall won’t be executed) then it’s a typeinf bug (I don’t see it on 0.7). If not than it’s just because this code is doing something very bad. Calling pointer_from_objref on isbits object is invalid and in this case the cached boxed value of Cint will be returned so you are corrupting a global cache which will cause all sorts of issues.
“passing an instance of the Type/struct” is ambiguous. You can pass the value of the struct or you can pass a pointer. All the above example are just showing how you can pass a pointer if you know what C code you want.
In your question, you asked about passing a field of C-style struct defined in Julia to be changed inside the function. Sure, you can do that easily as @yuyichao has already told you. A MWE is as follows:
# shapes.cc
#
# void incfunc(int* x1, int* y1, int* x2, int* y2) {
# *x1 += 1;
# *y1 += 1;
# *x2 += 1;
# *y2 += 1;
# }
#
# struct Points {
# int x1, y1, x2, y2;
# };
#
# void changeme(struct Points* pptr) {
# pptr->x1 += 1;
# pptr->y1 += 1;
# pptr->x2 += 1;
# pptr->y2 += 1;
# }
const libhandle = Libdl.dlopen(joinpath(pwd(), "libshapes.so"))
const funchandle = Libdl.dlsym(libhandle, :incfunc)
const changehndl = Libdl.dlsym(libhandle, :changeme)
type Points
x1::Int32
y1::Int32
x2::Int32
y2::Int32
end
obj = Points(1,2,3,4)
function changeme!(obj::Points)
ccall(changehndl, Void, (Ref{Points},), obj)
return obj
end
changeme!(obj)
function changemeagain!(obj::Points)
x1 = Ref(obj.x1)
x2 = Ref(obj.x2)
y1 = Ref(obj.y1)
y2 = Ref(obj.y2)
ccall(funchandle, Void, (Ref{Cint}, Ref{Cint}, Ref{Cint}, Ref{Cint}),
x1, x2, y1, y2)
obj.x1 = x1[]
obj.y1 = y1[]
obj.x2 = x2[]
obj.y2 = y2[]
return obj
end
res = changemeagain!(obj)
Why the s?
obj.x1 = x1obj.y1 = y1obj.x2 = x2
obj.y2 = y2Sent from Yahoo Mail on Android
last sentence in the corresponding part of the documentation.
Ok. Got it. Thanks for the working example. That helps a lot!
1 Like
The code
function attempt3!(obj::Points)
x1, y1, x2, y2 = obj.x1, obj.y1, obj.x2, obj.y2
ccall(C_NULL, Void, (Ptr{Cint}, Ptr{Cint}, Ptr{Cint}, Ptr{Cint}),
pointer_from_objref(x1), pointer_from_objref(y1), pointer_from_objref(x2),
pointer_from_objref(y2))
obj.x1, obj.y1, obj.x2, obj.y2 = x1, y1, x2, y2
return obj
end
attempt3!(obj)
simply gives ccall: null function pointer.
So basically, since isbits(obj.x1) == true, I am doing an illegal operation by passing its pointer to a C-function to mutate its value. isbits types are immutable in the end, or? Am I getting this correctly?
Roughtly yes. Basically pointer_from_objref is invalid on isbits types and trying to mutate the memory that belongs to a immutable object is also always invalid. There are multiple ways to conclude why the code above is invalid based on these.
OK. Guys - I have a different question not a ccall one. Yay!
The C++ code I am converting the author has an array of 3 characters that he assigns values to. He then converts the character array to an unsigned long.
Here’s the C++ code:
unsigned char send[3], receive[3];
send[0] = 0b00000001; // The Start Bit followed
// Set the SGL/Diff and D mode -- e.g., 1000 means single ended CH0 value
send[1] = 0b10000000; // The MSB is the Single/Diff bit and it is followed by 000 for CH0
send[2] = 0; // This byte doesn't need to be set, just for a clear display
busDevice->transfer(send, receive, 3);
Here’s part of the method call:
int SPIDevice::transfer(unsigned char send[], unsigned char receive[], int length){
struct spi_ioc_transfer transfer;
for (int i = 0; i < length; i++)
cout << "send[" << i << "]: " << (unsigned int) send[i] << endl;
cout << "(unsigned long) send: " << (unsigned long) send << endl;
transfer.tx_buf = (unsigned long) send;
cout << "sizeof(tx_buf): " << sizeof(transfer.tx_buf) << endl;
Here’s the output:
send[0]: 1
send[1]: 128
send[2]: 0
(unsigned long) send: 2124198388
sizeof(tx_buf): 8
I have tried to get this same unsigned long in Julia via bit-shifting and using reinterpret(), but I can never get a number that matches what he gets via this cast transfer.tx_buf = (unsigned long) send; And, tx_buf is defined in a struct found in spidev.h as __u64 tx_buf;.
Any ideas?
Never mind - apparently the ioctl routine is just going to look at the 3 bytes of the number and he never zeroes out the buffer ahead of time.
No, it’s taking the address.
Ok. So, he’s passing the address via the struct.
OK. I got spi_open, spi_setMode, spi_setSpeed, and spi_setBitsPerWord all working, but as always it’s the one with struct that is not.
So, here’s the C++ code:
int SPIDevice::transfer(unsigned char send[], unsigned char receive[], int length){
struct spi_ioc_transfer transfer;
transfer.tx_buf = (unsigned long) send;
transfer.rx_buf = (unsigned long) receive;
transfer.len = length;
transfer.speed_hz = this->speed;
transfer.bits_per_word = this->bits;
transfer.delay_usecs = this->delay;
transfer.pad = 0;
int status = ioctl(this->file, SPI_IOC_MESSAGE(1), &transfer);
if (status < 0) {
perror("SPI: Transfer SPI_IOC_MESSAGE Failed");
return -1;
}
return status;
}
And, how it’s being used:
cout << "Starting RPi SPI ADC Example" << endl;
SPIDevice *busDevice = new SPIDevice(0,0);
busDevice->printVars();
busDevice->setSpeed(488000); // Have access to SPI Device object
busDevice->setMode(SPIDevice::MODE0);
cout << "The SPI ADC is setup" << endl;
unsigned char send[8], receive[8];
send[0] = 0b00000001; // The Start Bit followed
send[2] = 0; // This byte doesn't need to be set, just for a clear display
send[3] = 0;
busDevice->transfer(send, receive, 3);
The documentation explains how to pass an array, but I’m not passing an array. According to @yuyichao, the tx_buf and rx_buf parts of the struct (both are UInt64) is using the address of the arrays passed in.
So, here’s what I tried:
function spi_transfer(spi::SPIDevice, send::Array{UInt8}, receive::Array{UInt8}, len::Int32)
transfer = spi_ioc_transfer()
#see <linux/spi/spidev.h> for details!
transfer.tx_buf = pointer(send)
transfer.rx_buf = pointer(receive)
transfer.len = len #number of bytes in vector
transfer.speed_hz = spi.speed
transfer.delay_usecs = spi.delay
transfer.bits_per_word = spi.bits
transfer.pad = 0
ret = ccall((:ioctl, "libc"), Cint, (Cint, Clong, Ref{spi_ioc_transfer}...), spi.file, SPI_IOC_MESSAGE(1), transfer)
if (ret < 0)
error("SPI: Transfer SPI_IOC_MESSAGE Failed")
return -1
end
#receive = shift_ulong_into_bytes(transfer.rx_buf)
return ret
end
and, here’s how it’s being called:
spi = spidev.SPIDevice(0,0)
ret = spidev.spi_open(spi)
ret = spidev.spi_setMode(spi, UInt8(SPI_MODE_0))
ret = spidev.spi_setBitsPerWord(spi, UInt8(8))
ret = spidev.spi_setSpeed(spi, UInt32(1000000)) # Have access to SPI Device object
println("The SPI ADC is setup")
send = Array{UInt8, 1}(3)
receive = Array{UInt8, 1}(3)
send[1] = 0b00000001 # The Start Bit followed
# Set the SGL/Diff and D mode -- e.g., 1000 means single ended CH0 value
send[2] = 0b10000000 # The MSB is the Single/Diff bit and it is followed by 000 for CH0
send[3] = 0 # This byte doesn't need to be set, just for a clear display
spidev.spi_transfer(spi, send, receive, 3)
So, I’m quite sure it’ll be something dumb, but can you point out what I’m doing wrong with the rx_buf and tx_buf fields. I’m sure that’s why the ccall is failing (returning a -1).