Losing bytes when reinterpreting

General Usage
1

Hi all, I am trying to get data over to C, and I went with a ccall that takes in a buffer from julia. The issue I am having is, the data at teh end of my struct is getting chopped off.

@kwdef struct CntlOutput
    time_ns::Int64                                           = 0 
    gyro_valid::Bool                                        = true      
    gyroThetaB_X::Cdouble                           = 0
    gyroThetaB_Y::Cdouble                           = 0
    gyroThetaB_Z::Cdouble  

   blah = CntlOutput(56, true, 456, 234, 2364)
   buf = reinterpret(UInt8, [blah])
   ret = ccall((:sendUDPbytes, "./udpSend.so"), Cint, (Ptr{UInt8}, Int64), buf , sizeof(buf ))

What I get: 56 1 0.000000 0.000000 0.000000

Now if I just pass the variables as normal (Int64, Bool, Cdouble, etc) the data comes out just fine. Also, I commented out the bool, and looked at the data, and there was some non-zero data at the end. Does anyone know what is going on here?

2

It’s hard to tell without a runnable example & the function signature on the C side you’re trying to call. Can you provide a fully-formed MWE that others can run on their machine, to help us help you?

You’re probably observing padding in the struct. You don’t generally need to reinterpret anything when using ccall, provided the structs in C & Julia match.

3

OK, here is the C:

#include <stdio.h> 
#include <strings.h> 
#include <sys/types.h> 
#include <arpa/inet.h> 
#include <sys/socket.h> 
#include <netinet/in.h> 
#include <unistd.h> 
#include <stdlib.h> 
#include <stdbool.h>
#include <string.h>

#define PORT 5000 
#define MAXLINE 1000 

typedef struct {
    int64_t t_imu_gps1980_system_ns;    // GPS seconds in the GPS epoch
    bool    gyro_valid;                 // Gyro validity flag
    double x;
    double y;
    double z;

} CntlOutput;
int sendUDPbytes(uint8_t* ptr, uint64_t size) 
{ 
	char buffer[100]; 
	// char *message = "Hello Server";
    // char message[sizeof(CntlOutput)];
    CntlOutput message, messageBuffer;
	memcpy(&messageBuffer, ptr, size);
    message.t_imu_gps1980_system_ns = messageBuffer.t_imu_gps1980_system_ns;
    message.gyro_valid = messageBuffer.gyro_valid;
    message.x = messageBuffer.x;
    message.y = messageBuffer.y;
    message.z = messageBuffer.z;

	char testMessage[size];
	memcpy(&testMessage, ptr, size);
	printf("\r\n");
	for( int i = 0; i < size; i++)
	{
		printf("0x%X",testMessage[i]);
	}
	printf("\r\n");


	// printf("%ld %i %lf %lf %lf\r\n",
	printf("%ld %lf %lf %lf\r\n",
			messageBuffer.t_imu_gps1980_system_ns,
			messageBuffer.gyro_valid,
			messageBuffer.x,
			messageBuffer.y,
			messageBuffer.z

			);

	int sockfd, n; 
	struct sockaddr_in servaddr; 
	(void)n;
	// clear servaddr 
	bzero(&servaddr, sizeof(servaddr)); 
	servaddr.sin_addr.s_addr = inet_addr("127.0.0.1"); 
	servaddr.sin_port = htons(PORT); 
	servaddr.sin_family = AF_INET; 
	
	// create datagram socket 
	sockfd = socket(AF_INET, SOCK_DGRAM, 0); 
	
	// connect to server 
	if(connect(sockfd, (struct sockaddr *)&servaddr, sizeof(servaddr)) < 0) 
	{ 
		printf("\n Error : Connect Failed \n"); 
		exit(0); 
	} 

	// request to send datagram 
	// no need to specify server address in sendto 
	// connect stores the peers IP and port 
	sendto(sockfd, &message, MAXLINE, 0, (struct sockaddr*)NULL, sizeof(servaddr)); 
	
	// waiting for response 
	recvfrom(sockfd, buffer, sizeof(buffer), 0, (struct sockaddr*)NULL, NULL); 
	puts(buffer); 

	// close the descriptor 
	close(sockfd);

    return 0;
}

If I don’t reinterpret it it gives me:

ERROR: LoadError: MethodError: no method matching unsafe_convert(::Type{Ptr{UInt8}}, ::CntlOutput)

Closest candidates are:
unsafe_convert(::Type{Ptr{UInt8}}, ::String)
@ Base pointer.jl:59
unsafe_convert(::Type{Ptr{UInt8}}, ::Symbol)
@ Base pointer.jl:57
unsafe_convert(::Type{Ptr{R}}, ::SubString{String}) where R<:Union{Int8, UInt8}
@ Base strings/substring.jl:122
…

4

Ok, I think I see the issue; there’s a few problems/misconceptions here.

  1. Julia structs match the layout of C structs, so there’s no need to go through uint8_t*. You can go through CntlOutput* just fine, without having to memcpy things around.
  2. You can then call sendUDPbytes by passing a foo = Ref(blah) like @ccall "./udpSend.so".sendUDPbytes(foo::Ptr{CntlOutput}, 1::Cint). If your intention is to pass a collection/an array of values to C, you can just pass that too (see these C wrapper examples from the manual).

Yes, because there’s no cconvert of the appropriate type defined. See e.g. Calling C and Fortran Code · The Julia Language

EDIT: you generally don’t need to define that either, unless you have a fancier wrapper struct that doesn’t match the layout on the C side anymore (e.g. it contains Arrays where C would have pointers… in those cases, you’ll want to have a “julia version” that you then cconvert to a different struct matching the C version).

1 Like
5

I changed my function signature to:

int sendUDPbytes(CntlOutput* ptr)

and in Julia:

ret = ccall((:sendUDPbytes, "./udpSend.so"), Cint, (foo::Ptr{CntlOutput}), blah)

Now I get this:

ERROR: LoadError: syntax: ccall argument types must be a tuple; try “(T,)” around /home/ig/git/julia-test-harness/NetworkComm/cfsPacking.jl:100

6

If you use the ccall version, you still need to give it the argument types seperately from the objects themselves. Only the @ccall macro can mix them like that. See also their respective docstrings:

ccall

ccall((function_name, library), returntype, (argtype1, ...), argvalue1, ...)
ccall(function_name, returntype, (argtype1, ...), argvalue1, ...)
ccall(function_pointer, returntype, (argtype1, ...), argvalue1, ...)

@ccall

@ccall library.function_name(argvalue1::argtype1, ...)::returntype
@ccall function_name(argvalue1::argtype1, ...)::returntype
@ccall $function_pointer(argvalue1::argtype1, ...)::returntype

Note that I used @ccall (the macro) above, since it makes it easier to figure out which argument will be cconverted to which type.

1 Like
7 
@ccall "./udpSend.so".sendUDPbytes(foo::Ptr{CntlOutput})::Cint

This worked!
Thank you

1 Like
8

The above is not a tuple. (foo::Ptr{CntlOutput},) is a tuple. You need the comma.

julia> (Ptr{CtrlOutput})
Ptr{CtrlOutput}

julia> (Ptr{CtrlOutput},)
(Ptr{CtrlOutput},)

julia> (Ptr{CtrlOutput}) isa Tuple
false

julia> (Ptr{CtrlOutput},) isa Tuple
true