Pointer (Julia) not modified after ccall to C function which uses another C function

General Usage
1

I am trying to call a C function from Julia which expects pointers that will be modified. That function is a pre and pro-processing one. Those pointers are modified inside another C function (hcubature, in my case). The function call from Julia works, but the values are unnaffected.

Here is an example:

val = 0.;
err = 0.;
ccall(("integrate", "/home/pedro/codes/shared.so"), Int, (Ref{Float64}, Ref{Float64}), val, err)

The C shared library:

// shared.c
///CUBATUREPATH = /usr/local/include/cubature/
//gcc -fPIC -Wall -fno-exceptions -Werror -I. -I$(CUBATUREPATH) -c shared.c -L. -lm
//shared library built with:
//gcc -shared -o shared.so shared.o $(CUBATUREPATH)hcubature.o

#include <cubature.h>
#include <stdlib.h>
#include <stdio.h>
#include <math.h>

int f(unsigned ndim, const double *x, void *fdata, unsigned fdim, double *fval) {
    double sigma = *((double *) fdata); // we can pass σ via fdata argument
    double sum = 0;
    unsigned i;
    for (i = 0; i < ndim; ++i) sum += x[i] * x[i];
    // compute the output value: note that fdim should == 1 from below
    fval[0] = exp(-sigma * sum);
    return 0; // success*
}

int integrate(double* val, double* err)
{
    double xmin[3] = {-2,-2,-2}, xmax[3] = {2,2,2}, sigma = 0.5;
    hcubature(1, f, &sigma, 3, xmin, xmax, 0, 0, 1e-4, ERROR_INDIVIDUAL, val, err);
    return 0;
}

I tested it in C, it works:

//testing.c
//gcc -g -Wall -fno-exceptions -Werror -I. -I$(CUBATUREPATH) -o testing testing.c shared.o $(CUBATUREPATH)hcubature.o -L. -lm

#include <stdio.h>
#include <math.h>
#include <cubature.h>
#include <shared.h>

int main()
{
    double *val = malloc(sizeof(double));
    double *err = malloc(sizeof(double));
    integrate(val, err);
    printf("----------------\nhcubature test\nComputed integral = %0.10g +/- %g\n", *val, *err);
    free(val);
    free(err);
    return 0;
}

Can anyone point me what is happening and how can I workaround? Maybe it has something to do with memory layout, I don’t know.

Thank you for your time,
Pedro

2

Can you provide a minimum example showing the problem?

In the julia code, val and err are normal Float64s (non-mutable) so clearly these cannot change. What if you do val = Ref(0.); err = Ref(0.)?

2 Likes
3

Yes you must do these.

1 Like
4

I am trying to call a C function from Julia which expects pointers that will be modified.

I think you mean “which expects (constant) pointers referencing allocated memory whose contents will be modified.” It’s hard to be too pedantic when dicussing pointers. :wink:

chv.c

int chval(double *x) {
  *x = 1.0;
  return 0;
}

> cc -shared -o chv.so chv.c

julia> v = Ref(0.)
Base.RefValue{Float64}(0.0)

julia> ccall(("chval","/home/lapeyre/chv.so"),Int,(Ref{Float64},),v); v
Base.RefValue{Float64}(1.0)
2 Likes
5

Just to elaborate a bit… Unlike C, which will give you a pointer to a local variable through which you can modify that variable, there’s no way to do this in Julia. When you do val = 0.0; Ref(val) the Ref wraps a copy of val not the local variable val itself: mutating the contents of the Ref will have no effect on the local variable val.

If you think about it, what the C compiler does here is pretty weird and tricky: the nature of the val variable is fundamentally changed by the fact that you asked for a pointer to it. If you hadn’t asked for a pointer then val would be stored in a register—which you can’t take a pointer to—or maybe not stored anywhere at all. Just because you took a pointer to it, the compiler completely changes the way the variable works and keeps it on the stack instead of a register and makes sure that the stack location is kept in sync with whatever register is used to keep a working copy of the variable.

6 Likes
6

I am confused. So as you say,

julia> v = 0.0
0.0
julia> pv = Ref(v)
Base.RefValue{Float64}(0.0)
julia> pv = 1.0
1.0
julia> v
0.0

Why did it then work in @jlapeyre’s example above? Wasn’t then a pointer to a copy of the variable v passed to the C function?

Edit: Right, I think I see it now. v is already a pointer. Which means that the contents of what it points to may be modified at will, but it is never possible to take a pointer to a Julia variable and have it modified.

7

That’s not part of the C semantics so it’s not really weird… C does have register variables and you are NOT allowed to take the address of. Other than that, EVERYTHING in C/C++ has an address. It’s not any different from the optimizations done in julia.

1 Like
8

Edit 2: Which makes the semantics of Ref interesting (weird?). You are allowed to take pointer of a constant and modify it :wink:

9

Sure, but what’s weird about is that is that everything in C has an address in the first place. At least to me, it’s strange for local variables to be defined in terms of memory.

10

No, you’re not modifying anything constant. You are replacing the contents of a mutable Ref object.

11

This is the struct and constructors

mutable struct RefValue{T} <: Ref{T}
    x::T
    RefValue{T}() where {T} = new()
    RefValue{T}(x) where {T} = new(x)
end

RefValue is nothing more than a wrapper for an object of type T, either uninitialized or initialized with a copy x. It looks like a pointer is only acquired when needed, via pointer_from_objref.

12

Follow up question:

I am printing the contents of a struct in C.

//shared.c
#include <stdio.h>

typedef struct {
    double* array;
    int length;
} Mystruct;

int foo(Mystruct* mystruct)
{
    for (int i = 0; i < mystruct->length; i++)
    {
        printf("%f\n", mystruct->array[i]);
    }
    return 0;
}

Then I call that function from julia as the following:

struct Mystruct
    array::Array{Float64, 1}
    length::Int
end
mystruct = Mystruct([1., 2., 3.], 3);
ccall(("foo", "./shared.so"), Int, (Ref{Mystruct},), mystruct);
ccall(("foo", "./shared.so"), Int, (Mystruct,), mystruct);

The first call (using Ref{}) prints all values as 0, while the second prints the expected 1,2,3. Why is that?

edit: if I print the length value, in both cases it outputs 3.

13

Both are undefined behavior basically. Julia array are not just a pointer.

That’s right, and that’s why I said it’s not any different from C. In julia, there’s also an user accessible address for EVERY objects. It may not be stable and all pointer stuff are “unsafe” but that’s still a well defined language semantics. Depending on what you are doing with the object, e.g. taking it’s address, passing it to a different function, etc, there are very different optimizations that can be done on them, including if the object is allocated on heap/stack/register.

14

So, what should I do If I have a struct that has an array field? How can I guarantee the array will be accessible in the C function?

15

The exact code depends on how are the lifetime managed (e.g. whether the struct need to stay alive for a long time). For the general idea, see How to keep a reference for C structure to avoid GC? - #4 by tk3369, including all the context.

16

In retrospect, semantics was not the right word. Appearance, I think, is a better word.
Ref(0.0) has the look of something that will point at zero, not the look of pointer to a place in memory that will initially hold a zero.

1 Like
17

Ref(0.0) has the look of something that will point at zero

It’s not clear what this statement could mean. But, I agree the “appearance” of Ref(0.0) makes me start down that path. The documentation for Ref is a little mysterious, too.