Finalizer only works with mutable structs?

[Mikhail_Kagalenko]

@yuyichao

No, you don’t need another one, you simply placed the one at the wrong place.

I can’t [reserve m_p before assigning to it, though.

[Mikhail_Kagalenko]

The text of the standard is quite plain.

[yuyichao]

@Mikhail_Kagalenko

I can’t [reserve m_p before assigning to it, though.

Errr, where exactly did I say preserving m_p? I said the return value of cconvert, again, cconvert, not unsafe_convert. m_p is the return value of unsafe_convert not cconvert.

Also, of course you cannot preserve something before it is generated. But

1. Assignment to an untyped local variable has absolutely zero significance so before or after assignment does not matter at all.
2. You are preserving it for the use in C code, As long as the preserve block includes all the C use it’s fine.

@Mikhail_Kagalenko

The text of the standard is quite plain.

Well, what I think it should behave agrees with all compilers I can find. I did not write either the standard or the C compilers so you can argue with me all day and I’m not going to change my mind unless the compilers changes their behaviors. If you think the standard disagree with the behavior of gcc and clang, feel free to bring it up to either parties.

OTOH, since you at least agree that arr[n] decayed to pointer (i.e. it wasn’t actually a pointer, just behave like pointer in the expression (and sizeof might not be an expression in this sense btw since it doesn’t evaluate anything)), I assume you understand that saying

@Mikhail_Kagalenko

The arr[n] value in C code is a pointer to struct, not the struct.

does not mean anything regarding the layout of arr.

[Mikhail_Kagalenko]

@yuyichao

and sizeof might not be an expression in this sense btw since it doesn’t evaluate anything))

It is one of the three exceptions

[c42f]

I think the confusion you’re having is about how an “array of one element arrays” is laid out in C. It’s actually one contiguous allocation (not an array of pointers!), so the way to do your libmp_test example is as follows:

#include <stdio.h>

typedef struct
{
  int a;
} _M;

typedef _M m_t[1];

void
print_M_arr(int N, m_t *ptr)
{
  for( int n = 0; n < N; n++) {
   fprintf(stdout, "M[%d]->a: %d\n",n, ptr[n]->a);
  }
}

struct M
    a::Cint
end
N=3
m = [M(n) for n = 1:N] # Layout is the same as a normal array!

ccall((:print_M_arr, "./libmp_test"), Cvoid, (Cint, Ptr{M},), N, m)

I’ll admit that the way that GMP defines mpq_t is very clever (allows allocation as a local variable and in a compact array, but is passed by reference to functions rather than by value). The downside is that it doesn’t follow the usual rules people are used to in C for normal structs which can lead to confusion.

[yuyichao]

@c42f

I’ll admit that the way that GMP defines mpq_t is very clever (allows allocation as a local variable and in a compact array, but is passed by reference to functions rather than by value). The downside is that it doesn’t follow the usual rules people are used to in C for normal struct s which can lead to confusion.

I assume you mean “implicity taking the address”? Yeah, that’s quite a intresting use though it fortunately don’t make much difference when calling from a different language.

The other case I’ve seen it is in jmp_buf so it’s probably a very old trick…

[c42f]

@yuyichao

I assume you mean “implicity taking the address”

Yes, users can declare a function foo(mpq_t a) which is functionally the same as foo(mpq_ptr a), a function bar(mpq_t *as) which is functionally the same as bar(mpq_ptr as) and have local variables of type mpq_t which can be passed to foo() by pointer decay. All of which provides a weird semantic that I didn’t know was even available in C. It’s an interesting trick.

[Mikhail_Kagalenko]

Blockquote[quote=“Chris_Foster, post:65, topic:28777”]
I think the confusion you’re having is about how an “array of one element arrays” is laid out in C. It’s actually one contiguous allocation (not an array of pointers!),
[/quote]

Yeah, the reasoning which made it easier for me to understand is that increasing index in ptr[n] by one moves in memory by sizeof(m_t), which is the byte size of the structure rather than the size of machine pointer.

So the immutable structs need to be used for mpq_t, and their cleanup after use by mpq_clear() needs to be explicitly written. There was some discussion above about calling finalizer upon variable going out of scope. Or I suppose one could put an array of immutable struct MPQ into mutable struct that would cleanup all mpq_t allocations.

[yuyichao]

You can add finaLizer to array.

[Mikhail_Kagalenko]

Looks like I can. For some reason I assumed that finalizer must be defined in the class declaration.

[Mikhail_Kagalenko]

@c42f

I’ll admit that the way that GMP defines mpq_t is very clever

So, if the function signature is different, like this:

#include <stdio.h>

typedef struct
{
  int a;
} _M;

typedef _M m_t[1];

void
print_M_arr(int N, m_t **ptr)
{
  for( int n = 0; n < N; n++) {
   fprintf(stdout, "M[%d]->a: %d\n",n, (*ptr)[n]->a);
  }
}

then the Julia call becomes

struct M
    a::Cint
end

N=3

m = [M(n) for n = 1:N]

m_r = Base.cconvert(Ref{M}, m)
GC.@preserve m_r begin
    m_p = Base.unsafe_convert(Ref{M}, m_r)
    ccall((:print_M_arr, "./libmp_test"), Cvoid, (Cint, Ref{Ptr{M}}), N, m_p)
end

Is it the right, idiomatic way to write this call in Julia? I have seen on stackoverflow a suggestion to overload Base.cconvert, but that would put this internal mechanism into the global scope, wouldn’t it?

[yuyichao]

@Mikhail_Kagalenko

So, if the function signature is different, like this:

Note that not only the function signature is different, the body is very different as well. AFIACT this is a very uncommon signature to have. You have a pointer to a pointer to array. The much more common API is to have a pointer to array of pointers. The extra pointer in your case basically serves no purpose…

@Mikhail_Kagalenko

Is it the right, idiomatic way to write this call in Julia?

That seems right. Though as I said, if you don’t need your code to be generic, you could just use pointer in this case after preserving m. The more generic version you have would work for Ref(M(n)) though and the pointer version won’t…

@Mikhail_Kagalenko

but that would put this internal mechanism into the global scope, wouldn’t it?

Well, this is entirely determined by what do you want to use this for. If this type is completely private then no you aren’t really leaking anything. No one will be able to call your method if they couldn’t get a hold of your type. If this type is public and this is the expected way to call C code then you should make the way to call C code publically usable. And finally if this type is public and the way you call C function using it is too obscure for anyone else to use in their code, then sure you should just do it manually. Though if you want to make your own life easier, you could also define it using a private type, like Ptr{PrivateTagTypeOnlyUsedForCCallConversion} as well so that you can write ccall(..., (Ptr{PrivateTagTypeOnlyUsedForCCallConversion},), m). If this is used only at one place then GC.@preserve is basically a way to implement a conversion rule locally. You basically put what should go into cconvert outside of preserve, put the result of it on the preserve and put what should go into unsafe_convert inside of the preserve. (And the reverse process is how you refactor out the local conversion into a reusable pair of cconvert and unsafe_convert implementatioins).

[Mikhail_Kagalenko]

@yuyichao

The extra pointer in your case basically serves no purpose.

The author of the library made it so that if you pass NULL, the C code allocates and initializes array of mpq_t before filling it.

Though as I said, if you don’t need your code to be generic, you could just use pointer in this case after preserving m .

Like this?

m_p = pointer(m)
GC.@preserve m begin
    ccall((:print_M_arr, "./libmp_test"), Cvoid, (Cint, Ref{Ptr{M}},), N, m_p)
end

If this type is completely private then no you aren’t really leaking anything. 

Is any type completely private in Julia?

[yuyichao]

@Mikhail_Kagalenko

The author of the library made it so that if you pass NULL, the C code allocates and initializes array of mpq_t before filling it.

OK sure, that’s fair.

@Mikhail_Kagalenko

Like this?

I would put the pointer call in the preserved block but this should be fine too.

@Mikhail_Kagalenko

Is any type completely private in Julia?

It can be as private as anything else (especially since you were talking about functions to begin with.)

[Mikhail_Kagalenko]

@yuyichao

s it possible to manually trigger memory corruption without GC.preserve ? I tried
GC.gc(true)
but the code still runs

Not in general. The compiler also may or may not decide tell the GC that certain objects aren’t used by your code anymore.

So, can the new 1.3 feature help with this type of bugs? From the release notes:

The ClangSA.jl static analysis package has been imported, which makes use of the clang static analyzer to validate GC invariants in Julia’s C code. The analysis may be run using make -C src analyzegc

This seems to talk about the C code, though.