Passing a BigInt (mpz_t for GMP) array to C

General Usage
1

Lately, I’ve managed to pass a BigInt to C and back (Passing GMP variables from C(++) to Julia - #10 by Questioner). Now suppose I want to generate Fibonacci numbers in C and pass them to Julia. I thought I just take the analogous code for arrays

#include<gmp.h>

void test(mpz_t* var[]) {
	mpz_init_set_ui(*var[1], 2);
}

but upon calling it

@ccall "./art.so".test(x::Ptr{Ref{BigInt}})::Cvoid

where x was

 x
2-element Vector{Ref{BigInt}}:
 Base.RefValue{BigInt}(3)
 Base.RefValue{BigInt}(3)

it ran, but crashed upon typing “x” and hitting enter, producing a lengthy error message.

I’d be grateful for any input on how to solve this problem.

2

I’ve finally found a workaround: I’m first creating an array in Julia

grr = Array{BigInt}(undef, 2)

then I save pointers to the elements of that array into a different array

arr = Array{Ptr{Nothing}}(undef, 2)
for j in 1:2
   arr[j] = pointer_from_objref(grr[j])
end

This last array is what I pass to C:

@ccall "./gmptest.so".modify2(arr::Ref{Ptr{Nothing}})::Cvoid

C then operates on arr, like in the following code:

void modify2(mpz_t* arr[]) {
	mpz_set_ui(*arr[1], 10);
}
3

Can’t you just pass grr to modify2 directly? i.e.

@ccall "./gmptest.so".modify2(grr::Ptr{BigInt})::Cvoid

since a Vector{BigInt} is stored under the hood as an array of pointers to BigInt objects (which aren’t stored inline since they are mutable structs), and BigInt objects are laid out identically to mpz_t (so that they can be passed by reference to GMP functions).

4

Hello there,

in the post Passing a BigInt (mpz_t for GMP) array to C I tried in vain to pass a GMP number to C. Upon inspecting the memory, it became apparent that C stored the “chunks” of the big integer in adjacent memory blocks, whereas in Julia the number was scattered.

Firstly, it would be necessary to make the big integer accessible to C, so that passing it to C works, and receiving it also works.

But then, for speed reasons it would seem recommendable to have Julia also store the number chunks in adjacent memory blocks.

I would probably enjoy implementing this myself; the only problem is that I don’t quite know where this functionality is in the Julia code.

5

I’ve been pointed to the right file (base/gmp.jl), but it has to be said that I don’t properly understand that code, since I don’t know many of the symbols and names therein.

6

That code in gmp.jl simply passes BigInt objects directly to ccall as Ref{BigInt}, which should be declared as an mpz_t* argument (a single pointer, not an array of pointers) on the C side.

That is, if you have a C function void foo(mpz_t *arg) { .... }, then you could call it in Julia as @ccall mylibrary.foo(n::Ref{BigInt})::Cvoid where n is a BigInt.

If you modify arg on the C side, then the changes will be visible in n on the Julia side.

(I said something similar in your other thread.)

7

I’ll certainly give it a try.

8

Allright, passing a single BigInt worked. That leaves the array case.

9

The array case failed.

C code

void AddToS(mpz_t arr[]) {
	mpz_add_ui(arr[1], arr[1], 2);
}

Julia code

x = BigInt(40000000000000000000000000000000000000000000000000000000000000000000000000000000000000)

y = BigInt(40000000000000000000000000000000000000000000000000000000000000000000000000000000000000)

a = [x, y]

@ccall "gmptest.so".AddToS(a::Ptr{BigInt})::Cvoid

There was no effect on y or a.

10

If you have a nums::Vector{BigInt} in Julia, e.g. nums = BigInt[1,2,3], I think you can just pass it as Ptr{BigInt} to a C function that takes an mpz_t *. That is, write a C function like:

void foo(mpz_t *nums, size_t len) {
    for (size_t i = 0; i < len; ++i)
        mpz_set_ui(&nums[i], i);
}

and then call it as e.g.

nums = BigInt[17, 23, 51]
@ccall mylibrary.foo(nums::Ptr{BigInt}, length(nums)::Csize_t)::Cvoid

after which the nums array should contain [0,1,2].

  • Note that the BigInt values are already initialized, so we call mpz_set_ui not mpz_init_set_ui.
  • Note that C indexing is 0-based, even though Julia indexing is 1-based.
11

I’ve done basically the exact same thing, but it didn’t work.

12

Does your code work on your system?

13

“Basically” exactly the same thing, or exactly the same thing? The code examples you posted above were not equivalent to this.

14

In what way was my code different? (Note that I’ve posted a new post with code!)

15

Oh, I see, this isn’t going to work because BigInt is a mutable struct in Julia, so a nums::Vector{BigInt} is internally an array of pointers to Julia objects (jl_value_t*).

16

Is there anything one can do about that?

17

Try treating Vector{BigInt} as an array of mpz_t*:

void foo(mpz_t **nums, size_t len) {
    for (size_t i = 0; i < len; ++i)
        mpz_set_ui(nums[i], i);
}

and then call it as:

nums = BigInt[17, 23, 51]
@ccall mylibrary.foo(nums::Ptr{BigInt}, length(nums)::Csize_t)::Cvoid
1 Like
18

I’ll give it a try; that should take about 10 min (since I’ve got problems with my Windows).

EDIT: It worked!

19

Just a note of caution: Different entries in an Array{BigInt} may actually be the same object. For example, when you do zeros(BigInt, N), Julia only allocates a single BigInt(0) that’s referenced by every element of the array. This is safe as long as you stick to Julia’s public API, which only provides out-of-place operations, but if you’re using in-place operations on the C side (or you’re using non-public Julia functions), you need to be careful. Minimal example of what can go wrong:

julia> xs = zeros(BigInt, 2)
2-element Vector{BigInt}:
 0
 0

julia> xs[1] === xs[2]  # Uh oh!
true

julia> Base.GMP.MPZ.add_ui!(xs[1], BigInt(1))  # Increment the first element of xs
1

julia> xs  # Oops...
2-element Vector{BigInt}:
 1
 1
2 Likes
20

If you call the BigInt constructor separately for each element that should guarantee distinct objects, e.g. [BigInt(0) for _ = 1:10] instead of zeros(BigInt, 10).

1 Like