Create `BigInt` directedly from MPZ pointer

General Usage
, , ,
1

I have some FFI code that interfaces with another library, and the target library has GNU multiprecision integers. Is it possible to create a BigInt directly from such a multiprecision integer pointer? The default constructor for BigInt is hidden, so I cannot construct a BigInt from limbs directly. julia/base/gmp.jl at 6d172b025e4befc4d274d9fbc9339917a8a86b65 · JuliaLang/julia · GitHub

mutable struct BigInt <: Signed
    alloc::Cint
    size::Cint
    d::Ptr{Limb}

    function BigInt(; nbits::Integer=0)
        b = MPZ.init2!(new(), nbits)
        finalizer(cglobal((:__gmpz_clear, libgmp)), b)
        return b
    end
end
2

Not a solution, but my immediate concern is whether the arbitrarily large heap allocations can be tracked by Julia. Finalizers can free C-side memory when Julia-side mutable wrappers are freed, but Julia won’t inherently know how big those allocations are and adapt the heap size and GC cycles. BigInt are fully tracked because Julia’s allocators were set via GMP’s allocation customization:

3

Is it possible to copy an existing mpz then?

4

IIRC you could in theory create a BigInt with something like

eval(Expr(:new, BigInt, alloc, size, d))

If that’s what you’re asking about. But you would still have to manually take care of GC (as benny mentioned), register the finalizer etc…

5

This is actually a potential problem with respect to initialization, isn’t it?

I.e. julia initializes libgmp. Then you dynamically load your foreign library. Then the foreign library might re-initialize libgmp, and we’re in trouble.

Ideally, your foreign library would first check whether libgmp is already initialized, and if so skip that step.

And likewise, julia should ideally check whether libgmp is already initialized, and if so warn / skip the initialization.

Otherwise, we could get into trouble if you embed julia: Process starts, initializes libgmp, allocates some bigints with allocator A, then loads and initializes julia, julia overwrites the existing initialization with its own allocator B, you free some old bigint, and everything sucks (using freeB on memory allocated by mallocA, bit no no).

Or am I missing something here?

6

I don’t think this is an issue since the library I’m calling (Lean 4) does not initialize libgmp or set any allocation functions.

7

The problem is that even so, if Lean 4 uses a separate instance of libgmp, when it allocates an mpz_t it will not update Julia’s GC counters properly, which is what

ccall((:__gmp_set_memory_functions, libgmp), Cvoid,
              (Ptr{Cvoid},Ptr{Cvoid},Ptr{Cvoid}),
              cglobal(:jl_gc_counted_malloc),
              cglobal(:jl_gc_counted_realloc_with_old_size),
              cglobal(:jl_gc_counted_free_with_size))

does. So to do what you’re suggesting, you would either have to manually handle the GC or somehow make Lean 4 use the same instance of libgmp as that instance of Julia you’re running…

So yeah, probably much easier to copy an existing mpz_t to a BigInt with __gmpz_set.