BigFloat struct size is larger than mpfr_t

Internals & Design Internals
1

The Julia BigFloat struct has an extra field, _d, compared with the corresponding MPFR type mpfr_t. This doesn’t affect Base because the additional field is at the end of the type. However, one disadvantage is that sizeof(BigFloat) == 40 while sizeof(mpfr_t) == 32, and if one wanted to pass a C-allocated array of big floats via a pointer, then Julia’s ccall would issue a segmentation fault to any routine accessing past the first entry. This can be worked around with a dummy Julia struct that exactly replicates mpfr_t and a conversion to BigFloat, but then one may ask why it must be this way.

After presenting a strong argument against non-conforming BigFloat structs, in my opinion, I hope to hear the arguments for inclusion of the extra field.

2

BigFloat is a GC managed structure so there is just no valid usecase of “pass a C-allocated array of big floats via a pointer”, no matter now the memory management of the extra memory works. The “argument against non-conforming BigFloat structs” breaks down right there.

3

Ok, assume that Julia retains control of the memory management of BigFloat.

Currently, one cannot allocate an array of BigFloats in Julia and pass them to a C function as an mpfr_t *.

4

Correct, and that’s unrelated to the object size. You can pass it as an array of mpfr_t**

1 Like
5

Sorry, silly follow-up question. Why does this code crash?

Richards-MBP-2:~ Mikael$ julia
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.2.1-pre.0 (2019-08-20)
 _/ |\__'_|_|_|\__'_|  |  release-1.2/93929550b6* (fork: 138 commits, 135 days)
|__/                   |

julia> import Base: unsafe_convert

julia> import Base.GMP: Limb

julia> import Base.MPFR: BigFloat, _BigFloat

julia> """
           mpfr_t <: AbstractFloat

       A Julia struct that exactly matches `mpfr_t`.
       """
       struct mpfr_t <: AbstractFloat
           prec::Clong
           sign::Cint
           exp::Clong
           d::Ptr{Limb}
       end
mpfr_t

julia> function mpfr_t(x::BigFloat)
           d = x._d
           d′ = GC.@preserve d unsafe_string(pointer(d), sizeof(d)) # creates a definitely-new String
           mpfr_t(x.prec, x.sign, x.exp, pointer(d′))
       end
mpfr_t

julia> function BigFloat(x::mpfr_t)
           nb = ccall((:mpfr_custom_get_size,:libmpfr), Csize_t, (Clong,), precision(BigFloat))
           nb = (nb + Core.sizeof(Limb) - 1) ÷ Core.sizeof(Limb) # align to number of Limb allocations required for this
           str = unsafe_string(Ptr{UInt8}(x.d), nb * Core.sizeof(Limb))
           _BigFloat(x.prec, x.sign, x.exp, str)
       end
BigFloat

julia> A = Matrix{BigFloat}(I, 2, 2)
2×2 Array{BigFloat,2}:
 1.0  0.0
 0.0  1.0

julia> B = mpfr_t.(A)
2×2 Array{mpfr_t,2}:
                    mpfr_t(256, 1, 1, Ptr{UInt64} @0x0000000118612b58)  mpfr_t(256, 1, -9223372036854775807, Ptr{UInt64} @0x0000000118612c58)
 mpfr_t(256, 1, -9223372036854775807, Ptr{UInt64} @0x0000000118612bd8)                     mpfr_t(256, 1, 1, Ptr{UInt64} @0x0000000118612c98)

julia> C = BigFloat.(B)
2×2 Array{BigFloat,2}:
/Applications/julia/deps/srccache/mpfr-4.0.2/src/get_str.c:157: MPFR assertion failed: size_s1 >= m

signal (6): Abort trap: 6
in expression starting at REPL[9]:0
__pthread_kill at /usr/lib/system/libsystem_kernel.dylib (unknown line)
Allocations: 8624618 (Pool: 8620748; Big: 3870); GC: 19
Abort trap: 6
Richards-MBP-2:~ Mikael$
6

If you are asking why your code could fail then it’s because the mpfr_t you create are invalid before you even return them since d′ can basically be a garbage pointer. There’s no way you can fix it as explained above (the same reason the additional member is there in the first place.)

If you are asking why it crashes in the way it does. I don’t know. It happens in the printing and is likely caused by whatever invalid memory you happens to feed mpfr …

P.S. There’s also no such garantee as “creates a definitely-new String”. Any such properties are compiler implementation detail.

7

Note that the code comment was copied from line 1031 in mpfr.jl Base.deepcopy_internal(x::BigFloat, stackdict::IdDict).

8

Base indeed contain quite a few questionable use/abuse of String since it’s currently the only variable size type…

9

Ok, I don’t know if this is worth filing an issue in base, but since Matrix{BigFloat}(I, m, n) calls fill!, changing one entry affects others:

julia> A = Matrix{BigFloat}(I, 4, 4)
4×4 Array{BigFloat,2}:
 1.0  0.0  0.0  0.0
 0.0  1.0  0.0  0.0
 0.0  0.0  1.0  0.0
 0.0  0.0  0.0  1.0

julia> A[1].exp = 0; A[2].sign = -1; A
4×4 Array{BigFloat,2}:
  0.50  -0.0   -0.0   -0.0 
 -0.0    0.50  -0.0   -0.0 
 -0.0   -0.0    0.50  -0.0 
 -0.0   -0.0   -0.0    0.50
10

No, you are using private API. BigFloat are not supposed to be mutated.