Segfault wrapping GNU multiprecision library MPFR

General Usage
1

I would like to wrap the GNU multiprecision library MPFR to do in-place operations, and I get a segfault on Windows10 (and Julia 1.4.0 and 1.4.1) when running include("test.jl"). The file test.jl (code below) wraps the two library calls mpfr_swap and mpfr_div_2si, the C signature for the second function being long int which on Windows is an Int32. I take inspiration from the Julia code in Base.

I expect the result to be 2.0 (=4.0/2): sometimes I get a segfault, some other times a number that begins with 2.0000… followed by other non-zero digits. Sometimes I get the correct result.

The error occurs fairly consistently when the precision exceeds about 300_000 and disappears when the variables are defined in a different order or either one of the two operations swap! and div_2! are commented out.

This is the content of test.jl:

#void mpfr_swap (mpfr_t x, mpfr_t y)
swap!(x::BigFloat, y::BigFloat) =
    ccall((:mpfr_swap, "libmpfr"), Cvoid, (Ref{BigFloat}, Ref{BigFloat}), x, y)

#int mpfr_div_2si (mpfr_t rop, mpfr_t op1, long int op2, mpfr_rnd_t rnd)
function div_2!(rop::BigFloat, op1::BigFloat, op2::Int32, rnd=Base.MPFR.MPFRRoundNearest)
    ccall((:mpfr_div_2si, "libmpfr"), Int32,
          (Ref{BigFloat}, Ref{BigFloat}, Clong, Base.MPFR.MPFRRoundingMode),
          rop, op1, op2, rnd)
    return rop
end

function func()
    prec = 300_000
    s_prev = BigFloat(1, precision=prec)
    π_prev = BigFloat(4, precision=prec)
    div_2!(π_prev, π_prev, Int32(1))
    π_next = BigFloat(precision=prec)

    swap!(π_prev, π_next)
    π_next
end

a = func();
a

The error message is the following:

This application has requested the Runtime to terminate it in an unusual way.
Please contact the application's support team for more information.

signal (22): SIGABRT
in expression starting at REPL[1]:0
crt_sig_handler at /cygdrive/d/buildbot/worker/package_win64/build/src\signals-win.c:92
raise at C:\WINDOWS\System32\msvcrt.dll (unknown line)
abort at C:\WINDOWS\System32\msvcrt.dll (unknown line)
mpfr_assert_fail at /workspace/srcdir/mpfr-4.0.2/src\mpfr-gmp.c:300
mpfr_get_str_aux at /workspace/srcdir/mpfr-4.0.2/src\get_str.c:157
mpfr_get_str at /workspace/srcdir/mpfr-4.0.2/src\get_str.c:2546
regular_eg at /workspace/srcdir/mpfr-4.0.2/src\vasprintf.c:1250
partition_number at /workspace/srcdir/mpfr-4.0.2/src\vasprintf.c:1850 [inlined]
sprnt_fp at /workspace/srcdir/mpfr-4.0.2/src\vasprintf.c:1921 [inlined]
mpfr_vasnprintf_aux at /workspace/srcdir/mpfr-4.0.2/src\vasprintf.c:2274
__gmpfr_vasprintf at /workspace/srcdir/mpfr-4.0.2/src\printf.c:195 [inlined]
mpfr_asprintf at /workspace/srcdir/mpfr-4.0.2/src\printf.c:187
string_mpfr at .\mpfr.jl:958
_string at .\mpfr.jl:1004 [inlined]
_string at .\mpfr.jl:1006 [inlined]
show at .\mpfr.jl:1016
show at .\multimedia.jl:47
display at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.4\REPL\src\REPL.jl:137
display at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.4\REPL\src\REPL.jl:141
display at .\multimedia.jl:323
jl_apply at /cygdrive/d/buildbot/worker/package_win64/build/src\julia.h:1700 [inlined]
do_apply at /cygdrive/d/buildbot/worker/package_win64/build/src\builtins.c:643
jl_f__apply at /cygdrive/d/buildbot/worker/package_win64/build/src\builtins.c:657 [inlined]
jl_f__apply_latest at /cygdrive/d/buildbot/worker/package_win64/build/src\builtins.c:693
#invokelatest#1 at .\essentials.jl:712 [inlined]
invokelatest at .\essentials.jl:711 [inlined]
print_response at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.4\REPL\src\REPL.jl:161
print_response at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.4\REPL\src\REPL.jl:146
jfptr_print_response_17530.clone_1 at C:\Users\michele.zaffalon\AppData\Local\Programs\Julia\Julia-1.4.1\lib\julia\sys.dll (unknown line)
do_respond at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.4\REPL\src\REPL.jl:729
jfptr_do_respond_14199.clone_1 at C:\Users\michele.zaffalon\AppData\Local\Programs\Julia\Julia-1.4.1\lib\julia\sys.dll (unknown line)
jl_apply at /cygdrive/d/buildbot/worker/package_win64/build/src\julia.h:1700 [inlined]
do_apply at /cygdrive/d/buildbot/worker/package_win64/build/src\builtins.c:643
jl_f__apply at /cygdrive/d/buildbot/worker/package_win64/build/src\builtins.c:657 [inlined]
jl_f__apply_latest at /cygdrive/d/buildbot/worker/package_win64/build/src\builtins.c:693
#invokelatest#1 at .\essentials.jl:712 [inlined]
invokelatest at .\essentials.jl:711 [inlined]
run_interface at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.4\REPL\src\LineEdit.jl:2354
run_frontend at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.4\REPL\src\REPL.jl:1055
run_repl at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.4\REPL\src\REPL.jl:206
#764 at .\client.jl:383
jfptr_#764_8751.clone_1 at C:\Users\michele.zaffalon\AppData\Local\Programs\Julia\Julia-1.4.1\lib\julia\sys.dll (unknown line)
jl_apply at /cygdrive/d/buildbot/worker/package_win64/build/src\julia.h:1700 [inlined]
do_apply at /cygdrive/d/buildbot/worker/package_win64/build/src\builtins.c:643
jl_f__apply at /cygdrive/d/buildbot/worker/package_win64/build/src\builtins.c:657 [inlined]
jl_f__apply_latest at /cygdrive/d/buildbot/worker/package_win64/build/src\builtins.c:693
#invokelatest#1 at .\essentials.jl:712 [inlined]
invokelatest at .\essentials.jl:711 [inlined]
run_main_repl at .\client.jl:367
exec_options at .\client.jl:305
_start at .\client.jl:484
jfptr__start_2087.clone_1 at C:\Users\michele.zaffalon\AppData\Local\Programs\Julia\Julia-1.4.1\lib\julia\sys.dll (unknown line)
unknown function (ip: 00000000004017E1)
unknown function (ip: 0000000000401BD6)
unknown function (ip: 00000000004013DE)
unknown function (ip: 000000000040151A)
BaseThreadInitThunk at C:\WINDOWS\System32\KERNEL32.DLL (unknown line)
RtlUserThreadStart at C:\WINDOWS\SYSTEM32\ntdll.dll (unknown line)
Allocations: 1669868 (Pool: 1669540; Big: 328); GC: 2
get_str.c:157: MPFR assertion failed: size_s1 >= m
2

I have a simplified MWE that consistently crash the interpreter. Here are the steps: the file MWE.jl contains

$ cat MWE.jl
#void mpfr_swap (mpfr_t x, mpfr_t y)
swap!(x::BigFloat, y::BigFloat) =
    ccall((:mpfr_swap, "libmpfr"), Cvoid, (Ref{BigFloat}, Ref{BigFloat}), x, y)

function func()
    prec = 300_000
    a = BigFloat(1.0, precision=prec)
    b = BigFloat(1.0, precision=prec)
    c = BigFloat(1.0, precision=prec)
    swap!(b, c)
    swap!(c, a)
    a
end

func()

I start a new Julia session and as the first command, I include("MWE.jl"): this crashes the REPL with the error message given in the previous post.

I believe I am not doing anything wrong with wrapping the ccall to mpfr_swap, and I am inclined to think this is a bug in Julia’s BigFloat.

My versioninfo():

Julia Version 1.4.1
Commit 381693d3df* (2020-04-14 17:20 UTC)
Platform Info:
  OS: Windows (x86_64-w64-mingw32)
  CPU: Intel(R) Core(TM) i7-6600U CPU @ 2.60GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-8.0.1 (ORCJIT, skylake)
3

I don’t know what is going on, but on my machine with Julia 1.4, I get the message before the crash:

get_str.c:157: MPFR assertion failed: size_s1 >= m

But no crash on Julia 1.5.

4

I see the same message also on 1.4.2.

5

mpfr_swap requires compatibility with the MPFR memory manager. Julia manages BigFloats within it’s normal GC system, which apparently does not provide such compatibility.

2 Likes
6

Thank you for the hint.

Is this because Julia’s BigFloats are constructed by allocating the memory in the constructor like this?

nb = ccall((:mpfr_custom_get_size,:libmpfr), Csize_t, (Clong,), precision)
        nb = (nb + Core.sizeof(Limb) - 1) ÷ Core.sizeof(Limb) # align to number of Limb allocations required for this
        #d = Vector{Limb}(undef, nb)
        d = _string_n(nb * Core.sizeof(Limb))

How about if I was to define them in this way?

mutable struct mpfr_t <: Real
    _prec::Clong
    _sign::Cint
    _exp::Clong
    _d::Ptr{Limb}
    function mpfr_t(; precision)
        b = new()
        ccall((:mpfr_init2, "libmpfr"), Cvoid, (Ref{mpfr_t}, Cint), b, precision) # errors checking skipped
        _f(x) = ccall((:mpfr_clear, "libmpfr"), Cvoid, (Ref{mpfr_t},), x)
        finalizer(_f, b)
    end
end

The memory is now managed by MPFR and deallocated by the finalizer when the Julia object goes out of scope: _d is used as a pointer to memory which Julia never directly access.

All the in-place operations that I have wrapped so far seem to work, although I am wary of using this argument to conclude that mpfr_t is an alternative to Julia’s BigFloats.

7

You could do that: the downside is then that the Julia garbage collector has no knowledge of how much memory MPFR is allocating, and so may not trigger as soon as it ordinarily would, resulting in out-of-memory errors.

Is there a reason you need a swap! function? Could you just do at the syntax level, i.e.

a,b = b,a
1 Like
8

No particular reason. I started wrapping the other methods for in place operations and got carried away.