# BigFloat memory usage

**URL:** <https://discourse.julialang.org/t/bigfloat-memory-usage/30136>\
**Category:** General Usage\
**Created:** [October 21, 2019, 2:12pm UTC](https://discourse.julialang.org/t/bigfloat-memory-usage/30136 "2019-10-21T14:12:54Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![ckoe-bccms](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ckoe-bccms/32/1816_2.png) [@ckoe-bccms](https://discourse.julialang.org/u/ckoe-bccms)\
**Post date:** [October 21, 2019, 2:12pm UTC](https://discourse.julialang.org/t/bigfloat-memory-usage/30136/1 "2019-10-21T14:12:54Z")

</div>

Hello,

I have a problem concerning the memory usage of BigFloats.

I have quite some trouble switching a self-written ODE solver as well as DifferentialEquations.jl (less tests with that) from `Float64` to `BigFloat` because of drastically increasing memory consumption. Calculations which would require only around 1 GByte with `Float64` would suddenly spiral out of control even with 64 bit mantissa BigFloats consuming more than 12 GByte (on a 16 GByte machine). Now, certainly BigFloats must have quite some overhead compared to a Float64, but still.

To better understand what was happening I constructed this MWE which is essentially what a Euler ODE solver would do:

```julia
setprecision(64)
CalcType=BigFloat

function rhs(t::T,x::Array{T,1}) where {T}
  back=zeros(T,3)
  back[1]=x[2]
  back[2]=-sin(x[1])
  back[3]=T(0.2)
  return back
end

function mtest!(out::Array{T,2},dt::T) where {T}
  for ii=2:size(out,2)
    vect=rhs(T(0.0),out[:,ii-1])
    for jj=1:length(vect)
      out[jj,ii]=out[jj,ii-1]+dt*vect[jj]
    end
  end
  return nothing
end

out=zeros(CalcType,3,10000000);
out[:,1]=[CalcType(0.25),CalcType(0.0),CalcType(0.0)]

mtest!(out,CalcType(1//100))

```

Running this in the REPL or, to make measuring memory easier, non-interactive uses roughly 4 GByte of memory according to `/usr/bin/time` and checked with `top`:

```julia
% /usr/bin/time ~/bin/julia-1.0.4/bin/julia mtest_loop.jl 
39.25user 1.68system 0:40.97elapsed 99%CPU (0avgtext+0avgdata 4030852maxresident)k
8inputs+0outputs (0major+1454329minor)pagefaults 0swaps

```

Same with 1.3rc3.

Running `mtest!(out,CalcType(1//100))` twice instead of once gives

```julia
% /usr/bin/time ~/bin/julia-1.0.4/bin/julia mtest_loop.jl   
86.74user 2.40system 1:29.15elapsed 99%CPU (0avgtext+0avgdata 4393876maxresident)k
8inputs+0outputs (0major+2091832minor)pagefaults 0swaps

```

This appears to be quite excessive. Naively one could expect that the array `out` in the MWE uses about `3*10000000*8/1024/1024=228 MByte`, although BigFloats probably have quite some overhead. What would be the expected in-memory size of the `out[]` array above with BigFloats ?

Forcing garbage collection by replacing the `mtest!()` function above with this

```julia
function mtest_gc!(out::Array{T,2},dt::T) where {T}
  for ii=2:size(out,2)
    vect=rhs(T(0.0),out[:,ii-1])
    for jj=1:length(vect)
      out[jj,ii]=out[jj,ii-1]+dt*vect[jj]
    end
    if (mod(ii,100000)==0.0)
      GC.gc()
    end
  end
  return nothing
end

```

reduces memory consumption considerably to only 2.8 GByte but with a huge run-time penalty:

```julia
% /usr/bin/time ~/bin/julia-1.0.4/bin/julia mtest_loop_gc.jl
317.42user 1.38system 5:18.90elapsed 99%CPU (0avgtext+0avgdata 2856636maxresident)k 
8inputs+0outputs (0major+1219701minor)pagefaults 0swaps

```

One can of course change the `CalcType` from `BigFloat` to `Float64` in the original MWE. Total memory usage is then about 430 MByte.

From this I believe that the garbage collector is not doing a very good job for BigFloats. Is this true ? Is there anything I can tune apart from manually forcing garbage collection to stop memory consumption before the machine swaps to death ? This is especially important for DifferentialEquations.jl where it is not really possible to force garbage collection manually.

---

<div class="post-metadata">

**Author:** ![ckoe-bccms](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ckoe-bccms/32/1816_2.png) [@ckoe-bccms](https://discourse.julialang.org/u/ckoe-bccms)\
**Post date:** [October 22, 2019, 2:20pm UTC](https://discourse.julialang.org/t/bigfloat-memory-usage/30136/2 "2019-10-22T14:20:11Z")

</div>

Addendum: I do not see any memory blow up comparable to BigFloats with Double64 (using 690 MByte) from DoubleFloats.jl or Float128 (using 670 MByte) from Quadmath.jl.