# Don't understand why code runs out of memory and crashes

**URL:** <https://discourse.julialang.org/t/dont-understand-why-code-runs-out-of-memory-and-crashes/123559>\
**Category:** General Usage\
**Tags:** memory, crash\
**Created:** [December 7, 2024, 8:24am UTC](https://discourse.julialang.org/t/dont-understand-why-code-runs-out-of-memory-and-crashes/123559 "2024-12-07T08:24:26Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![Davide98888](https://avatars.discourse-cdn.com/v4/letter/d/958977/32.png) [@Davide98888](https://discourse.julialang.org/u/Davide98888)\
**Post date:** [December 7, 2024, 8:24am UTC](https://discourse.julialang.org/t/dont-understand-why-code-runs-out-of-memory-and-crashes/123559/1 "2024-12-07T08:24:26Z")

</div>

Hello all

We are having a fantastic experience with Julia, but then had a confusing experience. I am doing a simulation study, multithreaded, and eventually Julia runs out of memory and crashes. This happens especially on Linux (on Intel) but can happen on an ARM (M2). When I run the code below, and monitor Julia’s memory use, it keeps climbing, until it crashes. Clearly, is is not doing garbage collection, especially on Linux. But depending on what else code does, same can happen on the M2. We are on the latest version.

best, d

```julia
using Pkg, Revise,StatsBase
function Simulate()
    Simulations=Int(1e7)
    Size=1000
    result = Array{Float64}(undef, Simulations, 1)
    Threads.@threads for i = 1:Simulations
         x = randn(Size)
         s = sort(x)
        result[i, 1] = s[1]
    end
    println(median(result))
end
for i in 1:1000
    println(i)
    Simulate()
end

```

---

<div class="post-metadata">

**Author:** ![Sukera](https://avatars.discourse-cdn.com/v4/letter/s/ce7236/32.png) [@Sukera](https://discourse.julialang.org/u/Sukera)\
**Post date:** [December 7, 2024, 9:01am UTC](https://discourse.julialang.org/t/dont-understand-why-code-runs-out-of-memory-and-crashes/123559/2 "2024-12-07T09:01:02Z")

</div>

Do you also get the problem on 1.10?

Possibly related:

> <https://github.com/JuliaLang/julia/issues/56759>
>
> We're seeing memory leaks in PySR/SymbolicRegression.jl that appear related to J…ulia 1.11's parallel GC. The user (@GoldenGoldy) tried various solutions including heap size hints and other parameter adjustments, but memory usage would steadily climb until OOM crashes occurred after 8-11 hours. The issue vanishes completely when switching to Julia 1.10 - no other changes needed. While we don't yet have a minimal working example in pure Julia, I wanted to raise this as it's causing OOM crashes in production workloads.
> 
> Full reproduction steps and details in: MilesCranmer/PySR#764, including detailed diagnostics on the memory usage \`​​​​

---

<div class="post-metadata">

**Author:** ![Davide98888](https://avatars.discourse-cdn.com/v4/letter/d/958977/32.png) [@Davide98888](https://discourse.julialang.org/u/Davide98888)\
**Post date:** [December 7, 2024, 10:07am UTC](https://discourse.julialang.org/t/dont-understand-why-code-runs-out-of-memory-and-crashes/123559/3 "2024-12-07T10:07:34Z")

</div>

Thanks @Sukera

yes, just tried on 1.10 and it also leaks memory.

d

---

<div class="post-metadata">

**Author:** ![rafael.guerra](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rafael.guerra/32/216610_2.png) [@rafael.guerra](https://discourse.julialang.org/u/rafael.guerra)\
**Post date:** [December 7, 2024, 12:15pm UTC](https://discourse.julialang.org/t/dont-understand-why-code-runs-out-of-memory-and-crashes/123559/4 "2024-12-07T12:15:27Z")

</div>

Use vectors instead of Nx1 matrices:

```julia
result = Array{Float64}(undef, Simulations)

```

and:

```julia
result[i] = s[1]

```

otherwise, keep the same code but compute the median along the long dimension of the matrix:

```julia
median(result, dims=1)

```

---

<div class="post-metadata">

**Author:** ![Salmon](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/salmon/32/22968_2.png) [@Salmon](https://discourse.julialang.org/u/Salmon)\
**Post date:** [December 7, 2024, 12:18pm UTC](https://discourse.julialang.org/t/dont-understand-why-code-runs-out-of-memory-and-crashes/123559/5 "2024-12-07T12:18:27Z")

</div>

Not saying that there shouldnt be a built-in safeguard for this but your code is allocating so much memory that its not suprising it runs out. Each thread is allocation two 1000 element arrays per iteration.

I dont know if this is just meant to be an MWE, but in general its a good idea to avoid so many allocations especially in multithreaded code.

---

<div class="post-metadata">

**Author:** ![Davide98888](https://avatars.discourse-cdn.com/v4/letter/d/958977/32.png) [@Davide98888](https://discourse.julialang.org/u/Davide98888)\
**Post date:** [December 7, 2024, 12:28pm UTC](https://discourse.julialang.org/t/dont-understand-why-code-runs-out-of-memory-and-crashes/123559/6 "2024-12-07T12:28:00Z")

</div>

@Salmon thanks!

I agree. That said, this code sample took out all the computationally intensive calculations, so the overhead is minimal the actual code.

Even then, I just dont know how to preallocate in multithreading code like this. Would need to pre-allocate one vector per core, and be able to use that correctly. Maybe possible, I just don’t know how, I’ll take a closer look at the docs.

and this problem is much worse on Linux than on a Mac under the same Julia version.

best, d

---

<div class="post-metadata">

**Author:** ![Davide98888](https://avatars.discourse-cdn.com/v4/letter/d/958977/32.png) [@Davide98888](https://discourse.julialang.org/u/Davide98888)\
**Post date:** [December 7, 2024, 12:29pm UTC](https://discourse.julialang.org/t/dont-understand-why-code-runs-out-of-memory-and-crashes/123559/7 "2024-12-07T12:29:18Z")

</div>

@rafael.guerra thanks, yes, agree in this sample code, but in the actual code, which does a lot of calculations, it needs to be a matrix, and I just carried it over.

best, d

---

<div class="post-metadata">

**Author:** ![Salmon](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/salmon/32/22968_2.png) [@Salmon](https://discourse.julialang.org/u/Salmon)\
**Post date:** [December 7, 2024, 12:36pm UTC](https://discourse.julialang.org/t/dont-understand-why-code-runs-out-of-memory-and-crashes/123559/8 "2024-12-07T12:36:13Z")

</div>

To avoid most of the allocations, the following should work (I havent checked if it runs)

```julia
using Pkg, Revise,StatsBase
import ChunkSplitters
function Simulate()
    Simulations=round(Int,1e7)
    Size=1000
    buffers = [zeros(Size) for _ in 1:Threads.nthreads()]
    result = zeros(Simulations,1)
    chunks = ChunkSplitters.chunk(1:Simulations,n=Threads.nthreads())

    Threads.@threads for (n_chunk,indices) in enumerate(chunks)
         x = buffers[n_chunk]
         for i = 1:Simulations
            randn!(x)
            sort!(x)
            result[i,1] = s[1] # I guess in this case sorting is technically not needed and could be replaced by maximum, but perhaps you need the sorting for other reasons
         end
    end
    println(median(result))
end
for i in 1:1000
    println(i)
    Simulate()
end

```

PS: note I also replaced `Int(1e7)` by rounding. Probably not absolutely needed in your use-case but seems less error-prone

---

<div class="post-metadata">

**Author:** ![Salmon](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/salmon/32/22968_2.png) [@Salmon](https://discourse.julialang.org/u/Salmon)\
**Post date:** [December 7, 2024, 12:46pm UTC](https://discourse.julialang.org/t/dont-understand-why-code-runs-out-of-memory-and-crashes/123559/9 "2024-12-07T12:46:42Z")

</div>

Maybe not to distract from the more fundamental problem, does it work if you run julia with zhe flag `julia --heap-size-hint=8G` (or whatever amount of memory you have available, say 80 percent of your RAM)  
in theory this should allow for more aggressive GC though im not sure how well it works for multithreaded code

---

<div class="post-metadata">

**Author:** ![lmiq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lmiq/32/18314_2.png) [@lmiq](https://discourse.julialang.org/u/lmiq)\
**Post date:** [December 7, 2024, 1:16pm UTC](https://discourse.julialang.org/t/dont-understand-why-code-runs-out-of-memory-and-crashes/123559/10 "2024-12-07T13:16:34Z")

</div>

> [@Davide98888](#):
>
> ```julia
> x = randn(Size)
> s = sort(x)
> result[i, 1] = s[1]
> 
> ```

You may get a better GC behavior if you put these inside a function.

---

<div class="post-metadata">

**Author:** ![Davide98888](https://avatars.discourse-cdn.com/v4/letter/d/958977/32.png) [@Davide98888](https://discourse.julialang.org/u/Davide98888)\
**Post date:** [December 7, 2024, 1:20pm UTC](https://discourse.julialang.org/t/dont-understand-why-code-runs-out-of-memory-and-crashes/123559/11 "2024-12-07T13:20:40Z")

</div>

Hi @Salmon

Thanks for this, had not seen `ChunkSplitters`, so many wonderful packages to discover.

This code, didn’t quite solve it. Besides the small problems of needing `using Random` and `ChunkSplitters.chunks`, it kept on allocating more memory on both Linux and Mac (and was much slower on the Linux, which usually is faster than the Mac). I think, perhaps it repeats the inner loop for all Simulations for all Threads, and one would need to spit the inner loop into Simulations/number of simulations chunks?

Not sure.

best, d

---

<div class="post-metadata">

**Author:** ![Davide98888](https://avatars.discourse-cdn.com/v4/letter/d/958977/32.png) [@Davide98888](https://discourse.julialang.org/u/Davide98888)\
**Post date:** [December 7, 2024, 1:33pm UTC](https://discourse.julialang.org/t/dont-understand-why-code-runs-out-of-memory-and-crashes/123559/12 "2024-12-07T13:33:18Z")

</div>

Hi @Salmon

That did indeed force it to do GC, but did not solve it.

a) the agressive GC really slows the code when it close to the limit

b) The code crashed previously since the inner simulation loop took memory allocation up to the limit (64G in the machines) but that meant it could not allocate for the post simulation processing. and crashed.

---

<div class="post-metadata">

**Author:** ![eldee](https://avatars.discourse-cdn.com/v4/letter/e/b5a626/32.png) [@eldee](https://discourse.julialang.org/u/eldee)\
**Post date:** [December 7, 2024, 1:35pm UTC](https://discourse.julialang.org/t/dont-understand-why-code-runs-out-of-memory-and-crashes/123559/13 "2024-12-07T13:35:44Z")

</div>

You could reuse the `buffers` and (at least in this example) `result` between different runs of `Simulate` by moving them into the arguments of the method:

```julia
function Simulate!(buffers, result)
    Simulations = size(result, 1)
    Size = length(first(buffers))
    chunks = ...
    ...
end 

function SimulateLoop(runs=1000, Simulations=10^7, Size=1000)
    buffers = [zeros(Size) for _ in 1:Threads.nthreads()]
    result = zeros(Simulations, 1)
    for i in 1:runs
       println(i)
       Simulate!(buffers, result)
    end
end

```

---

<div class="post-metadata">

**Author:** ![lmiq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lmiq/32/18314_2.png) [@lmiq](https://discourse.julialang.org/u/lmiq)\
**Post date:** [December 7, 2024, 1:35pm UTC](https://discourse.julialang.org/t/dont-understand-why-code-runs-out-of-memory-and-crashes/123559/14 "2024-12-07T13:35:52Z")

</div>

> [@Salmon](#):
>
> ```julia
> Threads.@threads for (n_chunk,indices) in enumerate(chunks)
> x = buffers[n_chunk]
> for i = 1:Simulations
> randn!(x)
> sort!(x)
> result[i,1] = s[1] # I guess in this case sorting is technically not needed and could be replaced by maximum, but perhaps you need the sorting for other reasons
> end
> end
> 
> ```

Here `s` is not defined and it is running all simulations repeatedly for each chunk.

---

<div class="post-metadata">

**Author:** ![Davide98888](https://avatars.discourse-cdn.com/v4/letter/d/958977/32.png) [@Davide98888](https://discourse.julialang.org/u/Davide98888)\
**Post date:** [December 7, 2024, 1:38pm UTC](https://discourse.julialang.org/t/dont-understand-why-code-runs-out-of-memory-and-crashes/123559/15 "2024-12-07T13:38:09Z")

</div>

Hi @Imiq

thanks for that, did not work.

---

<div class="post-metadata">

**Author:** ![lmiq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lmiq/32/18314_2.png) [@lmiq](https://discourse.julialang.org/u/lmiq)\
**Post date:** [December 7, 2024, 2:05pm UTC](https://discourse.julialang.org/t/dont-understand-why-code-runs-out-of-memory-and-crashes/123559/16 "2024-12-07T14:05:05Z")

</div>

```julia
using Pkg, Revise,StatsBase
import ChunkSplitters
function Simulate()
    Simulations=round(Int,1e7)
    Size=1000
    result = zeros(Simulations)
    Threads.@threads for c in ChunkSplitters.chunks(1:Simulations,n=Threads.nthreads())
         x = zeros(Size)
         for i in c
            randn!(x)
            sort!(x)
            result[i,1] = x[1] 
         end
    end
    println(median(result))
end
for i in 1:1000
    println(i)
    Simulate()
end

```

I think the above should work (I’m on the phone)

To be completely safe about allocations, you can preallocate the temporary arrays:

```julia
using Pkg, Revise,StatsBase, Random
import ChunkSplitters
function Simulate!(result, xt)
    result .= 0.0
    Threads.@threads for (ic, c) in enumerate(ChunkSplitters.chunks(eachindex(result),n=length(xt)))
         x = xt[ic]
         x .= 0.0
         for i in c
            randn!(x)
            sort!(x)
            result[i] = x[1] 
         end
    end
    println(median(result))
end
function run(; ntasks=Threads.nthreads())
    Simulations=10^7
    Size=1000
    result = zeros(Simulations)
    xt = [zeros(Size) for _ in ntasks]
    for i in 1:1000
        println(i)
        Simulate!(result, xt)
    end
end

```

@Davide98888 if your actual problem has this structure, the above can solve, in practice, the issues you are having. (It does not solve the GC bug)

Or you could parallelize the loop over the simulations, at a higher level.

---

<div class="post-metadata">

**Author:** ![Davide98888](https://avatars.discourse-cdn.com/v4/letter/d/958977/32.png) [@Davide98888](https://discourse.julialang.org/u/Davide98888)\
**Post date:** [December 7, 2024, 2:48pm UTC](https://discourse.julialang.org/t/dont-understand-why-code-runs-out-of-memory-and-crashes/123559/18 "2024-12-07T14:48:45Z")

</div>

Hi @lmiq

Wow, that is very nice! I would never have discovered that. Its about 20% faster that my original code.

It needs also `using Random` since `randn!()` needs that but not `randn()`

But, it still leaks memory, albeit slower than in my code.

d

---

<div class="post-metadata">

**Author:** ![LaurentPlagne](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/laurentplagne/32/10103_2.png) [@LaurentPlagne](https://discourse.julialang.org/u/LaurentPlagne)\
**Post date:** [December 7, 2024, 3:04pm UTC](https://discourse.julialang.org/t/dont-understand-why-code-runs-out-of-memory-and-crashes/123559/19 "2024-12-07T15:04:03Z")

</div>

Correct me if I am wrong but it seems to me that most (all ?) of the answers tend to improve the OP’s code reducing the allocations which is interesting _per se_ but do not address whether the original OP’s MWE actually illustrates a **genuine threading bug/pb with the GC** (It looks like one to me). If it is the case, I guess that an issue should be filled.

---

<div class="post-metadata">

**Author:** ![Sukera](https://avatars.discourse-cdn.com/v4/letter/s/ce7236/32.png) [@Sukera](https://discourse.julialang.org/u/Sukera)\
**Post date:** [December 7, 2024, 3:05pm UTC](https://discourse.julialang.org/t/dont-understand-why-code-runs-out-of-memory-and-crashes/123559/20 "2024-12-07T15:05:38Z")

</div>

How are you measuring the leakage? Just looking at top? I’m curious how your code behaves on my machine, so I’d like to reproduce your methodology. The example your showing does indeed allocate a lot of memory overall, but it shouldn’t leak this. There’s plenty of opportunities here for the GC to run, and eventually reuse those allocations internally.

---

<div class="post-metadata">

**Author:** ![Davide98888](https://avatars.discourse-cdn.com/v4/letter/d/958977/32.png) [@Davide98888](https://discourse.julialang.org/u/Davide98888)\
**Post date:** [December 7, 2024, 3:09pm UTC](https://discourse.julialang.org/t/dont-understand-why-code-runs-out-of-memory-and-crashes/123559/21 "2024-12-07T15:09:32Z")

</div>

@Sukera

I use btop on my Linux and Mac. My production code eventually crashes when it does processing of results after the simulation loop, and btop shows me it happens when Julia has taken all of the 64G on my machines (each simulation loop needs about 0.5G). The linux console gives an out of memory message also.

d

[Next page](https://discourse.julialang.org/t/dont-understand-why-code-runs-out-of-memory-and-crashes/123559.md?page=2)