# Debugging computer crash during ODE solve

**URL:** <https://discourse.julialang.org/t/debugging-computer-crash-during-ode-solve/90009>\
**Category:** Modelling & Simulations\
**Created:** [November 9, 2022, 10:21pm UTC](https://discourse.julialang.org/t/debugging-computer-crash-during-ode-solve/90009 "2022-11-09T22:21:08Z")\
**Posts on this page:** 1\
**Showing post:** 13

<div class="post-metadata">

**Author:** ![dlakelan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dlakelan/32/8491_2.png) [@dlakelan](https://discourse.julialang.org/u/dlakelan)\
**Post date:** [November 14, 2022, 5:28pm UTC](https://discourse.julialang.org/t/debugging-computer-crash-during-ode-solve/90009/13 "2022-11-14T17:28:45Z")

</div>

> [@xtalax](#):
>
> After the solves, RSS drops to 36.9GiB, but a look at the size of all variables seen in the snippet shows them all to be on the order of a few tens of MB each, so I don’t know to what this memory is allocated.

This may be memory fragmentation, and related to similar results I had when reading very large CSV files from the census:

> [@Determining size of DataFrame for memory management](https://discourse.julialang.org/t/determining-size-of-dataframe-for-memory-management/85277/34):
>
> If what you want to do is well known, then subselecting columns makes sense. If you start out wanting to explore what’s in the dataset… sometimes you just want to see what’s in there. But yeah, point taken!

With this issue filed:

> <https://github.com/JuliaLang/julia/issues/42566>
>
> This problem was \[originally reported in the helpdesk Slack\](https://julialang.s…lack.com/archives/C6A044SQH/p1633704159387700) against DataFrames, but I believe I have replicated it just using vectors of vectors.
> 
> OP was using Julia 1.5.0 with DataFrame 1.2.2 on Linux
> I used Julia 1.6.2 on WSL2 Ubuntu
> 
> Just using vectors, everything behaves as expected and julia's allocated memory does not increase:
> \`\`\`julia
> function inner\_df(Nrow,Ncol) # Create small dataframe
> rand(Nrow\*Ncol)
> end
> 
> function outer\_df(N) #Stack small dataframes
> Nrow=76
> Ncol=21
> df = Vector{Float64}(undef,0)
> for i = 1:N
> append!(df,inner\_df(Nrow,Ncol))
> end
> return df
> end
> 
> function iterated(Niter) # Create a large DataFrame many times.
> for i =1:Niter
> println(i)
> @time pan = outer\_df(1000\*10\*10)
> end
> nothing
> end
> 
> 
> \# (code and top lines after the code)
> iterated(1)
> GC.gc()
> \# 12142 user 20 0 1892304 215296 64744 S 0.0 1.3 0:19.41 julia
> iterated(10)
> GC.gc()
> \# 12142 user 20 0 1892416 226740 65412 S 0.0 1.4 1:51.96 julia
> iterated(10)
> GC.gc()
> \# 12142 user 20 0 1892416 227000 65416 S 0.0 1.4 3:25.04 julia
> iterated(20)
> GC.gc()
> \# 12142 user 20 0 1892416 227104 65416 S 0.0 1.4 6:32.83 julia
> \`\`\`
> 
> Using vectors of vectors, there seems to be a memory leak:
> \`\`\`julia
> function inner\_df(Nrow,Ncol) # Create small dataframe
> \[rand(Nrow) for i in 1:Ncol\]
> end
> 
> function outer\_df(N) #Stack small dataframes
> Nrow=76
> Ncol=21
> df=\[Vector{Float64}(undef, 0) for i in 1:Ncol\]
> for i = 1:N
> df2 = inner\_df(Nrow,Ncol)
> for j in 1:Ncol
> append!(df\[j\],df2\[j\])
> end
> end
> return df
> end
> 
> function iterated(Niter) # Create a large DataFrame many times.
> for i =1:Niter
> println(i)
> @time pan = outer\_df(1000\*10\*10)
> end
> nothing
> end
> 
> 
> \# (code and top lines after code)
> iterated(1)
> GC.gc()
> \# 9204 user 20 0 2275056 216848 64844 S 0.0 1.3 0:15.61 julia
> iterated(10)
> GC.gc()
> \# 9204 user 20 0 3339728 952280 64844 S 0.0 5.7 1:06.03 julia
> iterated(10)
> GC.gc()
> \# 9204 user 20 0 3339728 952876 64844 S 0.0 5.7 1:55.75 julia
> iterated(20)
> GC.gc()
> \# 9204 user 20 0 3343824 953112 64964 S 0.0 5.7 3:31.63 julia
> \`\`\`
> 
> Please feel free to close if this is a vagary of Linux memory management.

---

_[View the full topic](https://discourse.julialang.org/t/debugging-computer-crash-during-ode-solve/90009)._