# Memory error and garbage collection (Circuitscape)

**URL:** <https://discourse.julialang.org/t/memory-error-and-garbage-collection-circuitscape/87022>\
**Category:** New to Julia\
**Tags:** question\
**Created:** [September 9, 2022, 11:20pm UTC](https://discourse.julialang.org/t/memory-error-and-garbage-collection-circuitscape/87022 "2022-09-09T23:20:37Z")\
**Posts on this page:** 16\
**Page:** 1

<div class="post-metadata">

**Author:** ![icancellare](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/icancellare/32/42540_2.png) [@icancellare](https://discourse.julialang.org/u/icancellare)\
**Post date:** [September 9, 2022, 11:20pm UTC](https://discourse.julialang.org/t/memory-error-and-garbage-collection-circuitscape/87022/1 "2022-09-09T23:20:37Z")

</div>

Been working for weeks with the same code to run batches of models in Circuitscape. The machine has a LOT of memory and available storage, yet I am now getting an out of memory error. I don’t understand the Stacktrace or how to clear temporary data that might be eating up memory? Can someone please explain what the code below indicates and/or how to clear temp data? Does GC.gc() do this?

Super new to Julia!

```julia
ERROR: OutOfMemoryError()
Stacktrace:
  [1] Array
    @ .\boot.jl:457 [inlined]
  [2] spmatmul(A::SparseArrays.SparseMatrixCSC{Float64, Int64}, B::SparseArrays.SparseMatrixCSC{Float64, Int64})
    @ SparseArrays C:\Users\Imogene\AppData\Local\Programs\Julia-1.7.3\share\julia\stdlib\v1.7\SparseArrays\src\linalg.jl:196
  [3] *
    @ C:\Users\Imogene\AppData\Local\Programs\Julia-1.7.3\share\julia\stdlib\v1.7\SparseArrays\src\linalg.jl:166 [inlined]
  [4] _tri_matmul(A::LinearAlgebra.Adjoint{Float64, SparseArrays.SparseMatrixCSC{Float64, Int64}}, B::SparseArrays.SparseMatrixCSC{Float64, Int64}, C::SparseArrays.SparseMatrixCSC{Float64, Int64}, δ::Nothing)
    @ LinearAlgebra C:\Users\Imogene\AppData\Local\Programs\Julia-1.7.3\share\julia\stdlib\v1.7\LinearAlgebra\src\matmul.jl:1132
  [5] _tri_matmul
    @ C:\Users\Imogene\AppData\Local\Programs\Julia-1.7.3\share\julia\stdlib\v1.7\LinearAlgebra\src\matmul.jl:1124 [inlined]
  [6] *
    @ C:\Users\Imogene\AppData\Local\Programs\Julia-1.7.3\share\julia\stdlib\v1.7\LinearAlgebra\src\matmul.jl:1120 [inlined]
  [7] extend_hierarchy!(levels::Vector{AlgebraicMultigrid.Level{SparseArrays.SparseMatrixCSC{Float64, Int64}, SparseArrays.SparseMatrixCSC{Float64, Int64}, LinearAlgebra.Adjoint{Float64, SparseArrays.SparseMatrixCSC{Float64, Int64}}}}, strength::AlgebraicMultigrid.SymmetricStrength{Float64}, aggregate::AlgebraicMultigrid.StandardAggregation, smooth::AlgebraicMultigrid.JacobiProlongation{Float64}, improve_candidates::AlgebraicMultigrid.GaussSeidel{AlgebraicMultigrid.SymmetricSweep}, diagonal_dominance::Bool, keep::Bool, A::SparseArrays.SparseMatrixCSC{Float64, Int64}, B::Vector{Float64}, symmetry::AlgebraicMultigrid.HermitianSymmetry, bsr_flag::Bool)
    @ AlgebraicMultigrid C:\Users\Imogene\.julia\packages\AlgebraicMultigrid\ASpK7\src\aggregation.jl:81
  [8] smoothed_aggregation(A::SparseArrays.SparseMatrixCSC{Float64, Int64}, ::Type{Val{1}}; symmetry::AlgebraicMultigrid.HermitianSymmetry, strength::AlgebraicMultigrid.SymmetricStrength{Float64}, aggregate::AlgebraicMultigrid.StandardAggregation, smooth::AlgebraicMultigrid.JacobiProlongation{Float64}, presmoother::AlgebraicMultigrid.GaussSeidel{AlgebraicMultigrid.SymmetricSweep}, postsmoother::AlgebraicMultigrid.GaussSeidel{AlgebraicMultigrid.SymmetricSweep}, improve_candidates::AlgebraicMultigrid.GaussSeidel{AlgebraicMultigrid.SymmetricSweep}, max_levels::Int64, max_coarse::Int64, diagonal_dominance::Bool, keep::Bool, coarse_solver::Type, kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
    @ AlgebraicMultigrid C:\Users\Imogene\.julia\packages\AlgebraicMultigrid\ASpK7\src\aggregation.jl:37
  [9] smoothed_aggregation (repeats 2 times)
    @ C:\Users\Imogene\.julia\packages\AlgebraicMultigrid\ASpK7\src\aggregation.jl:16 [inlined]
 [10] macro expansion
    @ .\timing.jl:299 [inlined]
 [11] solve(prob::Circuitscape.GraphProblem{Float64, Int64, Circuitscape.AMGSolver}, #unused#::Circuitscape.AMGSolver, flags::Circuitscape.RasterFlags, cfg::Dict{String, String}, log::Bool)
    @ Circuitscape C:\Users\Imogene\.julia\packages\Circuitscape\XpftG\src\core.jl:155
 [12] single_ground_all_pairs (repeats 2 times)
    @ C:\Users\Imogene\.julia\packages\Circuitscape\XpftG\src\core.jl:66 [inlined]
 [13] _pt_file_no_polygons_path(rasterdata::Circuitscape.RasterData{Float64, Int64}, flags::Circuitscape.RasterFlags, cfg::Dict{String, String})
    @ Circuitscape C:\Users\Imogene\.julia\packages\Circuitscape\XpftG\src\raster\pairwise.jl:63
 [14] raster_pairwise(T::Type, V::Type, cfg::Dict{String, String})
    @ Circuitscape C:\Users\Imogene\.julia\packages\Circuitscape\XpftG\src\raster\pairwise.jl:29
 [15] _compute(T::Type, V::Type, cfg::Dict{String, String})
    @ Circuitscape C:\Users\Imogene\.julia\packages\Circuitscape\XpftG\src\run.jl:43
 [16] macro expansion
    @ .\timing.jl:299 [inlined]
 [17] compute(path::String)
    @ Circuitscape C:\Users\Imogene\.julia\packages\Circuitscape\XpftG\src\run.jl:31
 [18] top-level scope
    @ REPL[1]:1

```

---

<div class="post-metadata">

**Author:** ![Oscar\_Smith](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oscar_smith/32/25343_2.png) [@Oscar\_Smith](https://discourse.julialang.org/u/Oscar_Smith)\
**Post date:** [September 9, 2022, 11:29pm UTC](https://discourse.julialang.org/t/memory-error-and-garbage-collection-circuitscape/87022/2 "2022-09-09T23:29:53Z")

</div>

What version of Julia? Also, you should profile to see what is using the memory.

---

<div class="post-metadata">

**Author:** ![nsajko](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nsajko/32/221187_2.png) [@nsajko](https://discourse.julialang.org/u/nsajko)\
**Post date:** [September 9, 2022, 11:35pm UTC](https://discourse.julialang.org/t/memory-error-and-garbage-collection-circuitscape/87022/3 "2022-09-09T23:35:56Z")

</div>

> [@Oscar\_Smith](#):
>
> What version of Julia?

1.7.3, judging by the dir filename.

> [@icancellare](#):
>
> The machine has a LOT of memory

How much?

> [@icancellare](#):
>
> Does GC.gc() do this?

Yes, but you shouldn’t ever have to call it from the REPL.

> [@icancellare](#):
>
> I am now getting an out of memory error.

Have you been using the same REPL session for weeks? This may seem like a stupid question, but you provided very little info.

EDIT: Oscar’s gave a good suggestion, profiling may help with the diagnosis. Here’s the relevant part of the manual: [Profiling · The Julia Language](https://docs.julialang.org/en/v1/manual/profile/)

---

<div class="post-metadata">

**Author:** ![icancellare](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/icancellare/32/42540_2.png) [@icancellare](https://discourse.julialang.org/u/icancellare)\
**Post date:** [September 9, 2022, 11:36pm UTC](https://discourse.julialang.org/t/memory-error-and-garbage-collection-circuitscape/87022/4 "2022-09-09T23:36:39Z")

</div>

Version 1.7.3

---

<div class="post-metadata">

**Author:** ![icancellare](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/icancellare/32/42540_2.png) [@icancellare](https://discourse.julialang.org/u/icancellare)\
**Post date:** [September 9, 2022, 11:37pm UTC](https://discourse.julialang.org/t/memory-error-and-garbage-collection-circuitscape/87022/5 "2022-09-09T23:37:48Z")

</div>

Not a stupid question, I’m a very new user! I have been running models for several weeks, but have intermittently shut down Julia and the machine. I am not sure how to clear REPL (again, amateaur hour over here, learning as I go). Edit: I’ve used Ctrl + L but I’m still getting the outofmemory error.

---

<div class="post-metadata">

**Author:** ![icancellare](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/icancellare/32/42540_2.png) [@icancellare](https://discourse.julialang.org/u/icancellare)\
**Post date:** [September 9, 2022, 11:40pm UTC](https://discourse.julialang.org/t/memory-error-and-garbage-collection-circuitscape/87022/6 "2022-09-09T23:40:02Z")

</div>

Thank you both will try

---

<div class="post-metadata">

**Author:** ![nsajko](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nsajko/32/221187_2.png) [@nsajko](https://discourse.julialang.org/u/nsajko)\
**Post date:** [September 9, 2022, 11:45pm UTC](https://discourse.julialang.org/t/memory-error-and-garbage-collection-circuitscape/87022/7 "2022-09-09T23:45:23Z")

</div>

The latest Julia release, 1.8, has some relevant new features that may be useful to you, so you may wish to try it out now. For example, Profile.Allocs isn’t available with 1.7.3.

---

<div class="post-metadata">

**Author:** ![jmair](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jmair/32/35117_2.png) [@jmair](https://discourse.julialang.org/u/jmair)\
**Post date:** [September 10, 2022, 7:11am UTC](https://discourse.julialang.org/t/memory-error-and-garbage-collection-circuitscape/87022/8 "2022-09-10T07:11:21Z")

</div>

`Ctrl+L` as far as I know, just clears the REPL output screen, and doesn’t do anything with memory.

As long as a reference to a variable exists, that memory won’t be freed, you can accumulate a lot of these variables over time so it’s probably best to restart every so often. If you want to get rid of a variable, you can just assign it to nothing. E.g.

```julia
big_variable=nothing

```

and that should free up some memory.

If you use VS Code with the Julia extension (which I highly recommend), it has a section which lists the variables that you are currently using - very helpful!

---

<div class="post-metadata">

**Author:** ![jdad](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jdad/32/4739_2.png) [@jdad](https://discourse.julialang.org/u/jdad)\
**Post date:** [September 10, 2022, 10:53am UTC](https://discourse.julialang.org/t/memory-error-and-garbage-collection-circuitscape/87022/9 "2022-09-10T10:53:46Z")

</div>

You can use `varinfo()` in the REPL to get info on current variables.

---

<div class="post-metadata">

**Author:** ![icancellare](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/icancellare/32/42540_2.png) [@icancellare](https://discourse.julialang.org/u/icancellare)\
**Post date:** [September 10, 2022, 2:57pm UTC](https://discourse.julialang.org/t/memory-error-and-garbage-collection-circuitscape/87022/10 "2022-09-10T14:57:08Z")

</div>

Thanks! I’m not sure if this applies to my problem however, as I am not coding models in Julia. I have done all of this in R and am using the computer function in the CIrcuitscape package to generate resistance distances. The instructions for Circuitscape are comprised in a .ini file that details the location of the files used to create the matrices. Are there variables created in this process? I have downloaded the new version of Julia and restarted the machine and program, but no luck.

EDIT: I have 429 GB of storage available and 31 GB usable RAM.

---

<div class="post-metadata">

**Author:** ![SteffenPL](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/steffenpl/32/206270_2.png) [@SteffenPL](https://discourse.julialang.org/u/SteffenPL)\
**Post date:** [September 10, 2022, 3:33pm UTC](https://discourse.julialang.org/t/memory-error-and-garbage-collection-circuitscape/87022/11 "2022-09-10T15:33:46Z")

</div>

Maybe you could post the issue also here to get help: [Issues · Circuitscape/Circuitscape.jl · GitHub](https://github.com/Circuitscape/Circuitscape.jl/issues)

It could be that the issue is Circuitscape specific or that it depends on the inputs you use.  
(More information or a “minimal working example” would be useful either way, also for posting an issue on Circuitscape. The error log only shows that during the solve command somehow too much memory is needed, that could be for many different reasons.)

---

<div class="post-metadata">

**Author:** ![Sergey\_Novak](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/sergey_novak/32/37716_2.png) [@Sergey\_Novak](https://discourse.julialang.org/u/Sergey_Novak)\
**Post date:** [September 10, 2022, 3:35pm UTC](https://discourse.julialang.org/t/memory-error-and-garbage-collection-circuitscape/87022/12 "2022-09-10T15:35:29Z")

</div>

Can you share full code?

---

<div class="post-metadata">

**Author:** ![icancellare](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/icancellare/32/42540_2.png) [@icancellare](https://discourse.julialang.org/u/icancellare)\
**Post date:** [September 10, 2022, 3:38pm UTC](https://discourse.julialang.org/t/memory-error-and-garbage-collection-circuitscape/87022/13 "2022-09-10T15:38:37Z")

</div>

The code is very short, I call compute(“Pathway\_to\_file”) where the .ini files have all details for Circuitscape. Example of ini file contents/directions:

(This has been working for weeks)

```julia
Options for advanced mode]
ground_file_is_resistances = True
remove_src_or_gnd = keepall
ground_file = (Browse for a ground point file)
use_unit_currents = False
source_file = (Browse for a current source file)
use_direct_grounds = False

[Mask file]
mask_file = C:\ImogeneTemp\RWork\elevmask150m.asc
use_mask = True
[Calculation options]
low_memory_mode = False
parallelize = False
solver = cg+amg
print_timings = False
preemptive_memory_release = False
print_rusages = False
max_parallel = 1

[Short circuit regions (aka polygons)]
polygon_file = (Browse for a short-circuit region file)
use_polygons = False

[Options for one-to-all and all-to-one modes]
use_variable_source_strengths = False
variable_source_file = None

[Output options]
set_null_currents_to_nodata = False
set_focal_node_currents_to_zero = False
set_null_voltages_to_nodata = False
compress_grids = False
write_cur_maps = False
write_volt_maps = False
output_file = C:\ImogeneTemp\RWork\Precip\PrecipOutput\precip8pairwise.out
write_cum_cur_map_only = False
log_transform_maps = False
write_max_cur_maps = False

[Version]
version = 4.0.5

[Options for reclassification of habitat data]
reclass_file = (Browse for file with reclassification data)
use_reclass_table = False

[Logging Options]
log_level = INFO
log_file = C:\ImogeneTemp\RWork\Precip\precip8PairCircuit.log
profiler_log_file = None
screenprint_log = False

[Options for pairwise and one-to-all and all-to-one modes]
included_pairs_file = (Browse for a file with pairs to include or exclude)
use_included_pairs = False
point_file = C:\ImogeneTemp\RWork\Precip\sitesprecip1.asc

[Connection scheme for raster habitat data]
connect_using_avg_resistances = False
connect_four_neighbors_only = False

[Habitat raster or graph]
habitat_map_is_resistances = True
habitat_file = C:\ImogeneTemp\RWork\Precip\precipcost8.asc

[Circuitscape mode]
data_type = raster
scenario = pairwise
use_64bit_indexing = True

```

---

<div class="post-metadata">

**Author:** ![icancellare](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/icancellare/32/42540_2.png) [@icancellare](https://discourse.julialang.org/u/icancellare)\
**Post date:** [September 10, 2022, 3:40pm UTC](https://discourse.julialang.org/t/memory-error-and-garbage-collection-circuitscape/87022/14 "2022-09-10T15:40:27Z")

</div>

Pasted the ini file contents below in another comment, will post issue where suggested as well.

---

<div class="post-metadata">

**Author:** ![Sergey\_Novak](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/sergey_novak/32/37716_2.png) [@Sergey\_Novak](https://discourse.julialang.org/u/Sergey_Novak)\
**Post date:** [September 10, 2022, 3:53pm UTC](https://discourse.julialang.org/t/memory-error-and-garbage-collection-circuitscape/87022/15 "2022-09-10T15:53:38Z")

</div>

Are you tried use another scenario? For example from this page. [On Solvers and Computation Time · Circuitscape.jl Documentation](https://docs.circuitscape.org/Circuitscape.jl/latest/compute/)  
Also you have `low_memory_mode = False` try change on `True`.

---

<div class="post-metadata">

**Author:** ![Sergey\_Novak](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/sergey_novak/32/37716_2.png) [@Sergey\_Novak](https://discourse.julialang.org/u/Sergey_Novak)\
**Post date:** [September 22, 2022, 6:46am UTC](https://discourse.julialang.org/t/memory-error-and-garbage-collection-circuitscape/87022/16 "2022-09-22T06:46:17Z")

</div>

Did my last answer help you?