# Garbage collection not triggering on SLURM cluster

**URL:** <https://discourse.julialang.org/t/garbage-collection-not-triggering-on-slurm-cluster/95675>\
**Category:** Julia at Scale\
**Tags:** question\
**Created:** [March 7, 2023, 2:41pm UTC](https://discourse.julialang.org/t/garbage-collection-not-triggering-on-slurm-cluster/95675 "2023-03-07T14:41:08Z")\
**Posts on this page:** 1\
**Showing post:** 4

<div class="post-metadata">

**Author:** ![Sevi](https://avatars.discourse-cdn.com/v4/letter/s/c67d28/32.png) [@Sevi](https://discourse.julialang.org/u/Sevi)\
**Post date:** [March 7, 2023, 3:48pm UTC](https://discourse.julialang.org/t/garbage-collection-not-triggering-on-slurm-cluster/95675/4 "2023-03-07T15:48:44Z")

</div>

I’d be really curious to hear about possible solutions as well (I’ve run into a similar problem in the past). To my current understanding, the problem is that Julia “sees” the memory of the whole node, but not the limit that Slurm enforces (might be the wrong interpretation though).

The only solutions I found so far are to either specify more memory (e.g. allocate a whole node for the job) or reduce the allocations in the program.

If you just do matrix operations, it might be worth trying to use as many non-allocating functions as possible, which might also speed up the simulation in general.

* * *

Here are some related discussions/issues that might be helpful:

> [@Garbage collection not aggressive enough on Slurm Cluster](https://discourse.julialang.org/t/garbage-collection-not-aggressive-enough-on-slurm-cluster/61649):
>
> I’ve been using slurm for awhile w/ julia. I’ve only recently been running into issues where I’m running out of memory for long running processes (like 30hrs). The jobs are highly regular, effectively calling the same function over and over on the same model (i.e. a reinforcement learning agent). When I’m testing locally I don’t see any obvious memory leaks. What I’ve hypothesized is that on the cluster julia doesn’t know about the boundary slurm is putting up around memory and thus not garbage …

> <https://github.com/JuliaLang/julia/issues/17987>
>
> Should there be a command line flag to limit heap memory used by a Julia process…?
> Similar to \`-Xms\` and \`-Xmx\` for the JVM.
> 
> This may be useful while running Julia inside containers (docker) with memory limits or 
> running multiple Julia processes on the same machine.

---

_[View the full topic](https://discourse.julialang.org/t/garbage-collection-not-triggering-on-slurm-cluster/95675)._