Memory issues using command line argument

[Sevi]

Welcome to the forum @Atasattari

My first guess would be that there are some repeated allocations which cause the required memory to grow. This is not a problem per se (and it’s impossible to tell without knowing the code that the functions from your include call internally), but I encountered some issues with Slurm and memory management before, similar to what is described in this thread:

Garbage collection not triggering on SLURM cluster

Hi, So we are trying to run parallelised code on our SLURM based cluster however are quickly encountering large scale memory issues. The memory utilisation is much more than a single loop of code will generate suggesting that the garbage collection is not running and that all of the memory is just piling up. Currently we are requesting 16 cores and a total of 128GB of memory on a single node. After 10500 iterations we exceeded our memory limit. Each iteration generates a 121 step vector which a…

I guess this is also why you have this line?

rand(Distributions.Uniform(0,1)) < thr && GC.gc();

Since you report the script is running fine in the first version, here are a few things that might help to limit the memory consumption (although I doubt that the impact will be that large):

• Your function make_data is compiled only for a string, but you could just do the parse(Int, ...) when you call the function, such that it is already an Int when passed.
• Remove the N_events::Integer annotations. Not sure what they are intended to do, but they are not necessary – in the worst case, they will convert some variables to a different type in every iteration of the loop.
• Avoid try catch if possible?