Runtime increasing as I do more replications

General Usage Performance
1

Hi, I’m a new Julia user and would greatly appreciate your help.

Currently, I’m running a simulation with multiple replications, as below

glob_result = []
for i in 1:n_rep
    Random.seed!(1234+i);
    model_result = @time simulation(param)
    append!(glob_result, [model_result ])
end

As I append the recent result to glob_result, the simulation time increases like below

 1.758594 seconds (3.05 M allocations: 1.112 GiB, 11.81% gc time)
  1.863981 seconds (3.05 M allocations: 1.112 GiB, 10.96% gc time)
... to 
  7.548435 seconds (3.05 M allocations: 1.112 GiB, 63.28% gc time)
  8.421129 seconds (3.05 M allocations: 1.128 GiB, 75.80% gc time)
...
 11.236033 seconds (3.05 M allocations: 1.143 GiB, 80.77% gc time)
 20.993897 seconds (3.05 M allocations: 1.097 GiB, 86.13% gc time)
 44.633841 seconds (3.05 M allocations: 1.112 GiB, 93.60% gc time)
 36.744767 seconds (3.05 M allocations: 1.143 GiB, 92.03% gc time)
 25.545203 seconds (3.05 M allocations: 1.112 GiB, 91.43% gc time)
 43.805692 seconds (3.05 M allocations: 1.112 GiB, 91.81% gc time)
 57.387221 seconds (3.05 M allocations: 1.112 GiB, 94.57% gc time)

The increasing runtime does not appear when I don’t append the simulation result to global result.

I saw some posts advising to use struct. My type of model_result is Vector{Vector{Vector{NamedArray}}}. If I make it into struct type, would it help? Or, could someone help me how I can solve this issue? Thank you very much.

2

You are spending more and more time in garbage collection. You might not be releasing all the memory.

1 Like
3

The vector glob_result is of type Any, and it is constantly resized by append!. Initialize glob_result with the correct size and type, and it should be a lot faster.

1 Like
4

Is it possible to show a minimal code that we can run? I suspect that the issue is somewhere else, at least for me it does not make sense thar the running time of the solver is dependent on how one collects the result outside the function.

1 Like
5

Hi everyone, thank you very much for you reply! All your replies helped me.
Through some research, I learned to define type, allocate memory, and improve original simulation code.
For my code, I changed NamedArrays to Arrays with const. This resolved the runtime increasing issue. I will further optimize my code with specifying type and memory. Thanks!