Trying to understand parallel performance in Turing

[mikkoku]

@Pablo_Marchant

• single threaded:
  1676.549851 seconds (10.56 G allocations: 713.028 GiB, 54.02% gc time, 0.03% compilation time)
• multi threaded (6 threads, same as in the original post):
  1478.391515 seconds (10.53 G allocations: 712.670 GiB, 79.76% gc time, 0.17% compilation time
• distributed (6 processes, same as in the original post):
  527.737194 seconds (46.49 M allocations: 12.626 GiB, 0.84% gc time, 0.61% compilation time)

Since more than 50% of the time is spent in GC, one possible reason for the performance difference between Threads and Distributed could be that Distributed allows GC to run in parallel while GC in any thread stops all threads. The memory allocations could also be a limiting factor in the parallelization speedup. This should not be a problem in a bigger model where more time is spent in computation instead of GC.