That is an awesome resource, thank you for sharing!
One of the reasons I didn’t use the simple (“worker pool”) approach of spawning a task for each “chunk” (which seems to fit this sort of problem well as far as ease of programming):
@sync for i=1:chunksize:last
Threads.@spawn dowork!($i, chunksize)
end
…was that I thought that each @spawn has some overhead, and that since the pieces of work can be relatively small and number of time steps large, spawning once and having each thread do multiple units of work would be faster, especially for smaller problem sizes. Maybe this isn’t an issue, though, or at least doesn’t cause more overhead than using a Channel as an intermediary–I’ll have to optimize both versions and see if there’s any difference.
If I understand correctly, the original code uses the mentioned task farm approach; I’m just not sure what best to do with the main thread while waiting (aside from using a non-blocking take! and doing work manually), since I can’t return from update! until all the chunks have finished updating.