I havn’t really looked into multithreading yet, but I do make use of Distributed quite a bit. My workflow usually goes like the following. Suppose you have the following computationally expensive function:
function work()
# a function that does a lot of work (i.e. is computationally expensive)
end
I next addprocs(n) # add n workers, this launches n workers, i.e. julia processes with the --worker flag. Then I mainly use the pmap function to run parallel computations, so something like
results = pmap(1:10) do i
w = work(i)
p = process(w) # will run on the worker process!
end
You have to be careful here. If you have n workers all running work(), you have to make sure there is sufficient memory on your system (and remember all the results from work() are returned to the variable results so you must ensure there is memory left for this collection also.
Another caveat is that pmap is only really useful when work() is computationally expensive to offset the overhead. If the work function is simple and fast, it’s better to use multithreading.