Can anyone provide a tutorial and best practices for running distributed computations on a cluster? In particular using Folds
.
Given that there is so much content available online (much for free), including tons of forums (such as this one and stack overflow), I would personally not buy a book. Also, it seems to mention 1.0. There have been quite some developments between 1.0 and 1.9.
Can you provide some tutorial?
I have programmed some complex computation that I can multithread on my laptop, but now I’m trying to have it run on my computer cluster. I’m having trouble going from multithreading to multiprocessor (I’m using Folds
). I think it’s because of the way I have set up my script, with many of const
s that are scalars and matrices. I am struggling to find a MWE that crashes the same way my real program does.
you can try keyword search on this forum, e.g. HPC. I have no experience with clusters myself. But I did see that there is a julia package MPI.jl, might this be of relevance?
Posting your error message here might help too.