Computer specific slowdown on multi-threading on computer cluster (Linux)?

Potentially related discussion here: @everywhere takes a very long time when using a cluster - #9 by pfarndt

The problem is likely tied to your cluster’s filesystem - if many processes are trying to access the same set of precompilation cache files at the same time, you may see a significant slowdown if the filesystem’s handling of parallel access is particularly poor (more so if the precompilation cache is hosted in your home directory, which might be accessible to the cluster over a relatively slow network link). @johnh is the resident expert in alleviating this sort of pain - he’ll probably recommend something like copying your .julia/ directory to the node before launching Julia to avoid network contention, or at least making sure .julia/ is located on a fast filesystem (e.g. Lustre).

2 Likes