Hello everyone,
I’m currently experiencing a significant slowdown when launching Julia with MPI on my cluster. I suspect that the issue might be due to the long HDD access times.
To mitigate this, I am considering copying the Julia program to the SSD-based scratch space shared among the nodes. The scratch space is wiped after every job, so I would like to avoid the overhead of re-precompiling packages each time. Therefore, I want to copy the precompiled environment as well.
Here are my specific questions:
- Which Julia files or directories should I copy to the scratch space to ensure smooth and fast startup?
- Which environment variables need to be set or adjusted when running Julia from the scratch space?
- Are there any best practices or recommendations for managing precompiled packages in this context?
I would greatly appreciate any advice or insights from those who have faced similar challenges.
Thank you in advance!