Hello,
I have been trying to get some packages to run on a cluster. I have set the variable JULIA_DEPOT_PATH
to a custom folder where I have installed the required packages and my job script looks as follows:
#==================================================
#PBS -N job_name
#PBS -m abe
#PBS -e <dir_name>
#PBS -o <dir_name>
#PBS -t 0-48
#PBS -l mem=30GB
#PBS -l walltime=2:99:99
#PBS -l nodes=5:ppn=10
export JULIA_DEPOT_PATH=<custom_dir_path>
/dir/julia-1.4.2/bin/julia --machine-file=$PBS_NODEFILE <file_name.jl>
In the file, I have specified using Distributed
and @everywhere
as required. The problem is as follows: some of the packages load (for ex: JLD
and HDF5
load) while other packages do not load (for ex: ITensor
, RandomBasedArrays
). I have played around with the order of loading packages and hence notice that only few load while the others do not. Also, I have used @everywhere using <package_name>
as mentioned in the docs. The error I get is as follows:
ERROR: LoadError: On worker 2:
LoadError: ArgumentError: Package ITensors not found in current path:
- Run `import Pkg; Pkg.add("ITensors")` to install the ITensors package.
The problem seems to be that the packages are not being loaded on the worker. Any pointers on how to fix this would be helpful. Thanks!