I am trying Julia for the first time on a HPC cluster at my university. Julia v0.6 is already installed, I have a few questions: How to install packages on a custom folder in the cluster other than .julia/v0.6, and without internet connection? The cluster is configured with PBS for resource manage…

To address 1, assuming you have a networked file system, you should be able to put your packages in a directory accessible by every node then add prepend!(LOAD_PATH, "/some/path") in a .juliarc.jl file on the head node.

https://docs.julialang.org/en/latest/manual/packages/#Offline-Installation-of-Packages-1

@juliohm I asked a similar question recently. You might find this useful: [image] Package management - an intro? General Usage I see that there is a talk by Stefan Karpinski on Pkg3 package management. Python ‘virtualenv’ is mentioned. Would anyone be kind enough to poin…

@juliohm WRT your second question, an expert will be along in a minute. The examples from ClusterManagers.jl have the paradigm that you start a Julia session, then use ClusterManagers to submit a job (or start processes locally using affinity). That job will be started with the number of cores you …

I recently went through the same learning process on my university’s cluster, and your second question was the hardest part for me. I found the easiest way was to submit to one of the queues using a PBS submit script that tells Julia where to find all the processors via the PBS nodefile. Here’s a …

Thank you @ElOceanografo , that is really helpful already. I can give it a try and get things moving while I digest the workflow, possibly with ClusterManagers.jl.

@ElOceanografo , I am getting an error saying that the file /home/juliohm/.ssh cannot be created. This makes sense because in the cluster I am trying to run, users cannot write to /home. Did anyone had a similar issue? Also, for requesting more than one node on clusters, do we need to install MPI.jl…

The file permission error is something I would ask to your cluster’s admin about. On the cluster I’m using, we’re given write access in out home directory, so I didn’t run into this problem. As to your second question, MPI.jl is not required. You don’t need to use ClusterManagers either if you do…

I asked the cluster’s admin and he mentioned that the cluster doesn’t use SSH for communication, which I think is bad news for me. Julia will only be able to use one node in this case. Please correct me if I am wrong. He also mentioned that a MPI manager is required to have processes running across…

Ah…you may have to use MPI after all, in that case. I would see if you can get any of the MPI.jl examples here to run–you’ll use a qsub submit script that looks something like this: #!/bin/bash #PBS -l nodes=4:ppn=12,walltime=00:05:00 #PBS -N test_julia #PBS -q debug module load openmpi mpirun -n…

Help setting up Julia on a cluster

Specific Domains Julia at Scale

John_Hearns August 23, 2017, 2:37pm 4

@juliohm I asked a similar question recently. You might find this useful:

Topic		Replies	Views
Julia on Cluster with SSH Restriction General Usage question , cluster	18	4067	January 16, 2021
Is ClusterManagers.jl maintained? Or, how to do multi-node calculations in Julia? General Usage question , package	44	2588	July 13, 2024
Ask for official guide for Julia installation, deployment and use on Linux cluster Tooling question , hpc , linux , cluster , server	31	1759	December 1, 2025
ClusterManagers package not connecting for PBS batch system General Usage parallel , cluster	10	2172	April 17, 2018
ClusterManagers.jl hangs on pbs Julia at Scale parallel	9	2369	March 3, 2020

Help setting up Julia on a cluster

Related topics