Hello, I am having some issues with permissions getting julia started on a slurm cluster. I could use some assistance, if possible. The issue is explained here
https://stackoverflow.com/questions/67794194/permissions-for-installing-packages-on-julia-in-slurm-cluster
Well… the first thing I would try is to start an interactive job, ie a shell session on one of the compute nodes
srun -n 1 --pty bash -i
Can you open the Julia REPL in this session?
Depending on the cluster architecture, running programs from your home directory is frwoned upon. Usually /home is a small sized filesystem mounted by NFS. There should be a faster filesystem for programs to use. Maybe something like BeeGFS or Lustre
AHA - do you have a cluster where /home on the head node is an NFS mount and /home on the cluster nodes is something entirely different? that would be weird but might explain what you are seeing?
Let’s start that interactive job and find out.
Also please ask if there is a large or fast filesystem you which you should use.
1 Like
Although I did run julia from the home directory for testing, or to try and figure out this permissions problem, it’s not the main way I’m running julia. I used the directions here Installing apps in a Slurm cluster on Compute Engine | Cloud Architecture Center | Google Cloud to install julia in the apps folder.
Starting julia on the node (interactive or not) gives error
ERROR: SystemError (with /home/<my_user_name>/.julia/logs): mkdir: Permission denied
Actually, I do not see that the folder .julia/logs even exists in /home/<my_user_name>. The /home/<my_user_name>/.julia directory only contains folders lib and v0.4
I’m not sure how to proceed
@JDA I am betting that you are using a cluster where the home directory is not the same filesystem as on the cluster login nodes.
On a login node run df .
And do the same when you are running an interactive session
It could also be that for some reason the cluster mounts your home directory as read-only - however I have never seen that in the wild!
Ok, what am I looking for, with regards to the df . output?
Is the filesystem the same size?
mount | grep home might also be useful
I guess i am not sure. In my user home directory df . gives
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/sda1 23436730368 15873240736 7563489632 68% /home
and from a node gives
Filesystem 1K-blocks Used Available Use% Mounted on
cluster-name:/home 23436730368 15873240064 7563490304 68% /home
That looks like the same filesystem. On the cluster login node it is directly mounted as a partition on disk /dev/sda
On the node it looks like an NFS mount of the same
Stupid question - if you start an interactive slurm session can you create a new file by for example:
touch testfile
ALso run the commands ‘whoami’ and ‘id’ on both the cluster head node and during an interactive slurm session
touch testfile works
whoami and id gives me back my username. id also shows I am part of the wheel group
Try mkdir .julia/logs then
Permission denied, any suggestions on how to resolve it?
As you are in the wheel group I think that the filleystem is exported with root squash enabled.
I would ask the cluster administrator to change your group.
It does seem strange though that your primary group is wheel
I’m just a user not an administrator. Since I am the only one using julia on the cluster (that I know of) I was given privileges to install it myself. I will ask about changing the group.