# Trying to launch Julia cluster under PBS

**URL:** <https://discourse.julialang.org/t/trying-to-launch-julia-cluster-under-pbs/47558>\
**Category:** General Usage\
**Tags:** question\
**Created:** [September 30, 2020, 11:23pm UTC](https://discourse.julialang.org/t/trying-to-launch-julia-cluster-under-pbs/47558 "2020-09-30T23:23:13Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![Jeff\_Becker](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jeff_becker/32/10221_2.png) [@Jeff\_Becker](https://discourse.julialang.org/u/Jeff_Becker)\
**Post date:** [September 30, 2020, 11:23pm UTC](https://discourse.julialang.org/t/trying-to-launch-julia-cluster-under-pbs/47558/1 "2020-09-30T23:23:13Z")

</div>

Hi. I used PBS qsub to get four nodes on our cluster. I put the contents of $PBS\_NODEFILE in the array hosts below. After doing “using Distributed”, I tried using addprocs as below, and I don’t understand what to do about the error message. Suggestions welcome. Thanks.

-Jeff

```julia
**julia>** hosts = ["r409i4n16","r433i2n4","r433i2n8","r433i3n11"]

4-element Array{String,1}:

 "r409i4n16"

 "r433i2n4"

 "r433i2n8"

 "r433i3n11"

**julia>** addprocs(hosts,tunnel="true",exename="/u/jcbecker/julia-1.5.1/bin/julia")

 

exception launching on machine r409i4n16 : TypeError(:if, "", Bool, "true")

exception launching on machine r433i2n8 : TypeError(:if, "", Bool, "true")

exception launching on machine r433i3n11 : TypeError(:if, "", Bool, "true")

exception launching on machine r433i2n4 : TypeError(:if, "", Bool, "true")

Int64[]

```

---

<div class="post-metadata">

**Author:** ![simonbyrne](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/simonbyrne/32/19_2.png) [@simonbyrne](https://discourse.julialang.org/u/simonbyrne)\
**Post date:** [October 1, 2020, 8:53pm UTC](https://discourse.julialang.org/t/trying-to-launch-julia-cluster-under-pbs/47558/2 "2020-10-01T20:53:07Z")

</div>

You could try using [ClusterManagers.jl](https://github.com/JuliaParallel/ClusterManagers.jl), though I think the PBS support there might not be working.

---

<div class="post-metadata">

**Author:** ![johnh](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/johnh/32/3615_2.png) [@johnh](https://discourse.julialang.org/u/johnh)\
**Post date:** [October 1, 2020, 9:16pm UTC](https://discourse.julialang.org/t/trying-to-launch-julia-cluster-under-pbs/47558/3 "2020-10-01T21:16:25Z")

</div>

It is the boolean value true or false, not “true”  
try tunnel=true

An SGI ICE cluster with over 400 racks! I am impressed!

---

<div class="post-metadata">

**Author:** ![johnh](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/johnh/32/3615_2.png) [@johnh](https://discourse.julialang.org/u/johnh)\
**Post date:** [October 1, 2020, 9:19pm UTC](https://discourse.julialang.org/t/trying-to-launch-julia-cluster-under-pbs/47558/4 "2020-10-01T21:19:25Z")

</div>

Also I admit my ignorance. Why ssh tunnel to cluster nodes - you should be able to reach them directly?  
Clue stick coming my way…

---

<div class="post-metadata">

**Author:** ![Jeff\_Becker](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jeff_becker/32/10221_2.png) [@Jeff\_Becker](https://discourse.julialang.org/u/Jeff_Becker)\
**Post date:** [October 1, 2020, 9:29pm UTC](https://discourse.julialang.org/t/trying-to-launch-julia-cluster-under-pbs/47558/5 "2020-10-01T21:29:50Z")

</div>

Thanks - that worked

---

<div class="post-metadata">

**Author:** ![Jeff\_Becker](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jeff_becker/32/10221_2.png) [@Jeff\_Becker](https://discourse.julialang.org/u/Jeff_Becker)\
**Post date:** [October 1, 2020, 9:33pm UTC](https://discourse.julialang.org/t/trying-to-launch-julia-cluster-under-pbs/47558/6 "2020-10-01T21:33:31Z")

</div>

The ssh tunnel is needed on our cluster (under PBS) - otherwise the master has trouble connecting to workers.

Also, yesterday I tried addprocs\_pbs in ClusterManagers, but couldn’t get that to work. It seemed to want to add an illegal qsub parameter -t with the node range (1-numnodes) that I was asking for.

---

<div class="post-metadata">

**Author:** ![johnh](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/johnh/32/3615_2.png) [@johnh](https://discourse.julialang.org/u/johnh)\
**Post date:** [October 2, 2020, 7:53am UTC](https://discourse.julialang.org/t/trying-to-launch-julia-cluster-under-pbs/47558/7 "2020-10-02T07:53:07Z")

</div>

@Jeff_Becker That ClusterManagers package uses an older version of PBS. I think I have a commen ton the issues page there.

Sorry to be pushy here - why would you need to tunnel within a cluster?  
I have managed three SGI ICE clusters, using PBS.  
And two very large clusters using PBS in the Netherlands.

There is a PBS configuration using a PAM module which prevents users sshing into cluster nodes if you are NOT running a job on them. I think you are using this?

---

<div class="post-metadata">

**Author:** ![Jeff\_Becker](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jeff_becker/32/10221_2.png) [@Jeff\_Becker](https://discourse.julialang.org/u/Jeff_Becker)\
**Post date:** [October 2, 2020, 8:55pm UTC](https://discourse.julialang.org/t/trying-to-launch-julia-cluster-under-pbs/47558/8 "2020-10-02T20:55:59Z")

</div>

You are right - the default interface for my PBS nodes was selected incorrectly - if I specify node IPoIB addresses in my host array, it works fine without tunneling. Thanks for pushing 8^)

---

<div class="post-metadata">

**Author:** ![juliohm](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/juliohm/32/215266_2.png) [@juliohm](https://discourse.julialang.org/u/juliohm)\
**Post date:** [October 2, 2020, 9:28pm UTC](https://discourse.julialang.org/t/trying-to-launch-julia-cluster-under-pbs/47558/9 "2020-10-02T21:28:04Z")

</div>

@Jeff_Becker take a look at this guide: [https://github.com/juliohm/julia-distributed-computing](https://github.com/juliohm/julia-distributed-computing)

If it doesn’t work for your PBS cluster, please open an issue. The idea is to centralize all issues related to distributed computing in a single place where people can suggest improvements and update the instructions.