# How to specify absolute path for include("file.jl") on multiple workers?

**URL:** <https://discourse.julialang.org/t/how-to-specify-absolute-path-for-include-file-jl-on-multiple-workers/14382>\
**Category:** New to Julia\
**Created:** [August 31, 2018, 7:01pm UTC](https://discourse.julialang.org/t/how-to-specify-absolute-path-for-include-file-jl-on-multiple-workers/14382 "2018-08-31T19:01:59Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![thehalfspace](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/thehalfspace/32/7217_2.png) [@thehalfspace](https://discourse.julialang.org/u/thehalfspace)\
**Post date:** [August 31, 2018, 7:01pm UTC](https://discourse.julialang.org/t/how-to-specify-absolute-path-for-include-file-jl-on-multiple-workers/14382/1 "2018-08-31T19:01:59Z")

</div>

I have a bunch of scripts in a folder `JuliaSEM/src/ ` that I am trying to run on a cluster.

My directory structure looks like:

```julia
JuliaSEM folder contents:
    run.jl
    src/
    data/
    and some other folders.

src/ folder contents:
    main.jl
    parameters/defaultParameter.jl
    some other functions

```

My actual path to the main.jl file is `/home/prith/JuliaSEM/src/main.jl`

I have one file `run.jl` in which I include the main function  
`include("src/main.jl")`. This gives me an error on worker 2 `home/prith/src/main.jl : No such file exists`.

If I try `include("JuliaSEM/src/main.jl")`, it gives me an error on the master worker `home/prith/JuliaSEM/JuliaSEM/src/main.jl : No such file exists.`

If I try to include the absolute path, it still gives me error on the master process.

I have not seen this error when I am running it on my local system with 4 nodes (`julia -p 4 run.jl`). I think somehow my directory structure is not translated to multiple nodes, and on other workers it just searches the home directory.

I am running the program on cluster just by using `julia -p 4 run.jl` in my #PBS script file.

For a workaround, I copied all my files into the home directory and it works fine (though there are other errors, which I am working on). Any ideas?

---

<div class="post-metadata">

**Author:** ![SebastianM-C](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/sebastianm-c/32/2480_2.png) [@SebastianM-C](https://discourse.julialang.org/u/SebastianM-C)\
**Post date:** [August 31, 2018, 8:04pm UTC](https://discourse.julialang.org/t/how-to-specify-absolute-path-for-include-file-jl-on-multiple-workers/14382/2 "2018-08-31T20:04:34Z")

</div>

You can use `@ __DIR__ ` in front. See my post here: [Distributed parallelism within packages/applications - #7 by SebastianM-C](https://discourse.julialang.org/t/distributed-parallelism-within-packages-applications/14320/7) for an example

---

<div class="post-metadata">

**Author:** ![jbrea](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jbrea/32/3879_2.png) [@jbrea](https://discourse.julialang.org/u/jbrea)\
**Post date:** [August 31, 2018, 8:05pm UTC](https://discourse.julialang.org/t/how-to-specify-absolute-path-for-include-file-jl-on-multiple-workers/14382/3 "2018-08-31T20:05:32Z")

</div>

In `run.jl` you could try `include(joinpath(@ __DIR__ , "src", "main.jl"))`.

---

<div class="post-metadata">

**Author:** ![baggepinnen](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/baggepinnen/32/693_2.png) [@baggepinnen](https://discourse.julialang.org/u/baggepinnen)\
**Post date:** [September 1, 2018, 7:05am UTC](https://discourse.julialang.org/t/how-to-specify-absolute-path-for-include-file-jl-on-multiple-workers/14382/4 "2018-09-01T07:05:42Z")

</div>

You light find the `include_remote` function from [ParallelDataTransfer.jl/README.md at master · ChrisRackauckas/ParallelDataTransfer.jl · GitHub](https://github.com/ChrisRackauckas/ParallelDataTransfer.jl/blob/master/README.md)  
useful

---

<div class="post-metadata">

**Author:** ![thehalfspace](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/thehalfspace/32/7217_2.png) [@thehalfspace](https://discourse.julialang.org/u/thehalfspace)\
**Post date:** [September 4, 2018, 6:25pm UTC](https://discourse.julialang.org/t/how-to-specify-absolute-path-for-include-file-jl-on-multiple-workers/14382/5 "2018-09-04T18:25:32Z")

</div>

Thanks a lot! This works.

A quick follow up question: have you worked with PBS batch files?

I am doing

```julia
#PBS -l nodes=4:ppn=4,walltime=20:00:100  
julia -p 16 run.jl

```

and it is running without errors, but it seems much slower than the same program I was running on my local system with `julia -p 4 run.jl`. I think I am not adding procs correctly. Any suggestions?

---

<div class="post-metadata">

**Author:** ![markmont](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/markmont/32/3955_2.png) [@markmont](https://discourse.julialang.org/u/markmont)\
**Post date:** [September 4, 2018, 7:29pm UTC](https://discourse.julialang.org/t/how-to-specify-absolute-path-for-include-file-jl-on-multiple-workers/14382/6 "2018-09-04T19:29:15Z")

</div>

If you’re using Flux HPC cluster at the University of Michigan, Julia may not be configured properly there to use multiple nodes in a single job via SSH (another mechanism would be required).

Try this instead:

```nohighlight
#PBS -l nodes=1:ppn=16,walltime=20:00:100   
julia -p 16 run.jl

```

Your local HPC support staff should be able to help you with the specifics of Julia on the cluster you are using.

---

<div class="post-metadata">

**Author:** ![thehalfspace](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/thehalfspace/32/7217_2.png) [@thehalfspace](https://discourse.julialang.org/u/thehalfspace)\
**Post date:** [September 4, 2018, 8:22pm UTC](https://discourse.julialang.org/t/how-to-specify-absolute-path-for-include-file-jl-on-multiple-workers/14382/7 "2018-09-04T20:22:35Z")

</div>

Thanks. Yes, I am using the umich hpc flux.

What you said works fine, that was gonna be my backup plan. Manually running multiple jobs to get multiple simulation results. The only issue is that it will require walltime of 200 hrs.

I will talk to the system admin.

---

<div class="post-metadata">

**Author:** ![thehalfspace](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/thehalfspace/32/7217_2.png) [@thehalfspace](https://discourse.julialang.org/u/thehalfspace)\
**Post date:** [September 4, 2018, 9:56pm UTC](https://discourse.julialang.org/t/how-to-specify-absolute-path-for-include-file-jl-on-multiple-workers/14382/8 "2018-09-04T21:56:37Z")

</div>

I just noticed you work at U of M. I had actually talked to support to get julia/1.0.0 installed. I assumed that julia would natively support cluster management. I am using lsa-flux, standard account.

What would be the other alternative mechanism? I know that `MPI.jl` is not yet available for julia 1.0.0.

I have also not been able to install external packages like `ClusterManager` or `JLD2` on my login node. I am wondering if this is normal for umich flux users.

Thanks a lot!

---

<div class="post-metadata">

**Author:** ![markmont](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/markmont/32/3955_2.png) [@markmont](https://discourse.julialang.org/u/markmont)\
**Post date:** [September 4, 2018, 10:29pm UTC](https://discourse.julialang.org/t/how-to-specify-absolute-path-for-include-file-jl-on-multiple-workers/14382/9 "2018-09-04T22:29:33Z")

</div>

This probably isn’t a good forum in which to discuss institution-specific issues since they won’t apply to most people who are watching this topic. Contacting your local HPC support staff directly would be best.

MPI is the best solution for using multiple nodes on clusters that do not allow SSH between nodes in compute jobs.

A not-good solution which we have used before is to modify the code that runs `ssh name-of-node` in order to start processes on remote nodes to instead run either `pbsdsh -o -h name-of-node` or `mpirun --map-by ppr:1:node -H name-of-node` where `name-of-node` should be replaced with the name of the remote node on which the job is trying to start the process. Note that although the second command uses `mpirun`, it is just used to spawn processes on the remote node and isn’t actually using MPI (message passing) for interprocess communications.

The solution above is not good because neither `pbsdsh` nor `mpirun` is a drop-in replacement for SSH and you may have greater or lesser degrees of success depending on what the software you are using expects; I have not looked to Julia and its packages to see whether either of these would work, and they may not work.

My advice would be to wait until `MPI.jl` is working with Julia 1.0.0. Until then, you can use a single node, request all of the cores on the node, and run `julia -p auto` to start one local Julia worker process for each core your job has access to on the node.

---

<div class="post-metadata">

**Author:** ![thehalfspace](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/thehalfspace/32/7217_2.png) [@thehalfspace](https://discourse.julialang.org/u/thehalfspace)\
**Post date:** [September 4, 2018, 10:46pm UTC](https://discourse.julialang.org/t/how-to-specify-absolute-path-for-include-file-jl-on-multiple-workers/14382/10 "2018-09-04T22:46:34Z")

</div>

Yeah sorry, I was just excited to see someone from U of M on this forum.

Thanks for the advice, I’ll run on a single node for now.