Pkg.instantiate() takes a very long time on a remote worker

Specific Domains Julia at Scale
2

So I actually have 2 problems. The first, related to the above, is that my workers do not inherit the correct DEPOT_PATH variable (I have to change this so as to not install julia packages in my user folder on the HPC I use, since we have a “data” folder we should use instead. I have now added:

Distributed.@everywhere begin 
    DEPOT_PATH[1] = "/data/leuven/331/vsc33168/.julia"
    @info "Set DEPOT_PATH = $DEPOT_PATH"
    using Pkg
    Pkg.activate(dirname(@__FILE__))
    Pkg.instantiate()

    include(<path_to_script_containing_functions>)

The second is that Julia does not wait for Pkg.instantiate() to finish before continuing my script and calling using <pkg>, which leads to errors.

1 Like