I’m struggling to use Distributed
inside Juno without running into what seem to be pre-compilation race conditions. I can’t get @everywhere using
to work (for some packages) outside the default environment, though it works fine in the REPL.
Any tips on how to successfully use Distributed
in Juno?
Here is a simple consistently reproducing example:
I created a new environment, and added Images
into it as the only dependency. Now, inside Juno:
using Pkg
Pkg.activate("tmpenv")
using Images # to precompile
using Distributed
addprocs(2)
@everywhere $Pkg.activate("tmpenv")
@everywhere (using Pkg; println(Pkg.API.Context().env.project_file)) # shows the correct Project.toml path
@everywhere using Images
The last line errors with
From worker 2: ERROR: LoadError: IOError: unlink: resource busy or locked (EBUSY)
From worker 2: Stacktrace:
From worker 2: [1] uv_error at .\libuv.jl:97 [inlined]
From worker 2: [2] unlink(::String) at .\file.jl:888
From worker 2: [3] rm(::String; force::Bool, recursive::Bool) at .\file.jl:268
From worker 2: [4] checkfor_mv_cp_cptree(::String, ::String, ::String; force::Bool) at .\file.jl:306
From worker 2: [5] cp(::String, ::String; force::Bool, follow_symlinks::Bool) at .\file.jl:345
From worker 2: [6] rename(::String, ::String; force::Bool) at .\file.jl:897
From worker 2: [7] compilecache(::Base.PkgId, ::String) at .\loading.jl:1296
From worker 2: [8] _require(::Base.PkgId) at .\loading.jl:1030
From worker 2: [9] require(::Base.PkgId) at .\loading.jl:928
From worker 2: [10] require(::Module, ::Symbol) at .\loading.jl:923
From worker 2: [11] include(::Function, ::Module, ::String) at .\Base.jl:380
From worker 2: [12] include(::Module, ::String) at .\Base.jl:368
From worker 2: [13] top-level scope at none:2
From worker 2: [14] eval at .\boot.jl:331 [inlined]
From worker 2: [15] eval(::Expr) at .\client.jl:467
From worker 2: [16] top-level scope at .\none:3
From worker 2: in expression starting at C:\Users\sternlab\.julia\packages\Images\Khbnr\src\Images.jl:30
I’ve seen this with various other packages too (in the more complex environment I was actually trying to use). versioninfo:
julia> versioninfo()
Julia Version 1.5.2
Commit 539f3ce943 (2020-09-23 23:17 UTC)
Platform Info:
OS: Windows (x86_64-w64-mingw32)
CPU: Intel(R) Core(TM) i7-9800X CPU @ 3.80GHz
WORD_SIZE: 64
LIBM: libopenlibm
LLVM: libLLVM-9.0.1 (ORCJIT, skylake-avx512)
Environment:
JULIA_EDITOR = "C:\Users\sternlab\AppData\Local\atom\app-1.50.0\atom.exe" -a
JULIA_NUM_THREADS = 8