About WARNING: replacing module on @everywhere include

General Usage
1

I have a question about warnings when I tried to use @everywhere include.

# ParallelTest.jl

module ParallelTest

include("MyModule.jl")
using Distributions

export loop

function hello()
    println("Hello")
end

function loop()
    @sync @parallel for i = 1:10
        n = MyModule.f(rand(Uniform(0.0, 1.0)))
        println(n)
        hello()
    end
end

println("loaded")

end

#MyModule.jl

module MyModule

export f

f(x) = 1000x

end

I have these files and on REPL,

> julia
               _
   _       _ _(_)_     |  A fresh approach to technical computing
  (_)     | (_) (_)    |  Documentation: https://docs.julialang.org
   _ _   _| |_  __ _   |  Type "?help" for help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 0.6.2 (2017-12-13 18:08 UTC)
 _/ |\__'_|_|_|\__'_|  |  Official http://julialang.org/ release
|__/                   |  x86_64-apple-darwin14.5.0

julia> addprocs(Sys.CPU_CORES)
4-element Array{Int64,1}:
 2
 3
 4
 5

julia> @everywhere include("ParallelTest.jl")
loaded
WARNING: replacing module Distributions.
WARNING: replacing module Distributions.
WARNING: replacing module Distributions.
WARNING: replacing module Distributions.
	From worker 5:	loaded
	From worker 4:	loaded
	From worker 2:	loaded
	From worker 3:	loaded

julia> using ParallelTest

julia> loop()
	From worker 3:	248.26250919968572
	From worker 3:	Hello
	From worker 3:	675.4954509960418
	From worker 3:	Hello
	From worker 3:	662.744163133056
	From worker 3:	Hello
	From worker 5:	506.503005053337
	From worker 5:	Hello
	From worker 5:	935.7835204817176
	From worker 5:	Hello
	From worker 4:	75.66083075524399
	From worker 4:	Hello
	From worker 4:	528.1776227635611
	From worker 4:	Hello
	From worker 2:	62.40940261002525
	From worker 2:	Hello
	From worker 2:	382.29946951026574
	From worker 2:	Hello
	From worker 2:	164.9343457703678
	From worker 2:	Hello
4-element Array{Future,1}:
 Future(2, 1, 14, #NULL)
 Future(3, 1, 15, #NULL)
 Future(4, 1, 16, #NULL)
 Future(5, 1, 17, #NULL)

Why I get the WARNING: replacing module Distributions.?

2

I have basically the same question. Though, if you look at the manual, I think the answer is that when you call using Distributions on one process, then the module Distributions is loaded on all processes. Hence, every process causes a reload on every other process.

A simpler example is the following code:

addprocs(2)
@everywhere module MyModule
using Distributions
end

which results in

julia> addprocs(2)
2-element Array{Int64,1}:
 2
 3

julia> @everywhere module MyModule
       using Distributions
       end
WARNING: replacing module Distributions.
WARNING: replacing module Distributions.

I use this programming style all the time, and would love to hear of a solution!

EDIT: fix link

3

I don’t have a solution for modules defined at the REPL, but you shouldn’t need @everywhere for modules in your LOAD_PATH.

Create file MyModule.jl with the following:

module MyModule
    using Distributions
end

Place this file in one of the directories in your LOAD_PATH:
load_path_directory/MyModule/src/MyModule.jl

At the REPL, type the following to load MyModule on all processes.

addprocs()
using MyModule

Note that @everywhere should not be necessary.