Unable to create parallel package

General Usage
, ,
1

Hi, I’m trying to make a package in which I use Distributed.jl a bit.

Lets call it Intro.jl, and I have two files in src directory. Intro.jl contains this code:

module Intro
using Distributed 

export mad, meap
include("test.jl")

end

test.jl contains this code:

function mad() 
    pmap(_ -> println("$(myid())", 1:10)
end

@everywhere function lerp(message)
    println("$message from $(myid())")
end

function meap()
    @sync begin
         for pid in workers()
             @async begin
                 remotecall_fetch(lerp, pid,"YO")
             end 
         end 
    end
end

Now, Im able to open the REPL and run using Intro if I only use the mad function.
However when I use @everywhere on some functions I get this error:

1713466106
1713466106791×82 55 KB

Any tips on what I can do this to work?

I use my program as this

  1. julia --project=.
  2. using Distributed
  3. addprocs(4)
  4. using Intro

It then crashes

1 Like
2

Note there is a missing parenthesis here before the comma:

The error stems from the fact that you put @everywhere inside the module so it runs during precompilation which does not make a lot sense. What you should do instead is to remove the @everywhere from test.jl and use it when you load the package such that it is correctly loaded on all workers:
julia --project=.

julia> using Distributed; addprocs(4);
julia> @everywhere using Intro
julia> mad()
      From worker 4:	4
      From worker 3:	3
      From worker 5:	5
      From worker 2:	2
      From worker 4:	4
      From worker 3:	3
      From worker 2:	2
      From worker 4:	4
      From worker 5:	5
      From worker 3:	3
10-element Vector{Nothing}:
 nothing
 nothing
 nothing
 nothing
 nothing
 nothing
 nothing
 nothing
 nothing
 nothing
3 Likes