Issue with nested pmap

Specific Domains Julia at Scale
,
1

Hello, I have this script running in Julia 1.1.
If I run it withing a single process (no workers added) everything works fine. If I add process (e.g. addprocs(3)) nothing gets run and the execution hangs indefinitely

using Distributed
@everywhere function inner_process(task_id)
    task_id^2
end

@everywhere function outer_process(job_id)
    inner_task  = collect(1:2)
    pmap(inner_process, inner_task)
end

function do_job(jobs_list) 
    pmap(outer_process, jobs_list)
end

jobs_list = collect(1:10)
do_job(jobs_list)

Is that the expected behavior? Are pmaps not meant to be nested?

Thanks

2

This also gets stuck with just 1 worker process (addprocs(1)).

I didn’t find any obviously related issues on the Julia Github, so feel free to file one. I definitely don’t see why nested pmap shouldn’t work.

3

Thanks @jpsamaroo, I opened a ticket as your recommended.
https://github.com/JuliaLang/julia/issues/32293

1 Like