### Author:  [@marius311](https://discourse.julialang.org/u/marius311)
#### Post date: [January 21, 2021, 2:38am UTC](https://discourse.julialang.org/t/dynamically-schedule-work-onto-pmap-worker-pool/53701/1 "2021-01-21T02:38:49Z")
I’ve got `N` chunks of equally long-running work I want to compute across ` # of workers
@spawnat :any begin
println("starting unique chunk")
sleep(2)
end
pmap(1:N-1) do i
println("starting chunk $i")
sleep(2)
end
```
The problem is that this seems to sometime double up on a worker, rather use all (in this case) 4 workers. E.g. here’s a typical output:
```julia
From worker 4: starting unique chunk
From worker 4: starting chunk 1
From worker 3: starting chunk 2
From worker 2: starting chunk 3
```
I’ve figured out that it seems to work if I use `pmap` in both cases:
```julia
@sync begin
@async pmap(1:1) do _
println("starting unique chunk")
sleep(2)
end
pmap(1:3) do i
println("starting chunk $i")
sleep(2)
end
end
```
But I feel like there must be a cleaner way to do this. Any ideas? Thanks.
---
### Author:  [@greg\_plowman](https://discourse.julialang.org/u/greg_plowman)
#### Post date: [January 21, 2021, 4:00am UTC](https://discourse.julialang.org/t/dynamically-schedule-work-onto-pmap-worker-pool/53701/2 "2021-01-21T04:00:28Z")
Maybe you could use [WorkerPools](https://docs.julialang.org/en/v1/stdlib/Distributed/#Distributed.WorkerPool)?
```julia
using Distributed
N = 12 # chunks
M = 4 # workers
addprocs(M)
wpids = workers()
uniquePool = WorkerPool(wpids[1:1])
standardPool = WorkerPool(wpids[2:end])
pmap(uniquePool, 1:1) do i
println("starting unique chunk")
sleep(2)
end
pmap(standardPool, 2:N) do i
println("starting chunk $i")
sleep(2)
end
```
---
### Author:  [@marius311](https://discourse.julialang.org/u/marius311)
#### Post date: [January 21, 2021, 5:40am UTC](https://discourse.julialang.org/t/dynamically-schedule-work-onto-pmap-worker-pool/53701/3 "2021-01-21T05:40:58Z")
Thanks for the suggestion. The problem with this though is that even if I launch these with `@async`, after finishing the unique chunk, `wpids[1]` sits idle when what I want is that its helping finish the other chunks.
---
### Author:  [@greg\_plowman](https://discourse.julialang.org/u/greg_plowman)
#### Post date: [January 21, 2021, 7:31am UTC](https://discourse.julialang.org/t/dynamically-schedule-work-onto-pmap-worker-pool/53701/4 "2021-01-21T07:31:47Z")
Well, I think that’s a little easier:
```julia
using Distributed
N = 12 # chunks
M = 4 # workers
addprocs(M)
pmap(1:N) do i
if i == 1
println("starting unique chunk")
sleep(2)
else
println("starting chunk $i")
sleep(2)
end
end
```
---
### Author:  [@marius311](https://discourse.julialang.org/u/marius311)
#### Post date: [January 21, 2021, 8:00am UTC](https://discourse.julialang.org/t/dynamically-schedule-work-onto-pmap-worker-pool/53701/5 "2021-01-21T08:00:51Z")
Yea, its definitely not impossible, just that code organization wise its awkward to do in my case. Was hoping to find a way to just queue up one work chunk in a different part of the code from all the rest.
---
### Author:  [@DrChainsaw](https://discourse.julialang.org/u/DrChainsaw)
#### Post date: [January 21, 2021, 10:44am UTC](https://discourse.julialang.org/t/dynamically-schedule-work-onto-pmap-worker-pool/53701/6 "2021-01-21T10:44:11Z")
It seems you can push idle/new workers to an existing pool and they will be used by an ongoing computation. No idea if this is ‘safe’ or ‘correct’ though.
---
### Author:  [@marius311](https://discourse.julialang.org/u/marius311)
#### Post date: [January 22, 2021, 1:13am UTC](https://discourse.julialang.org/t/dynamically-schedule-work-onto-pmap-worker-pool/53701/7 "2021-01-22T01:13:24Z")
Thanks, yea it looks there’s a form of `remotecall*(::AbstractWorkerPool)` and this is exactly what it does. So I think this is the “cleanest” way to do what I was after:
```julia
pool = default_worker_pool()
@sync begin
@async remotecall_fetch(pool) do
println("starting unique chunk")
sleep(2)
end
@async pmap(pool, 1:N-1) do i
println("starting chunk $i")
sleep(2)
end
end
```