# @spawn map(x -\> f(x), collection) only returns Task (Done)

**URL:** https://discourse.julialang.org/t/spawn-map-x-f-x-collection-only-returns-task-done/89575
**Category:** New to Julia
**Tags:** multithreading, map
**Created:** [October 31, 2022, 5:42pm UTC](https://discourse.julialang.org/t/spawn-map-x-f-x-collection-only-returns-task-done/89575 "2022-10-31T17:42:42Z")
**Posts on this page:** 6
**Page:** 1

<div class="post-metadata">

### Author: ![JohnS](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/johns/32/38331_2.png) [@JohnS](https://discourse.julialang.org/u/JohnS)
#### Post date: [October 31, 2022, 5:42pm UTC](https://discourse.julialang.org/t/spawn-map-x-f-x-collection-only-returns-task-done/89575/1 "2022-10-31T17:42:42Z")

</div>

Hello!

I want to multiply a series of matrices by a vector, and it looks like @spawn is the fastest multithreading process. Unfortunately, the below MWE only returns Task Done.

```julia
x = collect(1:1:10)
y = x .+ x

vecmat = [[x+y for x in x, y in y] for i=1:length(x)]

newvecs = @spawn map(m -> m * x, vecmat)

```

The below works, but seems non-Julian.

```julia
x = collect(1:1:10)
y = x .+ x

vecmat = [[x+y for x in x, y in y] for i=1:length(x)]

newvecs = []

@spawn push!(newvecs, map(m -> m * x, vecmat))

```

Any thoughts on making @spawn and map return the value instead of Task Done in a Julian way?  
Or an alternative solution that’s just as fast?

Thanks for your help, keen for the education!

---

<div class="post-metadata">

### Author: ![bertschi](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bertschi/32/33462_2.png) [@bertschi](https://discourse.julialang.org/u/bertschi)
#### Post date: [October 31, 2022, 6:02pm UTC](https://discourse.julialang.org/t/spawn-map-x-f-x-collection-only-returns-task-done/89575/2 "2022-10-31T18:02:40Z")

</div>

`@spawn` is asynchronous and thus cannot immediately return a useful result. Instead, it spawns a new task doing the work and returns a handle to the task which can continue running.  
In order to retrieve the result you can use `fetch` which blocks/waits until the task is done and fetches the result, i.e.,

```julia
task = @spawn map(m -> m * x, vecmat)
newvecs = fetch(task)

```

Alternatively, you could have your tasks write the result onto a channel and get them from there, see the [manual section](https://docs.julialang.org/en/v1/manual/asynchronous-programming/#Communicating-with-Channels) for more details.

---

<div class="post-metadata">

### Author: ![JohnS](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/johns/32/38331_2.png) [@JohnS](https://discourse.julialang.org/u/JohnS)
#### Post date: [October 31, 2022, 6:21pm UTC](https://discourse.julialang.org/t/spawn-map-x-f-x-collection-only-returns-task-done/89575/3 "2022-10-31T18:21:39Z")

</div>

Thanks for the explanation @bertschi, got it!

---

<div class="post-metadata">

### Author: ![mikmoore](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mikmoore/32/31109_2.png) [@mikmoore](https://discourse.julialang.org/u/mikmoore)
#### Post date: [October 31, 2022, 9:20pm UTC](https://discourse.julialang.org/t/spawn-map-x-f-x-collection-only-returns-task-done/89575/4 "2022-10-31T21:20:24Z")

</div>

I’ll often write a parallel map as something like

```julia
results = map(x -> @spawn foo(x), inputs) .|> fetch

```

So the `map` returns a collection of `Task`s, which is then broadcasted into `fetch` to wait for and retrieve each of the results into a new collection.

---

<div class="post-metadata">

### Author: ![JohnS](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/johns/32/38331_2.png) [@JohnS](https://discourse.julialang.org/u/JohnS)
#### Post date: [November 1, 2022, 12:49pm UTC](https://discourse.julialang.org/t/spawn-map-x-f-x-collection-only-returns-task-done/89575/5 "2022-11-01T12:49:23Z")

</div>

Very elegant, cheers @mikmoore!

---

<div class="post-metadata">

### Author: ![dave.f.kleinschmidt](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dave.f.kleinschmidt/32/55_2.png) [@dave.f.kleinschmidt](https://discourse.julialang.org/u/dave.f.kleinschmidt)
#### Post date: [November 1, 2022, 2:46pm UTC](https://discourse.julialang.org/t/spawn-map-x-f-x-collection-only-returns-task-done/89575/6 "2022-11-01T14:46:23Z")

</div>

If you want parallel (multiprocess) map, check out [`pmap`](https://docs.julialang.org/en/v1/stdlib/Distributed/#Distributed.pmap) (discussed in [this section of the manual](https://docs.julialang.org/en/v1/manual/distributed-computing/#Parallel-Map-and-Loops)). It provides a nice wrapper around this pattern that allows you to have some control of scheduling, batching, etc.

There’s also `asyncmap` which spawns LOCAL tasks (e.g., green threads) on the calling process, and is good for things like IO/network access where you want to make many requests that have “downtime” between initiation and completion where julia itself isn’t doing anything (e.g., load a bunch of small objects from s3).

And finally, there are a whole [family of packages built on Transducers.jl](https://github.com/JuliaFolds/Transducers.jl#related-packages) which provide various abstractions for things like this that can be executed in parallel that are “execution engine agnostic” (can be done locally in one process, with multithreading, or with multiprocessing).
