# Multithreading and fetch() type stability

**URL:** <https://discourse.julialang.org/t/multithreading-and-fetch-type-stability/32465>\
**Category:** Performance\
**Tags:** question\
**Created:** [December 19, 2019, 12:07am UTC](https://discourse.julialang.org/t/multithreading-and-fetch-type-stability/32465 "2019-12-19T00:07:41Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![cshen](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cshen/32/217287_2.png) [@cshen](https://discourse.julialang.org/u/cshen)\
**Post date:** [December 19, 2019, 12:07am UTC](https://discourse.julialang.org/t/multithreading-and-fetch-type-stability/32465/1 "2019-12-19T00:07:42Z")

</div>

Hello,  
I was poking around at the new `@spawn` macro and multithreading in general.  
I did a `@code_warntype` on a function were i’d set up some task with `t = @spawn fun(args)`.  
The signature of fun is `(n::Int)::Int` in the definition but in the line where i fetch it back with `fetch(t)` i see a `Main.fetch(t)::Any`

Doesn’t that mean the the whole function is not treated as type stable by the compiler and hence not optimised?  
I would have expected the task return type to be inherited by the return type of `fun(args)` why is that not the case? Is it possible for the `fetch` function to actually return something else?

I tried (naively) to assert the type of `fetch(t)` back to Integers but the problem remains, the code has to be put inside a box.

---

<div class="post-metadata">

**Author:** ![tkf](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tkf/32/17635_2.png) [@tkf](https://discourse.julialang.org/u/tkf)\
**Post date:** [December 19, 2019, 2:02am UTC](https://discourse.julialang.org/t/multithreading-and-fetch-type-stability/32465/2 "2019-12-19T02:02:59Z")

</div>

The `Task` type is not parametrized at all

```julia
julia> typeof(Threads.@spawn nothing)
Task

```

so there is no way to infer exactly what `fetch` returns. I suppose that return type of the expression you passed to spawn has to be encoded in `Task` for `fetch` to be inferrable. However, as Julia interface generally tries to avoid returned value (a task here) to rely on the inference detail, my guess is that it’s unlikely that this would be “fixed.” Besides, given that it is rather expensive to launch a task, I wonder if there is a benefit of making `fetch` inferrable.

---

<div class="post-metadata">

**Author:** ![cshen](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cshen/32/217287_2.png) [@cshen](https://discourse.julialang.org/u/cshen)\
**Post date:** [December 19, 2019, 10:16am UTC](https://discourse.julialang.org/t/multithreading-and-fetch-type-stability/32465/3 "2019-12-19T10:16:30Z")

</div>

As overhead times get lower I wonder if actually having inferability impacts more. I don’t know enough about how the compiler manages type instability to come up with a real example where this could be a problem.

I find it strange that the Task type, present in julia since version 0.1 does not have any kind of type parameterization. Is there a specific theoretical reason that would make it unfeasible to work with? Or just design choices?  
My doubt arises from the fact that when I set out a task I already know what the task is going to give back. And this could be problematic.

Just giving an example, without type assertion the multi threaded Fibonacci function, returns `::Any` on `fetch` and then Any is passed down the rest of the operations. Now being that implementation recursive it has some kind of natural barrier. But if I had more complex operations in a function the `::Any` type would be carried along at every operation generating even more type instability.  
I’m certainly playing devil’s advocate here, but in this scenario I think it could be quite more impacting.

---

<div class="post-metadata">

**Author:** ![tkf](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tkf/32/17635_2.png) [@tkf](https://discourse.julialang.org/u/tkf)\
**Post date:** [December 20, 2019, 3:51am UTC](https://discourse.julialang.org/t/multithreading-and-fetch-type-stability/32465/4 "2019-12-20T03:51:24Z")

</div>

> [@cshen](#):
>
> I find it strange that the Task type, present in julia since version 0.1 does not have any kind of type parameterization. Is there a specific theoretical reason that would make it unfeasible to work with?

I’m interested in this, too. My guess is that, since you have to put all the tasks in a single queue (which presumably wraps something like a `Vector`), the task scheduler has to forget about the type at some level. Also, if you just `wait(task)` on a task it’s not desirable to pay the dynamic dispatch cost for each `typeof(task)`. So, it is desirable to pay the cost only when necessary; e.g., when calling `f` in `f(fetch(task))`.

---

<div class="post-metadata">

**Author:** ![cshen](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cshen/32/217287_2.png) [@cshen](https://discourse.julialang.org/u/cshen)\
**Post date:** [December 22, 2019, 1:11am UTC](https://discourse.julialang.org/t/multithreading-and-fetch-type-stability/32465/5 "2019-12-22T01:11:09Z")

</div>

When `wait`ing for a task to finish i don’t think you actually need the task subtype, If you don’t want to pay the dynamic dispatch price probably you can prevent it with `@nospecialize` but i’m not really sure how that works yet, i might be saying something stupid. The only place where it would be useful is when `fetch`ing the result of the task, since you are actually working with something.

I tried to look for the task definition in the source codes. The `fetch` function simply waits and returns the inner value of the task (which is initialized by a function). What i don’t understand is why the compiler can’t infer the type of the `t.result` value and push it through the fetch.

I couldn’t find the definition of the `Task` type in the source. The docs don’t point to it.

---

<div class="post-metadata">

**Author:** ![bashonubuntu](https://avatars.discourse-cdn.com/v4/letter/b/f19dbf/32.png) [@bashonubuntu](https://discourse.julialang.org/u/bashonubuntu)\
**Post date:** [December 22, 2019, 1:27am UTC](https://discourse.julialang.org/t/multithreading-and-fetch-type-stability/32465/6 "2019-12-22T01:27:20Z")

</div>

I was told a few times on this forum that as long we have function boundaries in the code these kinds of type instabilities are not necessarily a bottleneck.

I’m not sure if that applies here but even in the docs, we are cautioned to not treat every “type instability” as a performance bottleneck. For example, `Base.getindex` is type-unstable and so is `using Distributions; sum(rand(Uniform(), 10))` but these are typically not flagged as performance bottlenecks to worry about.

---

<div class="post-metadata">

**Author:** ![cshen](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cshen/32/217287_2.png) [@cshen](https://discourse.julialang.org/u/cshen)\
**Post date:** [December 22, 2019, 1:33am UTC](https://discourse.julialang.org/t/multithreading-and-fetch-type-stability/32465/7 "2019-12-22T01:33:00Z")

</div>

Yeah, makes sense, I fully realise that the performance loss here is negligible compared with spawning a new task on a new thread, and that i am just nit picking.  
It just spiked my curiosity. For example, i can see why `Base.getindex` is a type-ustable operation, but i can’t really see why `fetch` is too, and i hoped to acquire some insights on it, that’s all 🎉

---

<div class="post-metadata">

**Author:** ![tkf](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tkf/32/17635_2.png) [@tkf](https://discourse.julialang.org/u/tkf)\
**Post date:** [December 22, 2019, 2:30am UTC](https://discourse.julialang.org/t/multithreading-and-fetch-type-stability/32465/8 "2019-12-22T02:30:36Z")

</div>

> [@cshen](#):
>
> the performance loss here is negligible compared with spawning a new task on a new thread

For one-time operations like `fetch` I think I agree. But since `Task` is responsible for task local storage like logger, hypothetical parametrized task type can actually be beneficial for, e.g., compiling away `@debug` (“zero-cost” debug logging). But then this can also be dealt with function barriers.

---

<div class="post-metadata">

**Author:** ![cshen](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cshen/32/217287_2.png) [@cshen](https://discourse.julialang.org/u/cshen)\
**Post date:** [December 22, 2019, 1:20pm UTC](https://discourse.julialang.org/t/multithreading-and-fetch-type-stability/32465/9 "2019-12-22T13:20:20Z")

</div>

I’m not really sure at this point how a parametrized task would be defined. From what i could infer from looking at the source, the `::Any` of `fetch` comes from the inner function `task_result`.  
which is just an instance of `Base.getproperty` which is not type-stable.  
The good thing is that if you run

```julia
for p in propertynames(t)
  println(p, ' ', typeof(Base.getproperty(t, p)))
end

```

the type of the stored values is infered correctly, the only problem is that at compile time the compiler get stuck on `getproperty` and can’t infer further. I don’t know enough about julia to come up with a workaround to be honest.

Also, i found out that setting up a Task is actually a `ccal` and that the “tasked” function is passed with `@nospecialise`. Is this just a standard behaviour to pass arguments to `ccal`s to prevent unnecessary code generation or does it have more behind it?

---

<div class="post-metadata">

**Author:** ![tkf](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tkf/32/17635_2.png) [@tkf](https://discourse.julialang.org/u/tkf)\
**Post date:** [December 23, 2019, 12:20am UTC](https://discourse.julialang.org/t/multithreading-and-fetch-type-stability/32465/10 "2019-12-23T00:20:17Z")

</div>

> [@cshen](#):
>
> The good thing is that if you run
> 
> ```julia
> for p in propertynames(t)
> println(p, ' ', typeof(Base.getproperty(t, p)))
> end
> 
> ```
> 
> the type of the stored values is infered correctly

If by “infered correctly” you mean that actual types are printed, it’s not what inference means. In Julia, a runtime object always has a concrete type.

> [@cshen](#):
>
> I don’t know enough about julia to come up with a workaround to be honest.

Maybe you can try something like this

```julia
thunk = () -> begin
    ... the actual code here ...
end

T = Base._return_type(thunk, Tuple{})

ans = fetch(@spawn thunk()) :: T

```

This doesn’t do anything to the type instability of `fetch` but at least the code accessing `ans` may get statically dispatched.

---

<div class="post-metadata">

**Author:** ![cshen](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cshen/32/217287_2.png) [@cshen](https://discourse.julialang.org/u/cshen)\
**Post date:** [December 23, 2019, 6:57pm UTC](https://discourse.julialang.org/t/multithreading-and-fetch-type-stability/32465/11 "2019-12-23T18:57:51Z")

</div>

> If by “infered correctly” you mean that actual types are printed, it’s not what inference means. In Julia, a runtime object always has a concrete type.

Good correction, my bad. Then most of my observations don’t hold, probably, since i didn’t know that.

> [@tkf](#):
>
> Maybe you can try something like this

This workaround, while making sense, feels much heavier compilation wise.

Edit: i got curious and run a test on the fib function:

```julia
function fib(n::Int64)
  if n > 25
    t = @spawn fib(n-2)
    return fib(n-1) + fetch(t)
  else
    return serial_fib(n)
  end
end

function fib_(n::Int64)
  if n > 25
    return fib_(n-1) + fetch(@spawn fib_(n-2))
  else
    return serial_fib(n)
  end
end

```

the second version does not seem to gain any speed (both return the correct result):

```julia
@btime fib(46) # 2.802 s (301411 allocations: 20.78 MiB)
@btime fib_(46) # 20.232 s (343674 allocations: 21.42 MiB)

```

turns out that `fetch(@spawn f())` does not really work.

Moreover, i find this interesting:

```julia
julia> @code_warntype fib(1)
... etc

2 ─ %6 = Main.:(var"#18#19")::Core.Compiler.Const(var"#18#19", false)
│ %7 = Core.typeof(n)::Core.Compiler.Const(Int64, false)
│ %8 = Core.apply_type(%6, %7)::Core.Compiler.Const(var"#18#19"{Int64}, false)
│ (#18 = %new(%8, n))
│ %10 = #18::var"#18#19"{Int64}

... yadda yadda

```

couldn’t `%10` be the return type of the `fib(n-2)` before it gets wrapped in the task `t` by `@spawn`. If that’s the case then it is even more strange that

```julia
%19 = Main.fib(%18)::Any

```

the whole function is seen as returning `::Any`

At this point, probably the type instability of `fetch` could be felt more in recursive functions, due to the repeated calls to a function that the compiler sees as returning `::Any`, and hence has to check each time.

P.S: Sorry if i’m being so pedantic 😛

---

<div class="post-metadata">

**Author:** ![Mason](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mason/32/2423_2.png) [@Mason](https://discourse.julialang.org/u/Mason)\
**Post date:** [December 23, 2019, 8:30pm UTC](https://discourse.julialang.org/t/multithreading-and-fetch-type-stability/32465/12 "2019-12-23T20:30:42Z")

</div>

> [@cshen](#):
>
> turns out that `fetch(@spawn f())` does not really work.

That’s because `fetch(@spawn ...))` is just running in serial.