# Behavior of threads

**URL:** <https://discourse.julialang.org/t/behavior-of-threads/95769>\
**Category:** General Usage\
**Tags:** multithreading\
**Created:** [March 8, 2023, 9:07pm UTC](https://discourse.julialang.org/t/behavior-of-threads/95769 "2023-03-08T21:07:33Z")\
**Posts on this page:** 14\
**Page:** 2

<div class="post-metadata">

**Author:** ![Sukera](https://avatars.discourse-cdn.com/v4/letter/s/ce7236/32.png) [@Sukera](https://discourse.julialang.org/u/Sukera)\
**Post date:** [March 9, 2023, 11:46pm UTC](https://discourse.julialang.org/t/behavior-of-threads/95769/21 "2023-03-09T23:46:18Z")

</div>

> [@PetrKryslUCSD](#):
>
> Each task should work by accessing only a little bit of the array, not the whole thing. In order for it to stay within the range it is supposed to see, I can use the chunk number. If I spawn as many tasks as there are chunks, I can identify the chunk by the task. Does that make sense?

Yes, that makes sense, but why not just pass only the `view` of the global array the task needs, instead of passing in the whole array & indexing inside of each task? That achieves exactly the same thing, without both task local storage and without the need for accessing a hypothetical task id.

I guess it comes down to a difference in style, one the one side just having a global array and indexing into it with whatever scheme you deem good, versus only giving each task access to exactly what it needs, and no more.

But maybe I’m missing something that breaks in your particular use - a concrete example would be appreciated.

---

<div class="post-metadata">

**Author:** ![Sukera](https://avatars.discourse-cdn.com/v4/letter/s/ce7236/32.png) [@Sukera](https://discourse.julialang.org/u/Sukera)\
**Post date:** [March 9, 2023, 11:50pm UTC](https://discourse.julialang.org/t/behavior-of-threads/95769/22 "2023-03-09T23:50:33Z")

</div>

> [@j-fu](#):
>
> I am still puzzled by `threadid()` as a footgun. How and why is this different from `omp_get_thread_num()` ? My mental model for multitrheading comes from OpenMP (admittely also from a couple of years in the past).

The difference is that during execution in OpenMP, you only ever have up to the number of OpenMP threads different threads of execution. In Julia, this is not the case - we have _tasks_, which abstract over an OS/hardware thread and can in fact move between them. I.e., in one moment `threadid()` may give `1`, and in the next it may give `23` (in a cpu that has e.g. 24 hardware threads and julia spawned that many threads for its internal task queue). So when you have a global array and index into it with `threadid()`, you may end up writing to a different location than you did last time, which is a data race, since you presumably didn’t intend to write there. You can find some more information [here](https://juliafolds.github.io/FLoops.jl/dev/explanation/faq/#faq-state-threadid)

Our multithreading is decidedly _NOT_ the same as that of OpenMP, and shouldn’t be confused as such. You will run into issues otherwise.

---

<div class="post-metadata">

**Author:** ![PetrKryslUCSD](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/petrkryslucsd/32/215825_2.png) [@PetrKryslUCSD](https://discourse.julialang.org/u/PetrKryslUCSD)\
**Post date:** [March 10, 2023, 12:35am UTC](https://discourse.julialang.org/t/behavior-of-threads/95769/23 "2023-03-10T00:35:05Z")

</div>

Ok, so here is a sketch of the problem as I perceive it:

Consider a function like this:

```julia
function f(data, a::T) where {T}
    ... process data
    a(result) # and stash the result
    return a
end

```

There may be dozens of other functions like this, `g`, `h`, …  
which may be called either like this

```julia
f(the_whole_data, a)

```

or like this

```julia
@sync begin
Base.Threads.@threads for i in 1:N
    chunk = i-th chunk of the whole data
    @spawn f(the_whole_data[chunk], a)
end
end

```

The functions `f`, `g`, … do not care and should not care  
whether they are run in a serial fashion on the whole data  
or in a parallel fashion on chunks.  
However, the callable object `a` will know what the form of execution is  
and needs to stash the results accordingly: It refers  
to an array which either can be populated in its entirety (serial execution),  
or in sections corresponding to each chunk (parallel execution).  
I do not want to (cannot) redesign the interfaces of the  
functions `f`, `g`, … to pass in additional information regarding  
the chunks. So how will the object `a` know what chunk is being worked on?

---

<div class="post-metadata">

**Author:** ![ffevotte](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ffevotte/32/6587_2.png) [@ffevotte](https://discourse.julialang.org/u/ffevotte)\
**Post date:** [March 10, 2023, 8:02am UTC](https://discourse.julialang.org/t/behavior-of-threads/95769/24 "2023-03-10T08:02:44Z")

</div>

> [@PetrKryslUCSD](#):
>
> So how will the object `a` know what chunk is being worked on?

Couldn’t callable `a` store this information in its own state? (e.g. as a capture if `a` is a closure, or as an additional field if `a` is of a custom callable type)

Something along the lines of:

```julia
function f(input_data, stash::T) where {T}
    # ... process data
    stash(result) # and stash the result
    return a
end

function stash!(output_data, result)
    output_data .= result
end

let
    the_whole_data = ...
    the_whole_output = ...

    Base.Threads.@threads for i in 1:N
        chunk = i-th chunk of the whole data

        # create a specific callable that knows where to stash the results
        # (a closure in this case)
        stash(result) = stash!((@view the_whole_output[chunk]), result)

        # I don't think you need an explicit spawn here
        f(the_whole_data[chunk], stash)
    end
end

```

---

<div class="post-metadata">

**Author:** ![simsurace](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/simsurace/32/30216_2.png) [@simsurace](https://discourse.julialang.org/u/simsurace)\
**Post date:** [March 10, 2023, 8:38am UTC](https://discourse.julialang.org/t/behavior-of-threads/95769/25 "2023-03-10T08:38:31Z")

</div>

> [@PetrKryslUCSD](#):
>
> ```julia
> @sync begin
> Base.Threads.@threads for i in 1:N
> chunk = i-th chunk of the whole data
> @spawn f(the_whole_data[chunk], a)
> end
> end
> 
> ```

Here you are addressing the data by index `i`. That‘s fine because each `i` is a separate task so it‘s not racey. Indexing chunks by `threadid()` would be problematic.

The `@spawn` inside the loop looks strange and is not needed, though.

---

<div class="post-metadata">

**Author:** ![lmiq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lmiq/32/18314_2.png) [@lmiq](https://discourse.julialang.org/u/lmiq)\
**Post date:** [March 10, 2023, 9:14am UTC](https://discourse.julialang.org/t/behavior-of-threads/95769/26 "2023-03-10T09:14:02Z")

</div>

> [@PetrKryslUCSD](#):
>
> `f(the_whole_data[chunk], a)`

Usually one would make a copy of `a` for each chunk and have a reduction function later. Or if `a` is an array of the same length of the data, pass the slice of it the function as well.

That depends on what `a` is exactly. There isn’t a single solution for all types of reductions.

---

<div class="post-metadata">

**Author:** ![ffevotte](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ffevotte/32/6587_2.png) [@ffevotte](https://discourse.julialang.org/u/ffevotte)\
**Post date:** [March 10, 2023, 10:04am UTC](https://discourse.julialang.org/t/behavior-of-threads/95769/27 "2023-03-10T10:04:47Z")

</div>

> [@lmiq](#):
>
> Usually one would make a copy of `a` for each chunk and have a reduction function later. Or if `a` is an array of the same length of the data, pass the slice of it the function as well.

IIUC, in @PetrKryslUCSD’s design `a` is a callable object that performs the reduction. I.e. it is not an array (or slice) in which results are stored, but rather a function that stores the results in (a slice of) the destination array.

---

<div class="post-metadata">

**Author:** ![lmiq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lmiq/32/18314_2.png) [@lmiq](https://discourse.julialang.org/u/lmiq)\
**Post date:** [March 10, 2023, 10:12am UTC](https://discourse.julialang.org/t/behavior-of-threads/95769/28 "2023-03-10T10:12:01Z")

</div>

In that case make a copy of `a` for each chunk and reduce later. I don’t see anyway to `a` know what to update. If that is too large to store, I think one would need to create “chunk-sized” versions of `a` to that, like a structure similar to `a` but containing views of the final output with appropriate size. Also to be reduced later.

---

<div class="post-metadata">

**Author:** ![lmiq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lmiq/32/18314_2.png) [@lmiq](https://discourse.julialang.org/u/lmiq)\
**Post date:** [March 10, 2023, 11:22am UTC](https://discourse.julialang.org/t/behavior-of-threads/95769/29 "2023-03-10T11:22:42Z")

</div>

Something like this:

```julia
using ChunkSplitters

struct A{T}
    output_big_array::T
end
# some function that operates on the array contained in a, given
# a parameter x and an input array v
function (a::A)(x,v) 
    a.output_big_array .= a.output_big_array .+ x * v
    return a
end

# function that updates the array contained in A
function f!(input_big_array, a) 
    a(2.0, input_big_array)
end

input_big_array = ones(1000)
output_big_array = zeros(1000)

function foo(input_big_array, output_big_array; nchunks = Threads.nthreads())
    # create a structure for each chunk, with a slice of the array; r is a range
    a_chunks = [A(@view(output_big_array[r])) for (r,_) in chunks(output_big_array,nchunks) ]
    Threads.@threads for (r, ichunk) in chunks(input_big_array, nchunks)
        f!(@view(input_big_array[r]), a_chunks[ichunk])
    end 
    # create the final structure
    a = A(output_big_array)
    return a
end

```

---

<div class="post-metadata">

**Author:** ![PetrKryslUCSD](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/petrkryslucsd/32/215825_2.png) [@PetrKryslUCSD](https://discourse.julialang.org/u/PetrKryslUCSD)\
**Post date:** [March 10, 2023, 2:53pm UTC](https://discourse.julialang.org/t/behavior-of-threads/95769/30 "2023-03-10T14:53:51Z")

</div>

> [@simsurace](#):
>
> The `@spawn` inside the loop looks strange and is not needed, though.

Sorry, copy-paste error. It should have looked like this:

```julia
@sync begin
for i in 1:N
    chunk = i-th chunk of the whole data
    @spawn f(the_whole_data[chunk], a)
end
end

```

---

<div class="post-metadata">

**Author:** ![pitsianis](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pitsianis/32/26588_2.png) [@pitsianis](https://discourse.julialang.org/u/pitsianis)\
**Post date:** [March 24, 2023, 4:35pm UTC](https://discourse.julialang.org/t/behavior-of-threads/95769/31 "2023-03-24T16:35:44Z")

</div>

Wouldn’t this version

```julia
@sync begin
Base.Threads.@threads for i in 1:N
    chunk = i-th chunk of the whole data
    f(the_whole_data[chunk], a)
end
end

```

be more efficient than this one?

```julia
@sync begin
for i in 1:N
    chunk = i-th chunk of the whole data
    @spawn f(the_whole_data[chunk], a)
end
end

```

since the runtime will do only the necessary spawns instead of `N` spawns if `N` happens to be very large?

I am assuming that the dynamic thread scheduling of Julia is influenced by and does something similar to the `cilk_for` of OpenCilk to schedule loop iterations. `cilk_for` loop iterations are recursively split until the number of iterations reach “grainsize”, a metric of how much work should be in each parallel task. Grainsize can be declared by the programmer, or determined automatically by the Cilk runtime.

---

<div class="post-metadata">

**Author:** ![PetrKryslUCSD](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/petrkryslucsd/32/215825_2.png) [@PetrKryslUCSD](https://discourse.julialang.org/u/PetrKryslUCSD)\
**Post date:** [March 24, 2023, 4:55pm UTC](https://discourse.julialang.org/t/behavior-of-threads/95769/32 "2023-03-24T16:55:38Z")

</div>

I think in the first code fragment `@sync` is not needed, actually?

I elaborated these ideas in an application. The algorithm based on tasks does not work very well. Some of the tasks tend to take quite a bit longer than others. Perhaps you are onto something with the spawning?

> <https://github.com/PetrKryslUCSD/FinEtoolsHeatDiff.jl/blob/d041cd06035547e7bdb1422a94daf006594f1393/examples/steady_state/3-d/Poisson_examples.jl#L336>

The threaded version actually works quite well.

> <https://github.com/PetrKryslUCSD/FinEtoolsHeatDiff.jl/blob/d041cd06035547e7bdb1422a94daf006594f1393/examples/steady_state/3-d/Poisson_examples.jl#L479>

Some scaling data is provided in this thread: [Parallel assembly of a finite element sparse matrix - #19 by PetrKryslUCSD](https://discourse.julialang.org/t/parallel-assembly-of-a-finite-element-sparse-matrix/95947/19)

---

<div class="post-metadata">

**Author:** ![lmiq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lmiq/32/18314_2.png) [@lmiq](https://discourse.julialang.org/u/lmiq)\
**Post date:** [March 24, 2023, 5:27pm UTC](https://discourse.julialang.org/t/behavior-of-threads/95769/33 "2023-03-24T17:27:26Z")

</div>

> [@PetrKryslUCSD](#):
>
> Some of the tasks tend to take quite a bit longer

You can use `ntasks >> nthreads` to deal with that. And if there is any pattern that associates the task length with the index, using the `:scatter` chunking strategy may help as well.

---

<div class="post-metadata">

**Author:** ![PetrKryslUCSD](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/petrkryslucsd/32/215825_2.png) [@PetrKryslUCSD](https://discourse.julialang.org/u/PetrKryslUCSD)\
**Post date:** [March 24, 2023, 5:34pm UTC](https://discourse.julialang.org/t/behavior-of-threads/95769/34 "2023-03-24T17:34:29Z")

</div>

> [@lmiq](#):
>
> You can use `ntasks >> nthreads` to deal with that.

That is an interesting idea. I’ll check it out.

[Previous page](https://discourse.julialang.org/t/behavior-of-threads/95769.md?page=1)