# Raising a flag for specific condition with multithreading

**URL:** <https://discourse.julialang.org/t/raising-a-flag-for-specific-condition-with-multithreading/60535>\
**Category:** General Usage\
**Tags:** multithreading\
**Created:** [May 4, 2021, 5:15pm UTC](https://discourse.julialang.org/t/raising-a-flag-for-specific-condition-with-multithreading/60535 "2021-05-04T17:15:10Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![Gregstrq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gregstrq/32/20620_2.png) [@Gregstrq](https://discourse.julialang.org/u/Gregstrq)\
**Post date:** [May 4, 2021, 5:15pm UTC](https://discourse.julialang.org/t/raising-a-flag-for-specific-condition-with-multithreading/60535/1 "2021-05-04T17:15:10Z")

</div>

Suppose that I have a bunch of threads that process array of some data (apply some function elementwise). Suppose also, that for some entries in array, the processing may fail (for concretness, we can assume that function returns `nothing` or `missing` in this case). After the array is processed, I would like to quickly surmise whether there were any fails or not.

Practically, I could have a flag variable — boolean variable, which is set by default to zero. Should any of the threads encounter a fail with processing, it would set this flag variable to one. After processing is finished, one can inspect the flag to see if any fails occurred or not.

Since several threads may try to set this flag simultaneously, one should prevent the racing condition by adding a lock to this flag variable. On the other hand, it does not matter which thread sets the flag first, and waiting till the lock is released slows the threads.

**Is it safe to use this kind of flag variable without a lock? May be there are some special constructions in base Julia which exist specifically for this purpose?**

---

<div class="post-metadata">

**Author:** ![affans](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/affans/32/11911_2.png) [@affans](https://discourse.julialang.org/u/affans)\
**Post date:** [May 4, 2021, 5:17pm UTC](https://discourse.julialang.org/t/raising-a-flag-for-specific-condition-with-multithreading/60535/2 "2021-05-04T17:17:23Z")

</div>

Not an expert on this by any means, but that seems like a reasonable solution. Something like a `SharedArray` that is visible to all threads. Looking forward to some other solutions that may be posted here.

---

<div class="post-metadata">

**Author:** ![Jeff\_Emanuel](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jeff_emanuel/32/15440_2.png) [@Jeff\_Emanuel](https://discourse.julialang.org/u/Jeff_Emanuel)\
**Post date:** [May 4, 2021, 5:24pm UTC](https://discourse.julialang.org/t/raising-a-flag-for-specific-condition-with-multithreading/60535/3 "2021-05-04T17:24:48Z")

</div>

I try to avoid shared memory solutions any more. I would have the threads write a failure message to a Channel instead.

---

<div class="post-metadata">

**Author:** ![tkf](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tkf/32/17635_2.png) [@tkf](https://discourse.julialang.org/u/tkf)\
**Post date:** [May 4, 2021, 7:58pm UTC](https://discourse.julialang.org/t/raising-a-flag-for-specific-condition-with-multithreading/60535/4 "2021-05-04T19:58:52Z")

</div>

I agree with Jeff\_Emanuel that `Channel`-based solution would be cleaner and easier to get right. If you don’t need to preserve the order of input data, I recommend this as the first solution.

But, to directly answer your question, you can create `failed = Threads.Atomic{Bool}(false)` and then set it via `failed[] = true` in any thread without data races. This is much faster than using a lock.

If you want to terminate the processing as soon there is a failure, you can use [`TakeWhile`](https://juliafolds.github.io/Transducers.jl/dev/reference/manual/#Transducers.TakeWhile) transducer:

```julia
julia> [1, 2, 3, 10, 4, 10] |>
           Map(x -> x == 10 ? nothing : x) |>
           TakeWhile(!isnothing) |>
           tcollect
3-element Vector{Int64}:
 1
 2
 3

```

[FLoops.jl](https://github.com/JuliaFolds/FLoops.jl) supports `break` in parallel loop if you need more control.

---

<div class="post-metadata">

**Author:** ![Gregstrq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gregstrq/32/20620_2.png) [@Gregstrq](https://discourse.julialang.org/u/Gregstrq)\
**Post date:** [May 4, 2021, 8:29pm UTC](https://discourse.julialang.org/t/raising-a-flag-for-specific-condition-with-multithreading/60535/5 "2021-05-04T20:29:35Z")

</div>

By the way, what is the difference between locks and atomic variables?

---

<div class="post-metadata">

**Author:** ![tkf](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tkf/32/17635_2.png) [@tkf](https://discourse.julialang.org/u/tkf)\
**Post date:** [May 4, 2021, 8:43pm UTC](https://discourse.julialang.org/t/raising-a-flag-for-specific-condition-with-multithreading/60535/6 "2021-05-04T20:43:23Z")

</div>

Typically, atomics are directly mapped to hardware instructions and can only deal with a small amount of data (a few bytes). Locks are built on atomics to support arbitrary objects and code.

---

<div class="post-metadata">

**Author:** ![Gregstrq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gregstrq/32/20620_2.png) [@Gregstrq](https://discourse.julialang.org/u/Gregstrq)\
**Post date:** [May 4, 2021, 8:48pm UTC](https://discourse.julialang.org/t/raising-a-flag-for-specific-condition-with-multithreading/60535/7 "2021-05-04T20:48:42Z")

</div>

Thanks for the information!

Out of curiosity: if we have a data race, but the outcome does not depend on the order of the operations, what will happen if no atomics or locks are used?

---

<div class="post-metadata">

**Author:** ![foobar\_lv2](https://avatars.discourse-cdn.com/v4/letter/f/ee59a6/32.png) [@foobar\_lv2](https://discourse.julialang.org/u/foobar_lv2)\
**Post date:** [May 4, 2021, 8:54pm UTC](https://discourse.julialang.org/t/raising-a-flag-for-specific-condition-with-multithreading/60535/8 "2021-05-04T20:54:38Z")

</div>

The race condition is benign: Multiple threads can try to change the flag value from zero to one, but your threads don’t care about the flag value, and you don’t care which thread set it, and you don’t want to read the flag value until execution flow has joined again.

For performance reasons, I’d use something like `flag = Ref(false)`; `failure_condition && !flag[] && (flag[] = true)`.

Again, these data races are all benign; but if the failure condition is triggered often, then you don’t want all threads writing into the same cache line over and over again (because that is slow).

---

<div class="post-metadata">

**Author:** ![tkf](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tkf/32/17635_2.png) [@tkf](https://discourse.julialang.org/u/tkf)\
**Post date:** [May 5, 2021, 12:34am UTC](https://discourse.julialang.org/t/raising-a-flag-for-specific-condition-with-multithreading/60535/9 "2021-05-05T00:34:05Z")

</div>

> [@Gregstrq](#):
>
> what will happen if no atomics or locks are used?

It’s impossible to answer what will happen. My guess is “nothing interesting, probably equivalent to the code using atomics most of the time.” But the point is that we _can’t_ answer this question. _Data races imply undefined behavior._ You can’t trust the output of such a program.

> [@foobar\_lv2](#):
>
> The race condition is benign:

This is not what C/C++ memory model says. Since Julia is based on LLVM which is designed for C/C++, it’s the best model for reasoning about the code. It says program that mutates `Ref(false)` concurrently is broken. I recommend using relaxed memory order for what you are doing. In x86 it probably generates the same instructions as the one using `Ref` anyway.

However, using relaxed ordering in general is tricky due to problems such as out-of-thin-air. I find [https://www.youtube.com/watch?v=cWkUqK71DZ0](https://www.youtube.com/watch?v=cWkUqK71DZ0) and the accompanying paper highly informative for understanding how to use relaxed atomics reasonably safely.