# Boolean short circuit "fusion"

**URL:** <https://discourse.julialang.org/t/boolean-short-circuit-fusion/19641>\
**Category:** General Usage\
**Created:** [January 14, 2019, 10:33pm UTC](https://discourse.julialang.org/t/boolean-short-circuit-fusion/19641 "2019-01-14T22:33:10Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![jackf](https://avatars.discourse-cdn.com/v4/letter/j/b4bc9f/32.png) [@jackf](https://discourse.julialang.org/u/jackf)\
**Post date:** [January 14, 2019, 10:33pm UTC](https://discourse.julialang.org/t/boolean-short-circuit-fusion/19641/1 "2019-01-14T22:33:11Z")

</div>

I have an array `A`. I want to test if there are any values in `A` satisfying some test:

```julia
julia> f1(x) = any(x .> 99)

```

Easy code to read, but inefficient since we allocate the whole boolean array.

We could also define the following:

```julia
julia> function f2(xs)
           for x in xs
               if x > 99
                   return true
               end
           end
           false
       end

```

which will be faster, but we have to write a new for loop for every boolean expression.

Has anyone explored macros / tricks to get the syntactical lightness of the first with the performance of the second? i.e. to somehow get short-circuited broadcast fusion of elementwise boolean expressions? This should extend naturally to more complex expressions e.g. `any((A .> 0) .& (A .< 1) .| (B .== 5))`. The same pattern applies to `all` - i.e. return `false` as soon as the first `false` is encountered.

---

<div class="post-metadata">

**Author:** ![StefanKarpinski](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stefankarpinski/32/24_2.png) [@StefanKarpinski](https://discourse.julialang.org/u/StefanKarpinski)\
**Post date:** [January 14, 2019, 10:39pm UTC](https://discourse.julialang.org/t/boolean-short-circuit-fusion/19641/2 "2019-01-14T22:39:27Z")

</div>

```julia
f1(itr) = any(x > 99 for x in itr)

```

---

<div class="post-metadata">

**Author:** ![DNF](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dnf/32/10191_2.png) [@DNF](https://discourse.julialang.org/u/DNF)\
**Post date:** [January 14, 2019, 10:46pm UTC](https://discourse.julialang.org/t/boolean-short-circuit-fusion/19641/3 "2019-01-14T22:46:57Z")

</div>

Or

```julia
any(x->x>99, itr)

```

Since it is possible to do this:

```julia
any(==(99), itr)

```

would it be an idea to define `>(99)` too? (and `>=(99)`, `<=(99)`, etc?)

```julia
any(>(99), itr)

```

---

<div class="post-metadata">

**Author:** ![StefanKarpinski](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stefankarpinski/32/24_2.png) [@StefanKarpinski](https://discourse.julialang.org/u/StefanKarpinski)\
**Post date:** [January 14, 2019, 10:49pm UTC](https://discourse.julialang.org/t/boolean-short-circuit-fusion/19641/4 "2019-01-14T22:49:32Z")

</div>

Yes, it’s been proposed and I think it should be done.

---

<div class="post-metadata">

**Author:** ![jackf](https://avatars.discourse-cdn.com/v4/letter/j/b4bc9f/32.png) [@jackf](https://discourse.julialang.org/u/jackf)\
**Post date:** [January 14, 2019, 10:50pm UTC](https://discourse.julialang.org/t/boolean-short-circuit-fusion/19641/5 "2019-01-14T22:50:04Z")

</div>

Yes, thanks, doing it as an iterator works nicely for one array, but gets a bit clunky when you have a few arrays:

```julia
julia> any((a > 0) & (b < 0) & (c == 0) for (a, b, c) in zip(A, B, C))

```

---

<div class="post-metadata">

**Author:** ![bennedich](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bennedich/32/4894_2.png) [@bennedich](https://discourse.julialang.org/u/bennedich)\
**Post date:** [January 14, 2019, 10:56pm UTC](https://discourse.julialang.org/t/boolean-short-circuit-fusion/19641/6 "2019-01-14T22:56:49Z")

</div>

Note that you should use `&&` instead of `&` to short circuit the inner expression.

---

<div class="post-metadata">

**Author:** ![DNF](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dnf/32/10191_2.png) [@DNF](https://discourse.julialang.org/u/DNF)\
**Post date:** [January 14, 2019, 10:58pm UTC](https://discourse.julialang.org/t/boolean-short-circuit-fusion/19641/7 "2019-01-14T22:58:24Z")

</div>

```julia
any((x) -> x[1]>0 && x[2]<0 && x[3]==0, zip(X,Y,Z))

```

is a bit lighter, but, yes, it would be great if there were some (terse) way to get ‘fusion’ of broadcasting with reductions.

---

<div class="post-metadata">

**Author:** ![tkoolen](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tkoolen/32/1603_2.png) [@tkoolen](https://discourse.julialang.org/u/tkoolen)\
**Post date:** [January 15, 2019, 6:00am UTC](https://discourse.julialang.org/t/boolean-short-circuit-fusion/19641/8 "2019-01-15T06:00:55Z")

</div>

I like `do` syntax for this:

```julia
any(zip(A, B, C)) do (a, b, c)
    (a > 0) && (b < 0) && (c == 0)
end

```

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [January 15, 2019, 7:19am UTC](https://discourse.julialang.org/t/boolean-short-circuit-fusion/19641/9 "2019-01-15T07:19:02Z")

</div>

> [@DNF](#):
>
> would it be an idea to define `>(99)` too?

I think that ideally this should be solved by something like [#24990](https://github.com/JuliaLang/julia/pull/24990) in a generic way, instead of defining one-argument versions of multiple functions.

Or, for the slightly less generic case of partial application, one could define a `@pa` macro such that

```julia
@pa f(x)

```

expands to

```julia
y -> f(y, x)

```

which would take care of `>`, `isequal`, and similar.

---

<div class="post-metadata">

**Author:** ![DNF](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dnf/32/10191_2.png) [@DNF](https://discourse.julialang.org/u/DNF)\
**Post date:** [January 15, 2019, 8:02am UTC](https://discourse.julialang.org/t/boolean-short-circuit-fusion/19641/10 "2019-01-15T08:02:50Z")

</div>

My impression was that [#24990](https://github.com/JuliaLang/julia/pull/24990) was some distance away from reaching consensus. I really like that notation, but it’s not obvious to me if it would be equally well-suited to multi-argument lambdas, such as

```julia
(_ > 0) && (_ < 0) && (_ == 0)

```

A really generic solution might be some sort of lazy evaluation, perhaps obviating the need for the `reducer(f, itr)` pattern entirely.

---

<div class="post-metadata">

**Author:** ![DNF](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dnf/32/10191_2.png) [@DNF](https://discourse.julialang.org/u/DNF)\
**Post date:** [January 15, 2019, 8:07am UTC](https://discourse.julialang.org/t/boolean-short-circuit-fusion/19641/11 "2019-01-15T08:07:05Z")

</div>

> [@tkoolen](#):
>
> I like `do` syntax for this:
> 
> ```julia
> any(zip(A, B, C)) do (a, b, c)
> (a > 0) && (b < 0) && (c == 0)
> end
> 
> ```

For some reason, I haven’t become comfortable with the `do`-notation. Every time I encounter it, I try to read it as the `with/as` notation from Python. Some times that clarifies it, but some times it doesn’t.

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [January 15, 2019, 9:56am UTC](https://discourse.julialang.org/t/boolean-short-circuit-fusion/19641/12 "2019-01-15T09:56:47Z")

</div>

I am not sure that

> [@DNF](#):
>
> ```julia
> (_ > 0) && (_ < 0) && (_ == 0)
> 
> ```

is a multi-argument lambda — are the `_` placeholders different arguments?

In any case, while I am looking forward to the `_` syntax, I think I would mostly use it for simple stuff.

For some of that, a simple partial application construct is also feasible; the question is

1. whether we make some functions PA manually, like `isequal` (this has already happened, should it be extended like you suggested for `>`?),
2. introduce a macro to make (1) more convenient, but still manually applied (ie at function definition),
3. make PA automatic (I don’t think this meshes well with Julia’s semantics though, and is a large and breaking change),
4. introduce a macro for _using_ PA, like the `@pa` I suggested above.

I am always undecided between these (except 3, which I don’t consider to be a good solution) and alternatives (the `_` syntax for closures, etc). Sometimes I would find the more convenient, but at the same time closures are universal, quite readable, and just take a few extra characters.

---

<div class="post-metadata">

**Author:** ![DNF](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dnf/32/10191_2.png) [@DNF](https://discourse.julialang.org/u/DNF)\
**Post date:** [January 15, 2019, 10:01am UTC](https://discourse.julialang.org/t/boolean-short-circuit-fusion/19641/13 "2019-01-15T10:01:27Z")

</div>

> [@Tamas\_Papp](#):
>
> I am not sure that
> 
> ![](https://global.discourse-cdn.com/julialang/original/3X/2/e/2e872199259e32e817852af71210b55efa3327a3.png)  
> DNF:
> 
> > ```julia
> > (_ > 0) && (_ < 0) && (_ == 0)
> > 
> > ```
> 
> is a multi-argument lambda — are the `_` placeholders different arguments?

Exactly, it’s not obvious what the meaning ought to be. In this case I intended it to be different arguments, but it could just as easily be one argument.

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [January 15, 2019, 10:15am UTC](https://discourse.julialang.org/t/boolean-short-circuit-fusion/19641/14 "2019-01-15T10:15:25Z")

</div>

Yes, that’s why I find

```julia
(x, y, z) -> (x > 0) && (y < 0) && (z == 0)

```

to be the best solution, even if more verbose.

It’s not that there aren’t more concise ways of writing this, but none of these are as general and/or run into corner cases (as shown eg in the discussion of the PR above).

---

<div class="post-metadata">

**Author:** ![dawbarton](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dawbarton/32/215461_2.png) [@dawbarton](https://discourse.julialang.org/u/dawbarton)\
**Post date:** [January 15, 2019, 1:28pm UTC](https://discourse.julialang.org/t/boolean-short-circuit-fusion/19641/15 "2019-01-15T13:28:25Z")

</div>

I’ve wondered for a while whether it is possible to abuse the broadcast machinery to do something like this (basically to avoid `materialize` being called on the lazy broadcasted object). I’ve not found a nice way of doing it (i.e., a way that doesn’t obscure/muddy the semantics of dot broadcasting) but here is a not-so-nice way of doing it.

```julia
myany(x) = any(x) # avoid piracy!
Broadcast.broadcasted(::typeof(myany), x) = myany(x) # exploit the fact that the broadcasted object is iterable

```

On a simple test case this gives

```julia
julia> a = rand(10000) ; b = rand(10000) ; c = rand(10000) ;

julia> @btime any(($a .> 0.5) .& ($b .> 0.5) .& ($c .> 0.5))
  19.828 μs (3 allocations: 5.55 KiB)
true

julia> @btime myany.(($a .> 0.5) .& ($b .> 0.5) .& ($c .> 0.5))
  59.478 ns (10 allocations: 240 bytes)
true

```

So much faster than the original case but not as fast as the anonymous function or generator expression versions as mentioned above (and it allocates more).

```julia
julia> @btime any(((aa, bb, cc),) -> (aa > 0.5) && (bb > 0.5) && (cc > 0.5), zip($a, $b, $c))
  13.820 ns (2 allocations: 64 bytes)
true

julia> @btime any((aa > 0.5) && (bb > 0.5) && (cc > 0.5) for (aa, bb, cc) in zip($a, $b, $c))
  16.822 ns (3 allocations: 80 bytes)
true

```

---

<div class="post-metadata">

**Author:** ![antoine-levitt](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/antoine-levitt/32/4008_2.png) [@antoine-levitt](https://discourse.julialang.org/u/antoine-levitt)\
**Post date:** [January 15, 2019, 2:15pm UTC](https://discourse.julialang.org/t/boolean-short-circuit-fusion/19641/16 "2019-01-15T14:15:08Z")

</div>

That would be very nice if there was a way to make that work generically; this pattern comes up pretty frequently, e.g. `dot(x,y) = sum(x.*y), exit = all(abs.(err) <.= tol)` and friends. Can’t all reducers in Base just define `broadcasted` like you do?

---

<div class="post-metadata">

**Author:** ![DNF](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dnf/32/10191_2.png) [@DNF](https://discourse.julialang.org/u/DNF)\
**Post date:** [January 15, 2019, 2:22pm UTC](https://discourse.julialang.org/t/boolean-short-circuit-fusion/19641/17 "2019-01-15T14:22:56Z")

</div>

Wouldn’t that be bad if you want to apply the reduction on a collection of iterables? Seems like it would be bette to have a way to signal this explicitly.

---

<div class="post-metadata">

**Author:** ![dawbarton](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dawbarton/32/215461_2.png) [@dawbarton](https://discourse.julialang.org/u/dawbarton)\
**Post date:** [January 15, 2019, 2:51pm UTC](https://discourse.julialang.org/t/boolean-short-circuit-fusion/19641/18 "2019-01-15T14:51:02Z")

</div>

> [@DNF](#):
>
> Wouldn’t that be bad if you want to apply the reduction on a collection of iterables? Seems like it would be bette to have a way to signal this explicitly.

Yes, I don’t like this solution since it obscures the meaning of the dot notation. I’ve been trying to find a way to do this explicitly but I can’t find a way to coerce the lowering code to remove the final `materialize` call that appears when the dot notation is expanded out. (It doesn’t help that I don’t really know enough Scheme to be able to understand the lowering code properly.)

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [January 15, 2019, 5:35pm UTC](https://discourse.julialang.org/t/boolean-short-circuit-fusion/19641/19 "2019-01-15T17:35:01Z")

</div>

> [@dawbarton](#):
>
> I’ve wondered for a while whether it is possible to abuse the broadcast machinery to do something like this (basically to avoid `materialize` being called on the lazy broadcasted object).

See the discussion here:

> <https://github.com/JuliaLang/julia/issues/19198#issuecomment-388421815>
>
> The notation \`f.(v)\` effectively maps \`f\` over an array \`v\`. This operation is n…ot lazy, it materializes the mapped array.
> 
> I think it could be useful to have an analogous notation for lazy maps. Maybe \`f..(v)\`? Instead of materializing the mapped array, this should return something equivalent to the generator:
> 
> \`\`\`(f(x) for x in v)\`\`\`
> 
> For example, this could be useful to do something like:
> 
> \`\`\`sum(f..(v))\`\`\`
> 
> which is more efficient than materializing the intermediate array in:
> 
> \`\`\`sum(f.(v))\`\`\`
> 
> Of course right now \`sum\` takes an optional function argument, so one can write \`sum(f,v)\`. But see the discussion here: https://github.com/JuliaLang/julia/issues/19146. If one decides to remove the method \`sum(f,v)\`, I think the notation \`sum(f..(v))\` could be a nice alternative.

---

<div class="post-metadata">

**Author:** ![dawbarton](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dawbarton/32/215461_2.png) [@dawbarton](https://discourse.julialang.org/u/dawbarton)\
**Post date:** [January 15, 2019, 11:20pm UTC](https://discourse.julialang.org/t/boolean-short-circuit-fusion/19641/20 "2019-01-15T23:20:49Z")

</div>

After reading that issue I had some inspiration -

```julia
@inline _lazy(x) = x[1] # unwrap the tuple
@inline Broadcast.broadcasted(::typeof(_lazy), x) = (x,) # wrap the Broadcasted object in a tuple to avoid materializing
macro lazy(x)
	return esc(:(_lazy(_lazy.($x))))
end

```

Which gives

```julia
julia> a = rand(10000) ; b = rand(10000) ;

julia> @btime sum(a .+ b)
  10.896 μs (5 allocations: 78.27 KiB)
9902.652963666198

julia> @btime sum(@lazy a .+ b)
  13.095 μs (3 allocations: 64 bytes)
9902.652963666245

julia> @btime any(a .+ b .> 1.9)
  15.549 μs (7 allocations: 5.64 KiB)
true

julia> @btime any(@lazy a .+ b .> 1.9)
  1.016 μs (4 allocations: 96 bytes)
true

```

So not materializing the array isn’t always faster (depends on the operation) but it does save on allocations.

[Next page](https://discourse.julialang.org/t/boolean-short-circuit-fusion/19641.md?page=2)