# Mapreduce with broadcasting

**URL:** <https://discourse.julialang.org/t/mapreduce-with-broadcasting/77053>\
**Category:** Performance\
**Created:** [February 25, 2022, 9:19am UTC](https://discourse.julialang.org/t/mapreduce-with-broadcasting/77053 "2022-02-25T09:19:47Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![torrance](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/torrance/32/38990_2.png) [@torrance](https://discourse.julialang.org/u/torrance)\
**Post date:** [February 25, 2022, 9:19am UTC](https://discourse.julialang.org/t/mapreduce-with-broadcasting/77053/1 "2022-02-25T09:19:47Z")

</div>

I want to perform a mapreduce over multiple iterators that don’t have the same dimensions, but which can be broadcast together. For example:

```julia
arr1 = rand(1, 1, 3)
arr2 = rand(100, 100, 1)

mapreduce(+, *, arr1, arr2; dims=(1, 2))

ERROR: DimensionMismatch("dimensions must match: a has dims (Base.OneTo(1), Base.OneTo(1), Base.OneTo(3)), b has dims (Base.OneTo(100), Base.OneTo(100), Base.OneTo(1)), mismatch at 1")

```

Is this possible? Or if not, are there any Julia utilities that can act ‘expand’ the dimensions of an array along 1 or more axes, without expanding the memory.

The only other mention I can see of this is from 4 years ago and looks hacky: [Is there something like broadcast\_mapreduce?](https://discourse.julialang.org/t/is-there-something-like-broadcast-mapreduce/6076)

---

<div class="post-metadata">

**Author:** ![fabiangans](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/fabiangans/32/2624_2.png) [@fabiangans](https://discourse.julialang.org/u/fabiangans)\
**Post date:** [February 25, 2022, 1:52pm UTC](https://discourse.julialang.org/t/mapreduce-with-broadcasting/77053/2 "2022-02-25T13:52:14Z")

</div>

I think you can use LazyArrays.jl for this.

[https://github.com/JuliaArrays/LazyArrays.jl](https://github.com/JuliaArrays/LazyArrays.jl)

```julia
arr1 = rand(1, 1, 3)
arr2 = rand(100, 100, 1)
arr3 = LazyArray(@~ arr1 .> arr2)
sum(arr3,dims=(1,2))

```

---

<div class="post-metadata">

**Author:** ![torrance](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/torrance/32/38990_2.png) [@torrance](https://discourse.julialang.org/u/torrance)\
**Post date:** [February 25, 2022, 3:50pm UTC](https://discourse.julialang.org/t/mapreduce-with-broadcasting/77053/3 "2022-02-25T15:50:12Z")

</div>

Thanks for that suggestion. It seems to work for cpu code, but the ultimate destination for this mapreduce is on the GPU and CUDA interactions with LazyArrays seems to trigger scalar indexing.

For example:

```julia
using CUDA
using LazyArrays

arr1 = CUDA.rand(100, 100, 1)
arr2 = CUDA.rand(1, 1, 3)
arr3 = LazyArray(@~ arr1 .+ arr2)
CUDA.reduce(+, arr3, dims=(1, 2))

Warning: Performing scalar indexing on task Task (runnable)

```

(The reason I’m trying to use mapreduce or reduce here is because these functions are consistently faster than the manual reductions I am able to write myself.)

---

<div class="post-metadata">

**Author:** ![torrance](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/torrance/32/38990_2.png) [@torrance](https://discourse.julialang.org/u/torrance)\
**Post date:** [February 26, 2022, 6:33am UTC](https://discourse.julialang.org/t/mapreduce-with-broadcasting/77053/4 "2022-02-26T06:33:31Z")

</div>

@maleadt I saw [this commit](https://github.com/JuliaGPU/GPUArrays.jl/pull/270/commits/6e7560a1daea34da1e4359cb2a404a648087b26d) suggesting mapreduce in GPUArrays already supports broadcasting? Is this correct, and if so, is there a flag or similar required to enable it?

---

<div class="post-metadata">

**Author:** ![jling](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jling/32/212909_2.png) [@jling](https://discourse.julialang.org/u/jling)\
**Post date:** [February 26, 2022, 7:41am UTC](https://discourse.julialang.org/t/mapreduce-with-broadcasting/77053/5 "2022-02-26T07:41:16Z")

</div>

what does reduce with `>` do? check if it’s sorted along one dimension?

it may be easier for others to help if you can provide example input/output with an explanation of intention.

notice this doesn’t even work in vanilla julia:

```julia
julia> reduce(>, rand(100,100,3); dims=1)
ERROR: MethodError: no method matching reducedim_init(::typeof(identity), ::typeof(>), ::Array{Float64, 3}, ::Int64)

```

---

<div class="post-metadata">

**Author:** ![torrance](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/torrance/32/38990_2.png) [@torrance](https://discourse.julialang.org/u/torrance)\
**Post date:** [February 26, 2022, 8:26am UTC](https://discourse.julialang.org/t/mapreduce-with-broadcasting/77053/6 "2022-02-26T08:26:21Z")

</div>

@jling Yeah it’s a poor minimal example, but it does work - the error you’re seeing is due to `>` having no defined identity element. If you provide an init=0, for example, it will work.

The actual use case here is a type of 2d fourier transform. It’s a bit too heavy to get into the details, but the map operator is a little exponential, and the reduction sums across all the data points onto a 2D grid.

---

<div class="post-metadata">

**Author:** ![torrance](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/torrance/32/38990_2.png) [@torrance](https://discourse.julialang.org/u/torrance)\
**Post date:** [February 26, 2022, 8:30am UTC](https://discourse.julialang.org/t/mapreduce-with-broadcasting/77053/7 "2022-02-26T08:30:37Z")

</div>

I have one working work-around, but I don’t know if its optimal. It does, however, have the nice property that it works on the GPU too (in pseudocode):

```julia
function broadcast_reduce(f, op, A, B, C, ...; dims, init)
    bc = Broadcast.instantiate(Broadcast.broadcasted(f, A, B, C...)
    return reduce(op, bc; dims, init)
end

```

---

<div class="post-metadata">

**Author:** ![jling](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jling/32/212909_2.png) [@jling](https://discourse.julialang.org/u/jling)\
**Post date:** [February 26, 2022, 8:36am UTC](https://discourse.julialang.org/t/mapreduce-with-broadcasting/77053/8 "2022-02-26T08:36:29Z")

</div>

```julia
julia> reduce(>, [0,1,2]; init=0)
false

julia> reduce(>, [-1, -2, -3]; init=0)
true

julia> reduce(>, [2, 1, 0]; init=3)
false

```

I don’t think it does what you want it to do? this is basically checking if everything in the array is smaller than 0?

---

<div class="post-metadata">

**Author:** ![torrance](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/torrance/32/38990_2.png) [@torrance](https://discourse.julialang.org/u/torrance)\
**Post date:** [February 26, 2022, 8:43am UTC](https://discourse.julialang.org/t/mapreduce-with-broadcasting/77053/9 "2022-02-26T08:43:25Z")

</div>

@jling Please ignore the reduction operator I gave as an example in the first post. You can use + or \* or whatever you’d like.

---

<div class="post-metadata">

**Author:** ![maleadt](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/maleadt/32/10097_2.png) [@maleadt](https://discourse.julialang.org/u/maleadt)\
**Post date:** [March 2, 2022, 10:28am UTC](https://discourse.julialang.org/t/mapreduce-with-broadcasting/77053/10 "2022-03-02T10:28:50Z")

</div>

> [@torrance](#):
>
> @maleadt I saw [this commit](https://github.com/JuliaGPU/GPUArrays.jl/pull/270/commits/6e7560a1daea34da1e4359cb2a404a648087b26d) suggesting mapreduce in GPUArrays already supports broadcasting? Is this correct, and if so, is there a flag or similar required to enable it?

Kind of, but there’s no user-facing interface. This should be done with LazyArrays, we’re probably just missing some dispatches for those lazy values to be compatible with the GPU broadcast implementation.