# Broadcasting and function composition (circ)

**URL:** <https://discourse.julialang.org/t/broadcasting-and-function-composition-circ/46748>\
**Category:** General Usage\
**Tags:** question, broadcast, composition\
**Created:** [September 17, 2020, 5:04am UTC](https://discourse.julialang.org/t/broadcasting-and-function-composition-circ/46748 "2020-09-17T05:04:08Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![tbeason](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tbeason/32/15898_2.png) [@tbeason](https://discourse.julialang.org/u/tbeason)\
**Post date:** [September 17, 2020, 5:04am UTC](https://discourse.julialang.org/t/broadcasting-and-function-composition-circ/46748/1 "2020-09-17T05:04:09Z")

</div>

How does one easily compose and broadcast?

For example,

```julia
x = randn(100,10)
mapslices(cumprod ∘ exp, x; dims=1)

```

fails because we need to broadcast `exp`.

One workaround is to do the following

```julia
x = randn(100,10)
expdot(v) = exp.(v)
mapslices(cumprod ∘ expdot, x; dims=1)

```

Are there other, hopefully better, ways?

---

<div class="post-metadata">

**Author:** ![mcabbott](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mcabbott/32/6603_2.png) [@mcabbott](https://discourse.julialang.org/u/mcabbott)\
**Post date:** [September 17, 2020, 5:38am UTC](https://discourse.julialang.org/t/broadcasting-and-function-composition-circ/46748/2 "2020-09-17T05:38:27Z")

</div>

You can write

```julia
∘̇(f,g) = x -> f(g.(x))
mapslices(cumprod ∘̇ exp, x; dims=1)

```

but perhaps shouldn’t. Better:

```julia
using Underscores
@_ mapslices(cumprod(exp.(_)), x; dims=1)

```

---

<div class="post-metadata">

**Author:** ![xiaodai](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/xiaodai/32/15937_2.png) [@xiaodai](https://discourse.julialang.org/u/xiaodai)\
**Post date:** [September 17, 2020, 6:53am UTC](https://discourse.julialang.org/t/broadcasting-and-function-composition-circ/46748/3 "2020-09-17T06:53:44Z")

</div>

> [@tbeason](#):
>
> ```julia
> x = randn(100,10)
> mapslices(cumprod ∘ exp, x; dims=1)
> 
> ```

I would do

```julia
x = randn(100,10)
mapslices(xrow->exp.(xrow) |> cumprod, x; dims=1)

```

---

<div class="post-metadata">

**Author:** ![DNF](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dnf/32/10191_2.png) [@DNF](https://discourse.julialang.org/u/DNF)\
**Post date:** [September 17, 2020, 7:12am UTC](https://discourse.julialang.org/t/broadcasting-and-function-composition-circ/46748/4 "2020-09-17T07:12:32Z")

</div>

These suggestions all create an intermediate array that seems like it is redundant. But it’s not so easy to come up with a trivial solution that avoids it. I thought maybe `accumulate` would help, but I actually need something like `mapaccumulate`:

```julia
mapaccumulate(exp, *, x; dims=1)

```

Unfortunately, there is no `mapaccumulate`.

Looks like a loop is needed, unless I’m missing something. Perhaps Transducers.jl has something helpful.

---

<div class="post-metadata">

**Author:** ![mcabbott](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mcabbott/32/6603_2.png) [@mcabbott](https://discourse.julialang.org/u/mcabbott)\
**Post date:** [September 17, 2020, 7:26am UTC](https://discourse.julialang.org/t/broadcasting-and-function-composition-circ/46748/5 "2020-09-17T07:26:45Z")

</div>

`mapaccumulate` is an interesting idea. You could also do this as a lazy broadcast, to avoid making too many intermediates. This might even be fast:

```julia
reduce(hcat, map(eachcol(x)) do slice
    lazy = Broadcast.Broadcasted(exp, (slice,))
    cumsum(lazy)
end)

```

avoiding `mapslices` too. Unfortunately this doesn’t work:

```julia
y = similar(x);
foreach(eachcol(y), eachcol(x)) do ycol, xcol
    lazy = Broadcast.Broadcasted(exp, (xcol,))
    cumsum!(ycol, lazy; dims=1)
end

```

---

<div class="post-metadata">

**Author:** ![josuagrw](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/josuagrw/32/1015_2.png) [@josuagrw](https://discourse.julialang.org/u/josuagrw)\
**Post date:** [September 17, 2020, 8:52am UTC](https://discourse.julialang.org/t/broadcasting-and-function-composition-circ/46748/6 "2020-09-17T08:52:59Z")

</div>

The problem with the lazy approach seems to be that `cumprod` has no method for `Generator`s:

```julia
x = rand(10, 100)
lazy_exp(v) = (exp(y) for y in v)
lazy_cumprod(x) = Iterators.accumulate(*, x)
mapslices(collect ∘ lazy_cumprod ∘ lazy_exp, x; dims=1)

```

---

<div class="post-metadata">

**Author:** ![xiaodai](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/xiaodai/32/15937_2.png) [@xiaodai](https://discourse.julialang.org/u/xiaodai)\
**Post date:** [September 17, 2020, 9:49am UTC](https://discourse.julialang.org/t/broadcasting-and-function-composition-circ/46748/7 "2020-09-17T09:49:34Z")

</div>

> [@DNF](#):
>
> `mapaccumulate(exp, *, x; dims=1)`

`map(v->accumulate((x,y)->x*exp(y), x; dims=1)`

---

<div class="post-metadata">

**Author:** ![josuagrw](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/josuagrw/32/1015_2.png) [@josuagrw](https://discourse.julialang.org/u/josuagrw)\
**Post date:** [September 17, 2020, 9:59am UTC](https://discourse.julialang.org/t/broadcasting-and-function-composition-circ/46748/8 "2020-09-17T09:59:24Z")

</div>

```julia
mapslices(v->accumulate((x,y)->x*exp(y), v), x; dims=1)

```

EDIT: Oh, this doesn’t exponentiate the first element of each row

---

<div class="post-metadata">

**Author:** ![tbeason](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tbeason/32/15898_2.png) [@tbeason](https://discourse.julialang.org/u/tbeason)\
**Post date:** [September 17, 2020, 5:31pm UTC](https://discourse.julialang.org/t/broadcasting-and-function-composition-circ/46748/9 "2020-09-17T17:31:02Z")

</div>

this is not equivalent to the other methods, something must be off. Your previous post is slightly faster than my original method.

I should add that the dimensions I gave for the example are way off of the true size. The number of columns is way way bigger than rows. `randn(750,100000)` is pretty close to the actual size of the input.

---

<div class="post-metadata">

**Author:** ![josuagrw](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/josuagrw/32/1015_2.png) [@josuagrw](https://discourse.julialang.org/u/josuagrw)\
**Post date:** [September 18, 2020, 4:17pm UTC](https://discourse.julialang.org/t/broadcasting-and-function-composition-circ/46748/10 "2020-09-18T16:17:39Z")

</div>

This approach is 4 times fast than the mapslices approach

```julia

using BenchmarkTools
function cumprod_exp(x)
    y = similar(x)
    foreach(
        [view(y, :, i) for i in 1:size(y, 2)],
        [view(x, :, i) for i in 1:size(x, 2)]
    ) do v, u
        cumprod!(v, exp.(u))
    end
    return y
end

@btime cumprod_exp($(rand(10, 100)))

```

You can make it even faster if x is no longer needed

```julia
function cumprod_exp(x)
    y = similar(x)
    foreach(
        [view(y, :, i) for i in 1:size(y, 2)],
        [view(x, :, i) for i in 1:size(x, 2)]
    ) do v, u
        u .= exp.(u)
        cumprod!(v, u)
    end
    return y
end

```

Note, that you can easily save even more time by preallocating y – this just avoids the `vcat`s in `mapslices`

---

<div class="post-metadata">

**Author:** ![tbeason](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tbeason/32/15898_2.png) [@tbeason](https://discourse.julialang.org/u/tbeason)\
**Post date:** [September 18, 2020, 5:13pm UTC](https://discourse.julialang.org/t/broadcasting-and-function-composition-circ/46748/11 "2020-09-18T17:13:28Z")

</div>

Yea I ended up rolling my own along those lines once I realized that there was no in-place `mapslices` (why!?). I also started chaining together 2 additional operations, a trailing sum of `N` elements and then a log difference from `N` periods ago which meant that stuff would allocate regardless unless I was smart about those (since they don’t just use consecutive entries). I ended up using `CircularBuffer` from DataStructures.jl for that part. Now the only thing that allocates is the buffer.

I guess the limiting part is really just that I’m doing this 10s or 100s of thousands of times. Unfortunate that the way this is being used I can’t parallelize it across columns. Oh well. Thanks for the suggestions!