# Nested (non-syntactic?) loop fusion in 0.6

**URL:** <https://discourse.julialang.org/t/nested-non-syntactic-loop-fusion-in-0-6/2544>\
**Category:** General Usage\
**Created:** [March 8, 2017, 4:08pm UTC](https://discourse.julialang.org/t/nested-non-syntactic-loop-fusion-in-0-6/2544 "2017-03-08T16:08:57Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![marius311](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/marius311/32/3953_2.png) [@marius311](https://discourse.julialang.org/u/marius311)\
**Post date:** [March 8, 2017, 4:08pm UTC](https://discourse.julialang.org/t/nested-non-syntactic-loop-fusion-in-0-6/2544/1 "2017-03-08T16:08:57Z")

</div>

Is there a way for loop fusion in Julia to operate even recursively into function calls? By that I mean, consider,

```julia
julia> a=b=rand(2000,2000);

julia> @time a.+b.+a;
  0.064173 seconds (10.63 k allocations: 31.015 MiB, 2.54% gc time)

julia> myplus(a,b) = a.+b
myplus (generic function with 1 method)

julia> @time a.+myplus(a,b);
  0.058254 seconds (53 allocations: 61.037 MiB, 4.30% gc time)

```

Its clear the latter call wasn’t fused into a single loop since twice the memory was allocated.

My best guess is this isn’t supported, and maybe never will be because the ambiguity of what to do if `myplus` has other side-effects. However, is there any way to make something similar to this work, to allow shorthand for some longer operation than `myplus`? It certainly seems within the realm of reason that the compiler could figure out in simple cases like this that its safe to fuse.

---

<div class="post-metadata">

**Author:** ![PabloZubieta](https://avatars.discourse-cdn.com/v4/letter/p/ee7513/32.png) [@PabloZubieta](https://discourse.julialang.org/u/PabloZubieta)\
**Post date:** [March 8, 2017, 4:17pm UTC](https://discourse.julialang.org/t/nested-non-syntactic-loop-fusion-in-0-6/2544/2 "2017-03-08T16:17:58Z")

</div>

You can write `a .+ myplus.(a, b)`

---

<div class="post-metadata">

**Author:** ![StefanKarpinski](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stefankarpinski/32/24_2.png) [@StefanKarpinski](https://discourse.julialang.org/u/StefanKarpinski)\
**Post date:** [March 8, 2017, 4:29pm UTC](https://discourse.julialang.org/t/nested-non-syntactic-loop-fusion-in-0-6/2544/3 "2017-03-08T16:29:24Z")

</div>

In this case, @PabloZubieta’s suggestion will fuse, but it wouldn’t work if your user-defined function didn’t happen to also do the scalar operation you want – or more generally, if you have an operation that cannot be expressed as a vectorization of any scalar operation (e.g. matmul, matrix inversion). In such cases, we probably will never be able to guarantee fusion, although a sufficiently clever compiler optimization could observe that fusion is possible and do it for you.

At a higher level, the new syntactic loop fusion support means that one should shift programming styles away from what’s traditionally been done in vectorized languages:

- In vectorized languages, one wants to push vectorization as far “up” as possible: compose more complex operations out of simpler vectorized operations, and then let vectorization flow from the top down to individual, optimized vectorized operations.

- In Julia since 0.5, one wants to do the opposite and push scalar operations as far up as possible: express complex operations in entirely scalar form for as long as you can (i.e. until you hit an inherently array operation like a matmul or matrix inversion or whatever), and then vectorize at the very end by calling functions using `f.(args...)` syntax.

So in this case, if `myplus` can be defined to only operate on scalars, then do that – don’t define it to work on vectors at all. Instead, if you need to call it on vectors, use the `myplus.(a, b)` form.

---

<div class="post-metadata">

**Author:** ![marius311](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/marius311/32/3953_2.png) [@marius311](https://discourse.julialang.org/u/marius311)\
**Post date:** [March 8, 2017, 5:45pm UTC](https://discourse.julialang.org/t/nested-non-syntactic-loop-fusion-in-0-6/2544/4 "2017-03-08T17:45:23Z")

</div>

> [@StefanKarpinski](#):
>
> it wouldn’t work if your user-defined function didn’t happen to also do the scalar operation you want – or more generally, if you have an operation that cannot be expressed as a vectorization of any scalar operation (e.g. matmul, matrix inversion).

Thanks yea, I realized after @PabloZubieta’s answer that I made my example here too simple, indeed what I’m actually trying to do wouldn’t work because it involves a matrix multiply.

In any case, useful advice on programming style, I will keep it in mind!

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [March 8, 2017, 6:20pm UTC](https://discourse.julialang.org/t/nested-non-syntactic-loop-fusion-in-0-6/2544/5 "2017-03-08T18:20:52Z")

</div>

Instead of loop fusion for matrix multiplication and the like, you can just chain inplace operations. That will still avoid any memory allocations.