# Tullio - multiline syntax - Soft or hard local scope?

**URL:** <https://discourse.julialang.org/t/tullio-multiline-syntax-soft-or-hard-local-scope/77616>\
**Category:** Performance\
**Tags:** tullio\
**Created:** [March 9, 2022, 4:50am UTC](https://discourse.julialang.org/t/tullio-multiline-syntax-soft-or-hard-local-scope/77616 "2022-03-09T04:50:29Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![Emmanuel-R8](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/emmanuel-r8/32/11839_2.png) [@Emmanuel-R8](https://discourse.julialang.org/u/Emmanuel-R8)\
**Post date:** [March 9, 2022, 4:50am UTC](https://discourse.julialang.org/t/tullio-multiline-syntax-soft-or-hard-local-scope/77616/1 "2022-03-09T04:50:29Z")

</div>

I have a block of code using Tullio with a `begin ... end` syntax.

I discovered that, within the block, only `=` can be used. No `:=` will be acctepted.

My problem now is that on execution that `patch_mean` is an unknown symbol. If I extract that line as a single `@tullio` statement, `patch_mean` is defined, and then `patch_centered` becomes the unknown symbol.

If I extract all the lines with single statments (thought losing the benefit of better compilation), I get a new error `can't apply finaliser without a reduction`.

Nothing on Google / Stack Exchange. Any clue?

```julia
# Average input values (channel-wise)
@tullio sk[b] := begin
    patch_mean[in_c, b] = patch[w, h, in_c, b] / ($k * $k)

    # Second moment adjusted for DEFAULT_ϵ and q
    patch_centered[w, h, in_c, b] = patch[w, h, in_c, b] - patch_mean[in_c, b]

    # MORE LINES
    final_matrix[b]
end

```

---

<div class="post-metadata">

**Author:** ![Emmanuel-R8](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/emmanuel-r8/32/11839_2.png) [@Emmanuel-R8](https://discourse.julialang.org/u/Emmanuel-R8)\
**Post date:** [March 9, 2022, 9:21am UTC](https://discourse.julialang.org/t/tullio-multiline-syntax-soft-or-hard-local-scope/77616/2 "2022-03-09T09:21:29Z")

</div>

Found a new problem.

MWE:

```julia
A = rand(4, 4, 5, 5)
B = rand(4, 4)

@tullio C[k, l] := A[i, j, k, l] * B[i, j] # OK
@tullio C[k, l] := A[i, j, k, l] * B[i, j] |> sqrt # OK
@tullio C[k, l] := sqrt <| A[i, j, k, l] * B[i, j] # OK

@tullio C[k, l] := begin
    A[i, j, k, l] * B[i, j]
end # OK

@tullio C[k, l] := begin
    A[i, j, k, l] * B[i, j] |> sqrt
end # OK

@tullio C[k, l] := begin
    sqrt <| A[i, j, k, l] * B[i, j]
end # KO

```

---

<div class="post-metadata">

**Author:** ![goerch](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/goerch/32/29122_2.png) [@goerch](https://discourse.julialang.org/u/goerch)\
**Post date:** [March 9, 2022, 6:26pm UTC](https://discourse.julialang.org/t/tullio-multiline-syntax-soft-or-hard-local-scope/77616/3 "2022-03-09T18:26:18Z")

</div>

The error message seems clear to me:

```julia
ERROR: LoadError: UndefVarError: <| not defined

```

`<|` simply is not implemented.

Is there an easy way for us to reproduce your original problem?

---

<div class="post-metadata">

**Author:** ![Emmanuel-R8](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/emmanuel-r8/32/11839_2.png) [@Emmanuel-R8](https://discourse.julialang.org/u/Emmanuel-R8)\
**Post date:** [March 10, 2022, 1:11am UTC](https://discourse.julialang.org/t/tullio-multiline-syntax-soft-or-hard-local-scope/77616/4 "2022-03-10T01:11:33Z")

</div>

I asked questions on the github issues. Thanks to their answers, part of the errors relate to something I hadn’t understood. Within a `begin ... end` you need to apply pure Julia syntax. Therefore both `:=` and `<|` are not defined (not defined in `Base`) and will yield an error.

I submitted a PR to clarify the docs.

Still tracking the rest. I’ll make a MWE when I can isolate one.

---

<div class="post-metadata">

**Author:** ![Emmanuel-R8](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/emmanuel-r8/32/11839_2.png) [@Emmanuel-R8](https://discourse.julialang.org/u/Emmanuel-R8)\
**Post date:** [March 10, 2022, 3:36am UTC](https://discourse.julialang.org/t/tullio-multiline-syntax-soft-or-hard-local-scope/77616/5 "2022-03-10T03:36:42Z")

</div>

A lot of examples with inconsistent results. But bottom line, the behaviour breaks my assumption that macros define a hard local scope (they are hygienic).

What is even weirder is that the macro behaves a mix of soft and hard scope. I am utterly confused.

```julia
using Tullio #0.3.3 - Julia 1.6.5

A = rand(4, 4, 5, 5)
B = rand(4, 4)

# OK
@tullio C1[k, l] := A[i, j, k, l] * B[i, j]
C1

# Return a sum instead of 4 x 4 matrix. Intended behaviour?
@tullio C7 := A[i, j, k, l] * B[i, j]

# OK
@tullio C2[k, l] := begin
    A[i, j, k, l] * B[i, j]
end
C1 ≈ C2

# Return a sum instead of 4 x 4 matrix. Intended behaviour?
@tullio C6 := begin
    A[i, j, k, l] * B[i, j]
end

# Breaks because base Julia would expect the matrix to be already existing
@tullio C3[k, l] := begin
    TMP[k, l] = A[i, j, k, l] * B[i, j]
end

# Therefore this works
@tullio C3[k, l] := begin
    TMP = A[i, j, k, l] * B[i, j]
end
C1 ≈ C3

# TMP was just used, but defined in a local scope. Not exported.
# This doesn't work
@tullio C4[k, l] := begin
    TMP[k, l] = A[i, j, k, l] * B[i, j]
    TMP
end
C1 ≈ C4

TMP = zeros(5, 5)
# Breaks with error:
# ERROR: LoadError: MethodError: no method matching zero(::Type{Matrix{Float64}})
@tullio C5[k, l] := begin
    TMP[k, l] = A[i, j, k, l] * B[i, j]
    TMP
end
C1 ≈ C5

TMP = zeros(5, 5)
# But this works
@tullio C5[k, l] := begin
    TMP[k, l] = A[i, j, k, l] * B[i, j]
    TMP[k, l]
end
C1 ≈ C5

##########################################

A = rand(4, 4, 5, 5)
B = rand(5, 5, 6, 6)
C = rand(6, 6)

@tullio D1[i, j, m, n] := A[i, j, k, l] * B[k, l, m, n]
@tullio E1[i, j] := D1[i, j, m, n] * C[m, n]

# Breaks D3 undefined. Exoected with soft local scope. Not within macro that should define the array.
@tullio F2 := begin
    D2[i, j, m, n] = A[i, j, k, l] * B[k, l, m, n]
    E2[i, j] = D[i, j, m, n] * C[m, n]
end

# Breaks D3 undefined. No idea why
@tullio F3 := begin
    D3 = A[i, j, k, l] * B[k, l, m, n]
    D3[i, j, m, n] * C[m, n]
end

# Breaks D3 undefined
@tullio F3 := begin
    D3 = A[i, j, k, l] * B[k, l, m, n]
    E3 = D3[i, j, m, n] * C[m, n]
end

# Bounds error
D4 = zeros(4, 4, 6, 6)
@tullio F4 := begin
    D4 = A[i, j, k, l] * B[k, l, m, n]
    E4 = D4[i, j, m, n] * C[m, n]
end

# Bounds error
D8 = zeros(4, 4, 6, 6)
@tullio F8[i, j] := begin
    D8 = A[i, j, k, l] * B[k, l, m, n]
    E8 = D8[i, j, m, n] * C[m, n]
end

# Bounds error
D9 = zeros(4, 4, 6, 6)
E9 = zeros(4, 4)
@tullio F9[i, j] := begin
    D9 = A[i, j, k, l] * B[k, l, m, n]
    E9 = D9[i, j, m, n] * C[m, n]
end

# Returns a sum instead of 2 x 2 matrix. Intended?
D5 = zeros(4, 4, 6, 6)
@tullio F5 := begin
    D5[i, j, m, n] = A[i, j, k, l] * B[k, l, m, n]
    E5 = D5[i, j, m, n] * C[m, n]
end

# Breaks: E6 not defined
D6 = zeros(4, 4, 6, 6)
@tullio F6 := begin
    D6[i, j, m, n] = A[i, j, k, l] * B[k, l, m, n]
    E6[i, j] = D6[i, j, m, n] * C[m, n]
end

# Works!
# D7 needs to be defined outside of scope and captured
# E7 cannot be already defined. See example 9
D7 = zeros(4, 4, 6, 6)
@tullio F7[i, j] := begin
    D7[i, j, m, n] = A[i, j, k, l] * B[k, l, m, n]
    E7 = D7[i, j, m, n] * C[m, n]
end
D7

# Breaks D10 undefined
@tullio F10[i, j] := begin
    D10 = zeros(4, 4, 6, 6)
    D10[i, j, m, n] = A[i, j, k, l] * B[k, l, m, n]
    E10 = D10[i, j, m, n] * C[m, n]
end

```

---

<div class="post-metadata">

**Author:** ![mcabbott](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mcabbott/32/6603_2.png) [@mcabbott](https://discourse.julialang.org/u/mcabbott)\
**Post date:** [March 10, 2022, 9:16pm UTC](https://discourse.julialang.org/t/tullio-multiline-syntax-soft-or-hard-local-scope/77616/6 "2022-03-10T21:16:10Z")

</div>

I said this on the issue too, but: I strongly urge you not to write into an array within the block, like `left[i] := begin TMP[k, l] = ...` .

Tullio wants a pure function of some arrays of numbers, so that the only parallelisation issues it has to think about are those introduced by the sum (and the gradient). Once you write into other arrays, all bets are off.

I don’t see any soft scope weirdness here. All code is wrapped in a function by the macro.

> [@Emmanuel-R8](#):
>
> ```julia
> # Return a sum instead of 4 x 4 matrix. Intended behaviour?
> @tullio C7 := A[i, j, k, l] * B[i, j]
> 
> ```

Yes, this is a complete reduction, since the left hand side has no indices.

> [@Emmanuel-R8](#):
>
> ```julia
> # Breaks D3 undefined. No idea why
> @tullio F3 := begin
> D3 = A[i, j, k, l] * B[k, l, m, n]
> D3[i, j, m, n] * C[m, n]
> end
> 
> # Bounds error
> D4 = zeros(4, 4, 6, 6)
> 
> ```

The first line defines D3 as a scalar, but the second line tries to index it. So I don’t know what this is meant to do. But what happens is that, before it gets there, the code to decide what range of indices to iterate over does something like this:

```julia
ax_i = axes(A,1)
ax_i == axes(D3,1) || error("index i must match")

```

Supplying a different `D3` outside the loop may fool this, but won’t help inside the loop.

---

<div class="post-metadata">

**Author:** ![Emmanuel-R8](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/emmanuel-r8/32/11839_2.png) [@Emmanuel-R8](https://discourse.julialang.org/u/Emmanuel-R8)\
**Post date:** [March 11, 2022, 2:51am UTC](https://discourse.julialang.org/t/tullio-multiline-syntax-soft-or-hard-local-scope/77616/7 "2022-03-11T02:51:04Z")

</div>

FYI.

I am continuing this thread on the Gihub issue. It seems a more appropriate location.

See [https://github.com/mcabbott/Tullio.jl/issues/143](https://github.com/mcabbott/Tullio.jl/issues/143) in case you are interested.