Tullio - multiline syntax - Soft or hard local scope?

General Usage Performance
1

I have a block of code using Tullio with a begin ... end syntax.

I discovered that, within the block, only = can be used. No := will be acctepted.

My problem now is that on execution that patch_mean is an unknown symbol. If I extract that line as a single @tullio statement, patch_mean is defined, and then patch_centered becomes the unknown symbol.

If I extract all the lines with single statments (thought losing the benefit of better compilation), I get a new error can't apply finaliser without a reduction.

Nothing on Google / Stack Exchange. Any clue?

# Average input values (channel-wise)
@tullio sk[b] := begin
    patch_mean[in_c, b]                 = patch[w, h, in_c, b] / ($k * $k)

    # Second moment adjusted for DEFAULT_ϵ and q
    patch_centered[w, h, in_c, b]       = patch[w, h, in_c, b] - patch_mean[in_c, b]

    # MORE LINES
    final_matrix[b]
end
2

Found a new problem.

MWE:

A = rand(4, 4, 5, 5)
B = rand(4, 4)

@tullio C[k, l] := A[i, j, k, l] * B[i, j]                   # OK
@tullio C[k, l] := A[i, j, k, l] * B[i, j] |> sqrt       # OK
@tullio C[k, l] := sqrt <| A[i, j, k, l] * B[i, j]       # OK

@tullio C[k, l] := begin
    A[i, j, k, l] * B[i, j]
end                                    # OK

@tullio C[k, l] := begin
    A[i, j, k, l] * B[i, j] |> sqrt
end                                   # OK

@tullio C[k, l] := begin
    sqrt <| A[i, j, k, l] * B[i, j]
end                                   # KO
3

The error message seems clear to me:

ERROR: LoadError: UndefVarError: <| not defined

<| simply is not implemented.

Is there an easy way for us to reproduce your original problem?

4

I asked questions on the github issues. Thanks to their answers, part of the errors relate to something I hadn’t understood. Within a begin ... end you need to apply pure Julia syntax. Therefore both := and <| are not defined (not defined in Base) and will yield an error.

I submitted a PR to clarify the docs.

Still tracking the rest. I’ll make a MWE when I can isolate one.

5

A lot of examples with inconsistent results. But bottom line, the behaviour breaks my assumption that macros define a hard local scope (they are hygienic).

What is even weirder is that the macro behaves a mix of soft and hard scope. I am utterly confused.

using Tullio #0.3.3 - Julia 1.6.5

A = rand(4, 4, 5, 5)
B = rand(4, 4)

# OK
@tullio C1[k, l] := A[i, j, k, l] * B[i, j]
C1

# Return a sum instead of 4 x 4 matrix. Intended behaviour?
@tullio C7 := A[i, j, k, l] * B[i, j]

# OK
@tullio C2[k, l] := begin
    A[i, j, k, l] * B[i, j]
end
C1 ≈ C2

# Return a sum instead of 4 x 4 matrix. Intended behaviour?
@tullio C6 := begin
    A[i, j, k, l] * B[i, j]
end


# Breaks because base Julia would expect the matrix to be already existing
@tullio C3[k, l] := begin
    TMP[k, l] = A[i, j, k, l] * B[i, j]
end

# Therefore this works
@tullio C3[k, l] := begin
    TMP = A[i, j, k, l] * B[i, j]
end
C1 ≈ C3

# TMP was just used, but defined in a local scope. Not exported.
# This doesn't work
@tullio C4[k, l] := begin
    TMP[k, l] = A[i, j, k, l] * B[i, j]
    TMP
end
C1 ≈ C4

TMP = zeros(5, 5)
# Breaks with error:
# ERROR: LoadError: MethodError: no method matching zero(::Type{Matrix{Float64}})
@tullio C5[k, l] := begin
    TMP[k, l] = A[i, j, k, l] * B[i, j]
    TMP
end
C1 ≈ C5

TMP = zeros(5, 5)
# But this works
@tullio C5[k, l] := begin
    TMP[k, l] = A[i, j, k, l] * B[i, j]
    TMP[k, l]
end
C1 ≈ C5


##########################################

A = rand(4, 4, 5, 5)
B = rand(5, 5, 6, 6)
C = rand(6, 6)

@tullio D1[i, j, m, n] := A[i, j, k, l] * B[k, l, m, n]
@tullio E1[i, j] := D1[i, j, m, n] * C[m, n]

# Breaks D3 undefined. Exoected with soft local scope. Not within macro that should define the array.
@tullio F2 := begin
    D2[i, j, m, n] = A[i, j, k, l] * B[k, l, m, n]
    E2[i, j] = D[i, j, m, n] * C[m, n]
end

# Breaks D3 undefined. No idea why
@tullio F3 := begin
    D3 = A[i, j, k, l] * B[k, l, m, n]
    D3[i, j, m, n] * C[m, n]
end

# Breaks D3 undefined
@tullio F3 := begin
    D3 = A[i, j, k, l] * B[k, l, m, n]
    E3 = D3[i, j, m, n] * C[m, n]
end


# Bounds error
D4 = zeros(4, 4, 6, 6)
@tullio F4 := begin
    D4 = A[i, j, k, l] * B[k, l, m, n]
    E4 = D4[i, j, m, n] * C[m, n]
end

# Bounds error
D8 = zeros(4, 4, 6, 6)
@tullio F8[i, j] := begin
    D8 = A[i, j, k, l] * B[k, l, m, n]
    E8 = D8[i, j, m, n] * C[m, n]
end

# Bounds error
D9 = zeros(4, 4, 6, 6)
E9 = zeros(4, 4)
@tullio F9[i, j] := begin
    D9 = A[i, j, k, l] * B[k, l, m, n]
    E9 = D9[i, j, m, n] * C[m, n]
end


# Returns a sum instead of 2 x 2 matrix. Intended?
D5 = zeros(4, 4, 6, 6)
@tullio F5 := begin
    D5[i, j, m, n] = A[i, j, k, l] * B[k, l, m, n]
    E5 = D5[i, j, m, n] * C[m, n]
end

# Breaks: E6 not defined
D6 = zeros(4, 4, 6, 6)
@tullio F6 := begin
    D6[i, j, m, n] = A[i, j, k, l] * B[k, l, m, n]
    E6[i, j] = D6[i, j, m, n] * C[m, n]
end

# Works!
# D7 needs to be defined outside of scope and captured
# E7 cannot be already defined. See example 9
D7 = zeros(4, 4, 6, 6)
@tullio F7[i, j] := begin
    D7[i, j, m, n] = A[i, j, k, l] * B[k, l, m, n]
    E7 = D7[i, j, m, n] * C[m, n]
end
D7

# Breaks D10 undefined
@tullio F10[i, j] := begin
    D10 = zeros(4, 4, 6, 6)
    D10[i, j, m, n] = A[i, j, k, l] * B[k, l, m, n]
    E10 = D10[i, j, m, n] * C[m, n]
end
6

I said this on the issue too, but: I strongly urge you not to write into an array within the block, like left[i] := begin TMP[k, l] = ... .

Tullio wants a pure function of some arrays of numbers, so that the only parallelisation issues it has to think about are those introduced by the sum (and the gradient). Once you write into other arrays, all bets are off.

I don’t see any soft scope weirdness here. All code is wrapped in a function by the macro.

Yes, this is a complete reduction, since the left hand side has no indices.

The first line defines D3 as a scalar, but the second line tries to index it. So I don’t know what this is meant to do. But what happens is that, before it gets there, the code to decide what range of indices to iterate over does something like this:

ax_i = axes(A,1)
ax_i == axes(D3,1) || error("index i must match")

Supplying a different D3 outside the loop may fool this, but won’t help inside the loop.

2 Likes
7

FYI.

I am continuing this thread on the Gihub issue. It seems a more appropriate location.

See https://github.com/mcabbott/Tullio.jl/issues/143 in case you are interested.