How is destructuring transformed\lowered

[ndinsmore]

I have seen it mentioned before that:

function f((x,y))

is effectively transformed to

function f(temp)
    x , y = temp

But I don’t think that it is documented anywhere how x , y = temp is transformed.

Could somebody please explain?

[mbauman]

@ndinsmore

x , y = temp

julia> Meta.@lower  x , y = temp
:($(Expr(:thunk, CodeInfo(
 1 ─ %1 = (Base.indexed_iterate)(temp, 1)                                         │
 │   %2 = (Core.getfield)(%1, 1)                                                  │
 │        x = %2                                                                  │
 │        #s1 = (Core.getfield)(%1, 2)                                            │
 │   %5 = (Base.indexed_iterate)(temp, 2, #s1)                                    │
 │   %6 = (Core.getfield)(%5, 1)                                                  │
 │        y = %6                                                                  │
 └──      return temp                                                             │
))))

It’s just a slightly fancier version of manually unrolled iteration.

[ndinsmore]

How about the destructuring of next in the lowering of a for loop?
Does it get lower straight to a getfield?

[kristoffer.carlsson]

What do you mean next?

[ndinsmore]

Interfaces · The Julia Language

for i in iter   # or  "for i = iter"
    # body
end

is translated into:

next = iterate(iter)
while next !== nothing
    (i, state) = next
    # body
    next = iterate(iter, state)
end

I know that portion of the documenation is out of date it is more like:

next = iterate(iter)
next === nothing && @goto loop_end
    @label loop_head
    (i, state) = next
    # body
    next = iterate(iter, state)
    next === nothing && @goto loop_end
    @goto loop_head
@label loop_end

But it appears that lowering of for today may be breaking the interface by skiping the indexed_iterate of destructuring and going straight to:

next = iterate(iter)
next === nothing && @goto loop_end
    @label loop_head
    i = getfield(next,1)
    state = getfield(next,2)
    # body
    next = iterate(iter, state)
    next === nothing && @goto loop_end
    @goto loop_head
@label loop_end

[mbauman]

Yup. We should add an assertion to guarantee that iterate returns a tuple like it’s supposed to. I’d guess it’s this way because for loops are extraordinarily performance sensitive.

for loop lowering assumes iterate returns a tuple

[ndinsmore]

I think I understand the reasoning but check out the benchmark here

Iterator transformation results in 40% time reduction in Iteration over arrays

Recently it has bugged me that for i in 1:length(array) @inbounds v=array[i] body ... end Was on the order of 30 % faster than: for v in array body ... end So I wanted to see if I could transform the latter into the former. The solution I came up with is a little hacky at the moment but it works & doesn’t change the two argument iterate construct. This has been updated to include a fix to include getindex. # B original iterator type # I new iterator type # IE eltype of I …

@ndinsmore

For the mean time we need the following functions for benchmarking:

function for_sum(iter)
    ret=eltype(iter)(0)
    for v in iter
        ret += v
    end
    return ret
end

# goto_sum is needed because currently for is not lowered with the proper destructuring of next
function goto_sum(iter)
    ret=eltype(iter)(0)
    next = iterate(iter)
    (next === nothing) && @goto loop_end
    @label loop_head
        (v, state) = next
        ret+=v
        next = iterate(iter, state)
        next === nothing &&  @goto loop_end
        @goto loop_head
    @label loop_end
    return ret
end

Lets sanity check their equivlence:

julia> a=collect(1:1000);

julia> for_sum(a)
500500

julia> goto_sum(a)
500500

julia> @benchmark for_sum($a)
BenchmarkTools.Trial: 
  memory estimate:  0 bytes
  allocs estimate:  0
  --------------
  minimum time:     78.688 ns (0.00% GC)
  median time:      83.785 ns (0.00% GC)
  mean time:        91.614 ns (0.00% GC)
  maximum time:     883.004 ns (0.00% GC)
  --------------
  samples:          10000
  evals/sample:     964

julia> @benchmark goto_sum($a)
BenchmarkTools.Trial: 
  memory estimate:  0 bytes
  allocs estimate:  0
  --------------
  minimum time:     78.688 ns (0.00% GC)
  median time:      83.778 ns (0.00% GC)
  mean time:        91.086 ns (0.00% GC)
  maximum time:     793.855 ns (0.00% GC)
  --------------
  samples:          10000
  evals/sample:     957