Skipping parts of a for loop in the first iteration

I crave a macro that does

@for x in iter begin 
    dosomethings()
    if $first_iteration
        y = dothings_differently(x)
    else
        y = dothings(y, x)
    end
    dootherthings()
end

branch free by rewriting into a preamble and a while loop.
It’s slightly above my head, any input welcome.

Edit: I should say, my hope is of course that someone else now wants this so much that they have to implement it.

4 Likes

Something like this?

using MacroTools: postwalk, @capture

macro metafor(loop)
    firstexpr = nothing
    loop = postwalk(loop) do x
        if @capture(x, for idx_ ∈ iter_ inner_ end)
            quote
                for $idx ∈ $iter
                    $idx == first($iter) && continue
                    $inner
                end
            end
        elseif @capture(x, @first(y_))
            firstexpr = y
            nothing
        else
            x
        end
    end
    esc(quote
        $firstexpr
        $loop
    end)
end

which gives

@metafor for i ∈ 1:5
    @first println("first!")
    println("not first")
end
>>>
first!
not first
not first
not first 
not first

It’s a little messy and can certainly be improved on, but taht should give you a rough idea of a possible implementation. (Of course you could do endless variations based on which code you want to execute in which iteration.)

4 Likes

Thank you I will study this. Now I now why there is MacroTools. My failing attempt without it is already more than twice as much code.

I can contribute now a nice quote body:

quote let ϕ = iterate($iter)
        while ϕ != nothing
            $i, st = ϕ
            $(stms1...)
            while true
                ϕ = iterate($iter, st)
                ϕ != nothing && break
                $i, st = ϕ
                $(stms...)
            end
            break
        end
    end end

Edit: Small fix, still something goes wrong if one uses continue in the first iteration.

1 Like

Alas, I carve this as well but lack the macro-fu to help. Please post here the solution and a MWE once you’re content. Thanks!!!

Writing macros in Julia without the aid of MacroTools is indeed a nightmare. It really should have been in the stdlib. Once you mess around with MacroTools a bit, you’ll probably find it much easier. Just remember: you should think of a macro as a simple syntax transformation and you should always think about how you would write your code without a macro (i.e. the desired output of the macro) before you start writing the macro.

3 Likes

Heureka!

"""
    @unroll1 for-loop

Unroll the first iteration of a `for`-loop.
Set `$first` to true in the first iteration.

Example:
    @unroll1 for i in 1:10
        if $first
            a, state = iterate('A':'Z')
        else
            a, state = iterate('A':'Z', state)
        end
        println(i => a)
    end
"""
macro unroll1(expr)
    @assert expr isa Expr
    @assert expr.head == :for
    iterspec = expr.args[1]

    @assert iterspec isa Expr
    @assert  iterspec.head == :(=)
    i = iterspec.args[1]
    iter = iterspec.args[2]

    body = expr.args[2]


    body_1 = eval(Expr(:let, :(first = true), Expr(:quote, body)))
    body_i = eval(Expr(:let, :(first = false), Expr(:quote, body)))


    quote
        local st, $i
        @goto enter
        while true
            @goto exit
            @label enter
                ϕ = iterate($iter)
                ϕ === nothing && break
                $i, st = ϕ
                $(body_1)
            @label exit
            while true
                ϕ = iterate($iter, st)
                ϕ === nothing && break
                $i, st = ϕ
                $(body_i)
            end
            break
        end
    end
end
1 Like

Cool macro! My only objection would be that readability is a bit reduced, since developers not familiar with that macro might not understand at first glance what the code is doing.

Another alternative:

for (i,c) in enumerate('A':'D')
    if i == 1
        print("first")
    else
        print("not first")
    end
    println(" body $c")
end

Results in:

first body A
not first body B
not first body C
not first body D

A debugged version is in Trajectories.jl: https://github.com/mschauer/Trajectories.jl/blob/master/src/unroll1.jl

1 Like

I think I also like

using IterTools: flagfirst

function fsum(f, xs, default)
    local y
    for (isfirst, x) in flagfirst(xs)
        if isfirst
            y = f(x)
        else
            y = y + f(x)
        end
    end
    if @isdefined(y) 
        return y 
    else 
        return default 
    end
end

Edit: Handle empty iterators

I am not sure if you are aware of IterTools.flagfirst.

4 Likes

Thanks for implementing and pointing out that iterator.

1 Like

I find it more elegant to use Iterators.peel from the stdlib:

function fsum(f, itr)
    (x, rest) = Iterators.peel(itr)
    s = f(x)
    for x in rest
        s += f(x)
    end
    return s
end
3 Likes

That doesn’t help when itr is empty though.

That doesn’t seem to work for IterTools.flagfirst either.

Surprisingly, that is not a problem… see above.

But at this point the code is so long it’s more concise and readable to just do it manually:

function myfsum(f, xs, default)
    (isfirst, y) = (true, default)
    for x in xs
        if isfirst
            (isfirst, y) = (false, f(x))
        else
            y += f(x)
        end
    end
    return y
end

Also, there’s a performance issue:
Edit: performance is equal after restarting Julia.

julia> @btime fsum(sin, xs, 0.0) setup=(xs=rand(1000));
  644.010 μs (11000 allocations: 296.88 KiB)

julia> @btime myfsum(sin, xs, 0.0) setup=(xs=rand(1000));
  7.591 μs (0 allocations: 0 bytes)

julia> @btime fsum(sin, xs, 0.0) setup=(xs=rand(0));
  48.977 ns (2 allocations: 32 bytes)

julia> @btime myfsum(sin, xs, 0.0) setup=(xs=rand(0));
  2.892 ns (0 allocations: 0 bytes)
1 Like

I cannot reproduce your timing problem:

julia> @btime fsum(sin, xs, 0.0) setup=(xs=rand(1000));
  6.548 μs (0 allocations: 0 bytes)

julia> @btime myfsum(sin, xs, 0.0) setup=(xs=rand(1000));
  6.914 μs (0 allocations: 0 bytes)

julia> versioninfo()
Julia Version 1.3.1
Commit 2d5741174c (2019-12-30 21:36 UTC)
Platform Info:
  OS: macOS (x86_64-apple-darwin18.6.0)
  CPU: Intel(R) Core(TM) i5-7360U CPU @ 2.30GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.1 (ORCJIT, skylake)

I like your solution though, it’s clear and still uses the quirky initialisation in the loop trick.

1 Like

Yeah, the difference disappeared after a restart. Not sure what happened.

Not sure since when, but now we have Base.Iterators.drop():

julia> VERSION
v"1.8.0"

julia> for i = Iterators.drop(1:10, 5)  # drop first five
           println(i)
       end
6
7
8
9
10