Is it defined behavior to modify `iter` while `for`-looping over it?

If the local i used in the second pass associates to the same object to the i used in the first pass, then they are the same. (my understanding about julia is advanced again☺️)

julia> function test()
           v = Function[]
           for i = S(4)
               push!(v, () -> println("I'm ", i))
               map(x -> x(), v)
           end
       end
test (generic function with 1 method)

julia> test()
I'm [1]
I'm [1, 2]
I'm [1, 2]
I'm [1, 2, 3]
I'm [1, 2, 3]
I'm [1, 2, 3]
I'm [1, 2, 3, 4]
I'm [1, 2, 3, 4]
I'm [1, 2, 3, 4]
I'm [1, 2, 3, 4]

julia> function test()
           v = Function[]
           for i = 1:4
               push!(v, () -> println("I'm ", i))
               map(x -> x(), v)
           end
       end
test (generic function with 1 method)

julia> test()
I'm 1
I'm 1
I'm 2
I'm 1
I'm 2
I'm 3
I'm 1
I'm 2
I'm 3
I'm 4
1 Like

Mutate a variable is mainly for performance (I guess).

Reusing a name (re-bind it to an updated object) is also useful. Although sometimes lead to unexpected results (e.g. capturing), I have to admit that the existing design has its rationale.

@eldee Gave me a hint. I’m sorry but I’m more confused now.

I searched some implementations in the julia github but I fail to find an easy one, e.g. what is the implementation of iterate([2, 3, 4], 2) or iterate(1:3, 2)? (Forgive my poor searching ability)

From your transformation foo to bar here, I notice that item as a local variable within the while loop, lives until the while is quitted. Then how to explain the code in my #21 post? why is the older anonymous function not updated?

You can use @which to find which method is called and where it is defined:

julia> @which iterate([2, 3, 4], 2)
iterate(A::Array, i)
     @ Base array.jl:902

Via @less you can view the source code in your terminal, or via @edit in an editor.

julia> @less iterate([2, 3, 4], 2)  # q to exit
iterate(A::Array, i=1) = (@inline; (i - 1)%UInt < length(A)%UInt ? (@inbounds A[i], i + 1) : nothing)

## Indexing: getindex ##

"""
    getindex(collection, key...)

Retrieve the value(s) stored at the given key or index within a collection. The syntax
`a[i,j,...]` is converted by the compiler to `getindex(a, i, j, ...)`.

See also [`get`](@ref), [`keys`](@ref), [`eachindex`](@ref).

# Examples
```jldoctest
julia> A = Dict("a" => 1, "b" => 2)
Dict{String, Int64} with 2 entries:
  "b" => 2
  "a" => 1

julia> getindex(A, "a")
1
(...)
-- More (33%) --

(The Discourse highlighting is a bit too eager here (in my opinion).) The first line of the output is what you’re looking for:

iterate(A::Array, i=1) = (@inline; (i - 1)%UInt < length(A)%UInt ? (@inbounds A[i], i + 1) : nothing)

There are also more advanced options like Cthulhu.jl or Debugger.jl.

1 Like

I cannot figure out why the output isn’t “I’m 1, // 2, 2, // 3, 3, 3” as the item updates from 1 to 3.

julia> function test()
           iter = 1:3
           v = []
           next = iterate(iter)
           while next !== nothing
               item, state = next
               f = function()
                   println("I'm ", item)
               end
               push!(v, f)
               @info "item = $item"
               map(x -> x(), v)
               next = iterate(iter, state)
               end
       end;

julia> test()
[ Info: item = 1
I'm 1
[ Info: item = 2
I'm 1
I'm 2
[ Info: item = 3
I'm 1
I'm 2
I'm 3

I also don’t understand why in this situation there is no Core.Box, whereas if I shift f above for one line then Core.Box occurs.

(@info is merely for pretty printing, it is not relevant here and should be changed to println)

:smiling_face_with_tear:It’s very brain burning.


Why isn’t

        item, state = next
        f = function()
            println("I'm ", item)
        end

and

        f = function()
            println("I'm ", item)
        end
        item, state = next

THE SAME THING? As I wrote them inside a function. I thought they were the same.

The appearance of Core.box is likely a manifestation of the infamous issue 15276.

Regardless of whether item gets boxed, the value in the closure f should be consistent with the one from the current variable item. When in the second iteration you have item, state = next you create a new variable* called item (with value 2), but it is not the same as the old one (still with value 1). Within v you do still have access to the old one, via the (old) f, i.e. v[1].

This is the same behaviour as in the simpler

julia> function test_simple()
           iter = 1:3
           v = []  # stores the items
           next = iterate(iter)
           while next !== nothing
               item, state = next
               push!(v, item)
               next = iterate(iter, state)
           end
           v
       end;

julia> test_simple()
3-element Vector{Any}:
 1
 2
 3

In my example above the i were also different variables, but they all referred to the same Vector. In the loop this Vector was at various stages of construction, hence the different prints. But when you query all different is at the same time, as in collect you get the same Vector content.


Edit: * Due to scoping rules in loops. If you would manually unroll the loop as in

function test_unrolled()
    iter = 1:2
    v = []
    next = iterate(iter)
    item, state = next
    f = function()
        println("1: I'm ", item)
    end
    push!(v, f)
    @info "item = $item"
    map(f -> f(), v)
    next = iterate(iter, state)
    item, state = next
    f = function()
       println("2: I'm ", item)
    end
    push!(v, f)
    @info "item = $item"
    map(f -> f(), v)
    next = iterate(iter, state)
    item, state = next
    f = function()
        println("3: I'm ", item)
    end
    push!(v, f)
    @info "item = $item"
    map(f -> f(), v)
    next = iterate(iter, state)
end

you get different results

julia>  test_unrolled()
[ Info: item = 1
1: I'm 1
[ Info: item = 2
1: I'm 2
2: I'm 2
[ Info: item = 3
1: I'm 3
2: I'm 3
3: I'm 3

While I still think this is correct, I’m now confused why something like item = -1 does not constitute a new variable as far as the closure is concerned:

julia> function test2()
           iter = 1:3
           v = []
           next = iterate(iter)
           while next !== nothing
               item, state = next
               push!(v, () -> println("I'm ", item)
               item = -1
               map(f -> f(), v)
               next = iterate(iter, state)
           end
       end;

julia> test2()
I'm -1
I'm -1
I'm -1
I'm -1
I'm -1
I'm -1

This still happens if you change the type of item, e.g. item = rand().
(This is what I would expect from my experience and is sort of the point of boxing in closures, but seems inconsistent with my explanation.)


Edit: I guess the explanation in the documentation about assignment in soft scope implies the different item within one iteration are the same variable, but the item across iterations are different variables. The first part sounds a bit contrary to a variable being a name associated to a value, though.

Automatic boxing of variables sucks bigly. The reason is that the decision to box is taken by some heuristic early in the compile pipeline, during lowering, before types are known. That’s why boxed variables have unknown type, i.e. Core.Box is not parametrized by type, unlike Ref. This is also the reason why seemingly innocent changes to the code may affect boxing.