# Iterator Invalidation

**URL:** <https://discourse.julialang.org/t/iterator-invalidation/22979>\
**Category:** General Usage\
**Created:** [April 9, 2019, 11:40pm UTC](https://discourse.julialang.org/t/iterator-invalidation/22979 "2019-04-09T23:40:43Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![JaredCrean2](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jaredcrean2/32/3574_2.png) [@JaredCrean2](https://discourse.julialang.org/u/JaredCrean2)\
**Post date:** [April 9, 2019, 11:40pm UTC](https://discourse.julialang.org/t/iterator-invalidation/22979/1 "2019-04-09T23:40:43Z")

</div>

I wrote some code that looks like

```julia
for val in values # values is a Vector
  idx = findfirst(values, val)
  values[idx] = 0
  if too many zeros in values
    ... move nonzero values to beginning of vector
    resize!(values, number of non zeros)
  end
end

```

What does this code do when `values` gets resized? I looked at the docs for information on what operations invalidate iterators but can’t find it.

---

<div class="post-metadata">

**Author:** ![sagartewari01](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/sagartewari01/32/7810_2.png) [@sagartewari01](https://discourse.julialang.org/u/sagartewari01)\
**Post date:** [April 10, 2019, 5:04am UTC](https://discourse.julialang.org/t/iterator-invalidation/22979/2 "2019-04-10T05:04:47Z")

</div>

In case of arrays, the iteration seems to use indexes. This code:

```julia
a=collect(10:20)
for v in a
    println(v)
    deleteat!(v, 1)
end

```

gives this as output:

10  
12  
14  
16  
18  
20

Not sure about the other collections.

---

<div class="post-metadata">

**Author:** ![Per](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/per/32/10387_2.png) [@Per](https://discourse.julialang.org/u/Per)\
**Post date:** [April 10, 2019, 5:47am UTC](https://discourse.julialang.org/t/iterator-invalidation/22979/3 "2019-04-10T05:47:54Z")

</div>

I would say don’t write code that changes the size of the array that you are iterating over. Even if you get it to work correctly, the code will be very hard to read afterwards. (The statement `val in values` will no longer mean what the reader thinks it means.)

I’d like to give an example of a better way to do the above loop, but I’m not sure what the goal is here? It seems on the first iteration `idx` will be `1`, so `values[1]` will be set to `0`. On the second iteration, either `values[2]` is already `0`, or it will be set to `0` because it is the first element equal to `val`. So in the end all of `values` will be zero?

Edit: Maybe this is what you want:

```julia
while !isempty(values)
    val = popfirst!(values)
    # do something
end

```

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [April 10, 2019, 6:03am UTC](https://discourse.julialang.org/t/iterator-invalidation/22979/4 "2019-04-10T06:03:51Z")

</div>

> [@JaredCrean2](#):
>
> information on what operations invalidate iterators

Iterators `for v in x` are just syntactic sugar for the [iteration interface](https://docs.julialang.org/en/v1/manual/interfaces/#man-interface-iteration-1), most importantly `iterate`. Just imagine your code expanded as described there.

If you happen to change the original object you are iterating over, generally consequences are undefined, So don’t do it 😉

Also, don’t use `findfirst`, since it will be spurious for duplicates. Consider

```julia
for (idx, val) in enumerate(values)
    ...
end

```

---

<div class="post-metadata">

**Author:** ![rdeits](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rdeits/32/286_2.png) [@rdeits](https://discourse.julialang.org/u/rdeits)\
**Post date:** [April 10, 2019, 5:23pm UTC](https://discourse.julialang.org/t/iterator-invalidation/22979/5 "2019-04-10T17:23:18Z")

</div>

Iterators in C++ are, essentially, just raw pointers that you hope will point into the memory owned by your container. Modifying that container might break the assumption that those pointers belong to your container, so in C++ we have to worry a lot about iterator invalidation. In Julia, we don’t usually deal with raw pointers, and instead with have a nice `iterate` interface that results in, roughly, the behavior of a C++ loop like `for (int i = 0; i < v.size(); i++) { do something with v[i]...}`, but without having to keep track of the index variable manually.

So, for example, iterating over a vector while appending to it in Julia is fine:

```julia
julia> v = [1, 2, 3]
f3-element Array{Int64,1}:
 1
 2
 3

julia> for x in v
         @show x
         if x == 1
           push!(v, 4)
         end
       end
x = 1
x = 2
x = 3
x = 4

```

because the code expands to:

```julia
julia> let
         next = iterate(v)
         while next !== nothing
           x, state = next
           @show x
           if x == 1
             push!(v, 4)
           end
           next = iterate(v, state)
         end
       end
x = 1
x = 2
x = 3
x = 4

```

which still checks (by checking if the result of `iterate(v, state)` is `nothing` ) if we’ve reached the end of `v`, even after modification of `v`. In essence, any questions about what is safe or valid to do with an iterator in Julia should be answerable by falling back to that `while` loop construct, as documented in [Interfaces · The Julia Language](https://docs.julialang.org/en/v1/manual/interfaces/index.html#man-interface-iteration-1)

The equivalent C++ code, on the other hand, produces garbage because modifying `v` causes the iterators to silently start pointing to _completely invalid memory_:

```cpp
C++ > #include <iostream>
true

C++ > std::vector<int> v = {1, 2, 3};

C++ > for (auto iter = v.begin(); iter != v.end(); ++iter) {
        std::cout << "x: " << *iter << std::endl;
        if (*iter == 1) {
          v.push_back(4); // oops, `iter` now points to garbage. All bets are off. 
        }
      }
x: 1
x: 0
x: 3
x: 0
x: 956330853
x: 13616
x: 513
x: 0
x: 2020515472

```

---

<div class="post-metadata">

**Author:** ![kristoffer.carlsson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kristoffer.carlsson/32/22_2.png) [@kristoffer.carlsson](https://discourse.julialang.org/u/kristoffer.carlsson)\
**Post date:** [April 10, 2019, 6:05pm UTC](https://discourse.julialang.org/t/iterator-invalidation/22979/6 "2019-04-10T18:05:27Z")

</div>

> [@rdeits](#):
>
> which still checks (by checking if the result of `iterate(v, state)` is `nothing` ) if we’ve reached the end of `v` , even after modification of `v` .

Some iterators hoist the computation of e.g. `length` which could give similar problems in Julia (not sure if any iterator that would be susceptible to this actually does this hoisting though).