# Nested Comprehensions with Stateful
**URL:** https://discourse.julialang.org/t/nested-comprehensions-with-stateful/58888
**Category:** General Usage
**Tags:** bug
**Created:** [April 8, 2021, 11:34pm UTC](https://discourse.julialang.org/t/nested-comprehensions-with-stateful/58888 "2021-04-08T23:34:25Z")
**Posts on this page:** 7
**Page:** 1
### Author:  [@cortner](https://discourse.julialang.org/u/cortner)
#### Post date: [April 8, 2021, 11:34pm UTC](https://discourse.julialang.org/t/nested-comprehensions-with-stateful/58888/1 "2021-04-08T23:34:25Z")
A `Stateful` iterator “empties” itself as one iterates through it. The result is the following fun behaviour (if one thinks about for a minute it becomes obvious why this happens):
```julia
using Base.Iterators: Stateful
x = [1,2,3,4]
# INTENDED BEHAVIOUR
A = [a * b for a in x, b in x]
# 4×4 Matrix{Int64}:
# 1 2 3 4
# 2 4 6 8
# 3 6 9 12
# 4 8 12 16
# UNINTENDED (I THINK?!)
B = [a * b for a in Stateful(x), b in Stateful(x)]
# 3×3 Matrix{Int64}:
# 1 4 0
# 2 0 0
# 3 0 0
```
I understand why this happens and it is easy enough to avoid, but I was simply curious: is this really intended? Or a bug in `Stateful`? or a bug in nested comprehensions?
P.S.: More generally this will of course happen to every iterator that cannot re-initialize after looping through it.
```julia
```
---
### Author:  [@ettersi](https://discourse.julialang.org/u/ettersi)
#### Post date: [April 9, 2021, 2:33am UTC](https://discourse.julialang.org/t/nested-comprehensions-with-stateful/58888/2 "2021-04-09T02:33:11Z")
I’d argue it is a bug in nested comprehensions that `[i for i in I, j in 1:2]` does not throw an error if the length of `I` shrinks from `j = 1` to `j = 2`. Everything else is as intended, I guess?
---
### Author:  [@cortner](https://discourse.julialang.org/u/cortner)
#### Post date: [April 9, 2021, 2:49am UTC](https://discourse.julialang.org/t/nested-comprehensions-with-stateful/58888/3 "2021-04-09T02:49:12Z")
really? Would one not expect that `[f(i,j) for i in I, j in J]` gives the same result as `[f(i,j) for i in collect(I), j in collect(J)]`?
Also note that if I use nested for loops
```julia
for a in Stateful(x), j in Stateful(x)
# ...
end
```
the behaviour is exactly as I would have expected.
---
### Author:  [@tkf](https://discourse.julialang.org/u/tkf)
#### Post date: [April 9, 2021, 3:00am UTC](https://discourse.julialang.org/t/nested-comprehensions-with-stateful/58888/4 "2021-04-09T03:00:30Z")
Just to throw more confusions into the mix 🙂
```julia
julia> [a * b for a in Iterators.Stateful(1:4), b in Iterators.Stateful(1:4)] # as in the OP
3×3 Matrix{Int64}:
1 4 0
2 0 0
3 0 0
julia> [a * b for a in Iterators.Stateful(1:4), b in Iterators.Stateful(1:4) if true]
4-element Vector{Int64}:
1
2
3
4
julia> [a * b for a in Iterators.Stateful(1:4) for b in Iterators.Stateful(1:4)]
16-element Vector{Int64}:
1
2
3
4
2
4
6
8
3
6
9
12
4
8
12
16
```
If you know the implementation, you can see that these are all leaky abstractions. Not sure what was the intention of the design, though.
---
### Author:  [@ettersi](https://discourse.julialang.org/u/ettersi)
#### Post date: [April 9, 2021, 3:19am UTC](https://discourse.julialang.org/t/nested-comprehensions-with-stateful/58888/5 "2021-04-09T03:19:01Z")
If I understand correctly, the issue is whether
```julia
for i in Is(), j in Js()
[loop body]
end
```
should be lowered to
```julia
# Approach A
cJs = Js()
cIs = Is()
for j in cJs
for i in cIs
[loop body]
end
end
```
or
```julia
# Approach B
cJs = Js()
for j in cJs
cIs = Is()
for i in cIs
[loop body]
end
end
```
It seems that `for` loops currently follow Approach B, while list comprehensions do something even more complicated.
---
### Author:  [@ettersi](https://discourse.julialang.org/u/ettersi)
#### Post date: [April 9, 2021, 6:08am UTC](https://discourse.julialang.org/t/nested-comprehensions-with-stateful/58888/6 "2021-04-09T06:08:01Z")
If you want to make your code work with the current definition of list comprehensions, then this should work:
```julia
struct Stateless{GI}
generate_iterable::GI
end
Base.IteratorSize(::Type{Stateless{GI}}) where {GI<:Function} = Base.IteratorSize(GI.instance())
Base.IteratorSize(::Type{Stateless{GI}}) where {GI} = Base.SizeUnknown()
Base.IteratorEltype(::Type{Stateless{GI}}) where {GI<:Function} = Base.IteratorEltype(GI.instance())
Base.IteratorEltype(::Type{Stateless{GI}}) where {GI} = Base.EltypeUnknown()
Base.eltype(::Type{Stateless{GI}}) where {GI<:Function} = Base.eltype(GI.instance())
Base.length(s::Stateless) = length(s.generate_iterable())
Base.size(s::Stateless, dim...) = size(s.generate_iterable(), dim...)
function Base.iterate(s::Stateless)
iter = s.generate_iterable()
tmp = iterate(iter)
isnothing(tmp) && return nothing
next,state = tmp
return next, (iter,state)
end
function Base.iterate(s::Stateless, (iter,state))
tmp = iterate(iter,state)
isnothing(tmp) && return nothing
next,state = tmp
return next, (iter,state)
end
```
Example:
```julia
julia> [(a,b) for a in Stateless(()->Stateful(1:4)), b in Stateless(()->Stateful(1:4))]
4×4 Array{Tuple{Int64,Int64},2}:
(1, 1) (1, 2) (1, 3) (1, 4)
(2, 1) (2, 2) (2, 3) (2, 4)
(3, 1) (3, 2) (3, 3) (3, 4)
(4, 1) (4, 2) (4, 3) (4, 4)
```
---
### Author:  [@cortner](https://discourse.julialang.org/u/cortner)
#### Post date: [April 10, 2021, 2:05pm UTC](https://discourse.julialang.org/t/nested-comprehensions-with-stateful/58888/7 "2021-04-10T14:05:37Z")
Thanks for that suggestion.