How to access partitions used in previous iterations of a loop over a partitioned set?

New to Julia
, ,
1

Suppose I am looping over a partitioned set. I would like to check a condition or compare a value in the current partition to that of a partition used in a previous iteration or to be used in a future iteration. What would be the best way to do so?

In an unpartitioned set I could perform comparisons like

set = rand(collect(1:10),100)

for k in eachindex(set)
    if k >1 
      println(max(set[k], set[k-1]))
    end
end

But how would something similar be accomplished in a partitioned set of n elements? Say the goal is to compare the ith element of a partition to the corresponding element in the previous iteration. Where i = n it would be something like.

n = 5
g = Iterators.partition(set,n)

for (k,x) in enumerate(g)
    if k >1 
       println( max( x[end] , set[(k-1)*n]))
    end
end

Could I access the partition x from the k-1 iteration or more generally the k +/- j iteration directly without having to reference set?

Because this would make it easier for situations where i != n or for cycling through set continuously where reverting to a simple % would yield an error when set[100 ] is indexed as set[0] e.g.

for (k,x) in enumerate(Iterators.cycle(g))
   if k>1 
     println( max( x[end] , set[((k-1)*n)%100]))
    end
end

Thanks!

2

something like?

prev_set = nothing
for set in sets
    if prev_set === nothing
        prev_set = set
        continue
    end
    max(set, prev_set)
end
1 Like
3

It’s not entirely clear to me what various comparisons you need to do, but perhaps such a scheme (a “second-level” partition, so to speak) can be useful

g2=IterTools.partition(g,2,1)
1 Like
4

I would say that you could handle this need via IterTools.partition(itr, n*j,n)

1 Like
5

or this way for general j-shift

j=3
g = Iterators.partition(set,n)
for (l,r) in zip(g,Iterators.drop(g,j))
    println( max( l[end] , r[end]))
end
1 Like
6

Thanks so much! I didn’t know I could approach it like this. Just to clarify I would have to update prev_set if I wanted to continuously compare to set to the set immediately preceding it?

so it would be like

prev_set = nothing
for set in sets
    if prev_set == nothing
        prev_set = set
        continue
    end
    max(set, prev_set)
    prev_set = set
end

Also just to make sure I understand these soft scope rules correctly. This would only work within the body of a function or in a REPL right? Because Julia does not allow for the assignment to a global variable within a for loop from a file.

7

Outside a function you’d need to add a global there to resolve the scope ambiguity. But you shouldn’t be doing that anyways, just use a function.

Also, a minor detail, use

if isnothing(prevset) ...

instead of prevset == nothing.

It is a detail, but generally recommended because the == could be overloaded to mean something else. At least just to remove the linting in VSCode…

1 Like
8

Thank you so much for pointing this out! I have not tried IterTools in Julia yet but I will look at it tonight, I’m not sure how to mark multiple solutions on the site. If you have a moment, I’m just wondering In terms of the zip solution, is it possible to use it with enumerate or Iterators.rest? Either of the following

for (h,l,r) in enumerate(zip(g,Iterators.drop(g,j)))
      if h % K = 0 
         fun( l[end], r[end])
      end
end 

for (l,r) in Iterators.rest(zip(g,Iterators.drop(g,j), start)
    println(l,r)
end

results in a BoundsError

9

It’s not clear to me what the purpose of these lines is, but still if you change like this you don’t get the BoundsError:

j=2
K=3
for (h,(l,r)) in enumerate(zip(g,Iterators.drop(g,j)))
    if h % K == 0 
       max( l[end], r[end])
    end
end

in these cases, to debug, I try to see what’s inside some parts.
For example if I do collect(zip(g,Iterators.drop(g,j))) I see what the iterator contains and what shape. So I understand that they are tuples of two elements of which the second is, in turn, a tuple.

for the second part what do you want to achieve?

PS

I doubt that it is not clear what the snippet that I have proposed does.
I try to describe it below.

  1. The function g=partition(itr,n) creates the iteratoreg containing the successive blocks of n-elements.
  2. drop(g, j) drops the first j blocks (rest(g,j) would take blocks from j onwards)
  3. zip(g, drop(g,j)) creates the iterator of the pairs of blocks. Thus the first pair is (g[1], g[j+1]) (that is, in fact, the first and j+1-th group of g) ; the second (g[2], g[j+2]) (that is, in fact, the second and the j+2-th group of g), etc. until the shortest iterator is consumed.
1 Like
10

Thanks so much for this detailed explanation. Sorry a quick confirmation on why the added () in (h,(l,r)) removes the bounds error because I want to make sure I understand the dynamic correctly.

Given

j = 2
g = Iterators.partition([1:10;],2)

then

julia> collect(enumerate(zip(g,Iterators.drop(g,j))))
3-element Vector{Tuple{Int64, Tuple{SubArray{Int64, 1, Vector{Int64}, Tuple{UnitRange{Int64}}, true}, SubArray{Int64, 1, Vector{Int64}, Tuple{UnitRange{Int64}}, true}}}}:
 (1, ([1, 2], [5, 6]))
 (2, ([3, 4], [7, 8]))
 (3, ([5, 6], [9, 10]))

Means the x in for x = set has to follow the same format of ( _ ,( _ ,_ ) ) or ( _ , ( _ _ ), ( _ _ )) ?

In terms of the Iterators.rest() I was thinking of a hypothetical that required cycling through the set zip(g,Iterators.drop(g,j)) where cycle() is initiated at position n determined by an input in the function.

In the above example, it would be something like

(1,([2,3], [6,7])) # enumerated first iteration starting at the second position of the `zip` set 
(2,([4,5], [8,9])) # the loop terminates upon the last complete tuple

After playing around with collect as you mentioned, I was able to get something similar with.

collect(enumerate(zip(Iterators.rest(g,2),Iterators.drop(Iterators.rest(g,2),j))))
3-element Vector{Tuple{Int64, Tuple{SubArray{Int64, 1, Vector{Int64}, Tuple{UnitRange{Int64}}, true}, SubArray{Int64, 1, Vector{Int64}, Tuple{UnitRange{Int64}}, true}}}}:
 (1, ([2, 3], [6, 7]))
 (2, ([4, 5], [8, 9]))
 (3, ([6, 7], [10]))
11

depends on what you want to do with it, you can use one or the other of the shapes

julia> cez=collect(enumerate(zip(g,Iterators.drop(g,j))))       
3-element Vector{Tuple{Int64, Tuple{SubArray{Int64, 1, Vector{Int64}, Tuple{UnitRange{Int64}}, true}, SubArray{Int64, 1, Vector{Int64}, Tuple{UnitRange{Int64}}, true}}}}:
 (1, ([1, 2], [5, 6]))
 (2, ([3, 4], [7, 8]))
 (3, ([5, 6], [9, 10]))

julia> (a,(b,c))=cez[1]
(1, ([1, 2], [5, 6]))

julia> a
1

julia> b
2-element view(::Vector{Int64}, 1:2) with eltype Int64:
 1
 2

julia> c
2-element view(::Vector{Int64}, 5:6) with eltype Int64:
 5
 6


# or 

julia> (a,((b1,b2),(c1,c2)))=cez[1]
(1, ([1, 2], [5, 6]))

julia> a
1

julia> b1
1

julia> b2
2

julia> c1
5

julia> c2
6
1 Like