# How to get around lack of indexing to Base.Iterators.product to allow threaded access

**URL:** <https://discourse.julialang.org/t/how-to-get-around-lack-of-indexing-to-base-iterators-product-to-allow-threaded-access/46118>\
**Category:** New to Julia\
**Tags:** question, multithreading\
**Created:** [September 5, 2020, 7:04pm UTC](https://discourse.julialang.org/t/how-to-get-around-lack-of-indexing-to-base-iterators-product-to-allow-threaded-access/46118 "2020-09-05T19:04:19Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![mkarikom](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mkarikom/32/7096_2.png) [@mkarikom](https://discourse.julialang.org/u/mkarikom)\
**Post date:** [September 5, 2020, 7:04pm UTC](https://discourse.julialang.org/t/how-to-get-around-lack-of-indexing-to-base-iterators-product-to-allow-threaded-access/46118/1 "2020-09-05T19:04:19Z")

</div>

The following loop works without threading but fails because it cant seem to index the iterator:

```julia
arr1 = [5:10;]
arr2 = [20:5:30;]
comb = Base.Iterators.product(arr1,arr2)

function threadedComb(comb)
    Threads.@threads for c in comb
        println(c)
    end
end

function singleComb(comb)
    for c in comb
        println(c)
    end
end

```

> **Without threading**
>
> ```julia
> singleComb(comb)
> (5, 20)
> (6, 20)
> (7, 20)
> (8, 20)
> (9, 20)
> (10, 20)
> (5, 25)
> (6, 25)
> (7, 25)
> (8, 25)
> (9, 25)
> (10, 25)
> (5, 30)
> (6, 30)
> (7, 30)
> (8, 30)
> (9, 30)
> (10, 30)
> 
> ```

With threading:

```julia
threadedComb(comb)
ERROR: TaskFailedException:
MethodError: no method matching firstindex(::Base.Iterators.ProductIterator{Tuple{Array{Int64,1},Array{Int64,1}}})
Closest candidates are:
  firstindex(::Core.SimpleVector) at essentials.jl:601
  firstindex(::Cmd) at process.jl:639
  firstindex(::Base64.Buffer) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.4/Base64/src/buffer.jl:18
  ...
Stacktrace:
 [1] (::var"#409#threadsfor_fun#37"{Base.Iterators.ProductIterator{Tuple{Array{Int64,1},Array{Int64,1}}}})(::Bool) at ./threadingconstructs.jl:46
 [2] (::var"#409#threadsfor_fun#37"{Base.Iterators.ProductIterator{Tuple{Array{Int64,1},Array{Int64,1}}}})() at ./threadingconstructs.jl:28
Stacktrace:
 [1] wait(::Task) at ./task.jl:267
 [2] macro expansion at ./threadingconstructs.jl:69 [inlined]
 [3] threadedComb(::Base.Iterators.ProductIterator{Tuple{Array{Int64,1},Array{Int64,1}}}) at /mnt/evo512/insync/Software_a1/testCUDA/test_threaded_combos.jl:8
 [4] top-level scope at /mnt/evo512/insync/Software_a1/testCUDA/test_threaded_combos.jl:48

```

How can I modify `threadedComb` to allow threaded access?

---

<div class="post-metadata">

**Author:** ![pbayer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pbayer/32/11675_2.png) [@pbayer](https://discourse.julialang.org/u/pbayer)\
**Post date:** [September 5, 2020, 8:05pm UTC](https://discourse.julialang.org/t/how-to-get-around-lack-of-indexing-to-base-iterators-product-to-allow-threaded-access/46118/2 "2020-09-05T20:05:07Z")

</div>

```julia
function threadedComb(comb)
    Threads.@threads for c in Tuple(comb)
        println(c)
    end
end

```

works

---

<div class="post-metadata">

**Author:** ![jling](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jling/32/212909_2.png) [@jling](https://discourse.julialang.org/u/jling)\
**Post date:** [September 5, 2020, 8:17pm UTC](https://discourse.julialang.org/t/how-to-get-around-lack-of-indexing-to-base-iterators-product-to-allow-threaded-access/46118/3 "2020-09-05T20:17:36Z")

</div>

you need to make it “not iterator” first because threads would access later index at the same time; a common function to use is `collect`

---

<div class="post-metadata">

**Author:** ![mkarikom](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mkarikom/32/7096_2.png) [@mkarikom](https://discourse.julialang.org/u/mkarikom)\
**Post date:** [September 5, 2020, 8:41pm UTC](https://discourse.julialang.org/t/how-to-get-around-lack-of-indexing-to-base-iterators-product-to-allow-threaded-access/46118/4 "2020-09-05T20:41:27Z")

</div>

Thanks @jling, is this implicitly done in the solution by @pbayer? I like the tuple method because of immutability.

---

<div class="post-metadata">

**Author:** ![jling](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jling/32/212909_2.png) [@jling](https://discourse.julialang.org/u/jling)\
**Post date:** [September 5, 2020, 8:48pm UTC](https://discourse.julialang.org/t/how-to-get-around-lack-of-indexing-to-base-iterators-product-to-allow-threaded-access/46118/5 "2020-09-05T20:48:35Z")

</div>

yes:

```julia
...
In _totuple(#unused#, itr, s) at tuple.jl:258
>258 _totuple(::Type{Tuple}, itr, s...) = (collect(Iterators.rest(itr,s...))...,)

```

```julia
julia> @btime Tuple($comb);
  596.106 ns (20 allocations: 1.22 KiB)

julia> @btime collect($comb);
  39.652 ns (1 allocation: 368 bytes)

```

and because `Tuple` is `collet` then `splat` (in this case), it’s slower

---

<div class="post-metadata">

**Author:** ![tkf](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tkf/32/17635_2.png) [@tkf](https://discourse.julialang.org/u/tkf)\
**Post date:** [September 5, 2020, 10:24pm UTC](https://discourse.julialang.org/t/how-to-get-around-lack-of-indexing-to-base-iterators-product-to-allow-threaded-access/46118/6 "2020-09-05T22:24:27Z")

</div>

FYI [GitHub - JuliaFolds/FLoops.jl: Fast sequential, threaded, and distributed for-loops for Julia—fold for humans™](https://github.com/JuliaFolds/FLoops.jl) and [https://github.com/tkf/ThreadsX.jl](https://github.com/tkf/ThreadsX.jl) support `Iterators.product`. They can also be used with other iterable types supported by [https://github.com/JuliaFolds/SplittablesBase.jl](https://github.com/JuliaFolds/SplittablesBase.jl). It may be better than `collect`ing the iterator first when the resulting array is large.