# Overflow with Iterators.product

**URL:** https://discourse.julialang.org/t/overflow-with-iterators-product/116532
**Category:** General Usage
**Tags:** question, bug, iterators
**Created:** [July 2, 2024, 3:52pm UTC](https://discourse.julialang.org/t/overflow-with-iterators-product/116532 "2024-07-02T15:52:37Z")
**Posts on this page:** 1
**Showing post:** 2

<div class="post-metadata">

### Author: ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)
#### Post date: [July 2, 2024, 4:34pm UTC](https://discourse.julialang.org/t/overflow-with-iterators-product/116532/2 "2024-07-02T16:34:24Z")

</div>

> [@minjaep](#):
>
> This sometimes works, sometimes does not. I rand this code 4 times, and it gave this error on my 4th run. (The number of runs to get an error is arbitrary for each try.)

I can’t reproduce your error. Your posted code works fine for me. I also tried putting your code into a function to make it a bit faster, and accumulated a sum of the data in the loop to make sure the compiler doesn’t optimize it away:

```julia
function f(data, L, d)
    s = zero(eltype(data))
    for n in Iterators.product(repeat([1:L],d)...)
        s += data[n...]
    end
    return s
end

```

and this also runs without error for me on your example inputs `L = 300; d = 3; data = randn((L,L,L));`.

Are you sure you didn’t accidentally change `L` or `d` to be inconsistent with `data`?

> [@minjaep](#):
>
> In addition, are there better ways to get d-dimensional indices than Iterators.product(repeat([1:L],d)…)?

Much, much better ways. First, for efficiency, you _really_ want the number of loops (`d`) to be a compile-time constant here, and to use tuples rather than arrays so that the compiler knows the length. A simple modification of your code is to use:

```julia
Iterators.product(axes(data)...)

```

instead. However, it’s even cleaner to use

```julia
CartesianIndices(data)

```

instead — that’s what `CartesianIndices` are for. Note that iterating over `CartesianIndices` returns a `CartesianIndex`, not a tuple, but you can efficiently convert it to a tuple via `Tuple(index)`. In particular, you can do:

```julia
for n in CartesianIndices(data)
    s += data[n] # no splatting required
end

```

or:

```julia

for n in CartesianIndices(data)
    t = Tuple(n)
    s += data[t...] # splatting a tuple of indices
end

```

Quantitatively, your original `repeat` method gives (via BenchmarkTools.jl):

```julia
julia> @btime f($data, $L, $d);
  6.186 s (162000023 allocations: 5.63 GiB)

```

due to the type instability of your runtime-dependent number of loops (`d`). Using `Iterators.product(axes(data)...)` gives me a **200× speedup** :

```julia
julia> @btime f($data);
  31.091 ms (0 allocations: 0 bytes)

```

and using `CartesianIndices(data)` gives about the same performance, but more elegant code:

```julia
julia> @btime f($data);
  31.013 ms (0 allocations: 0 bytes)

```

PS. Please quote your code in the future to make it easier to read and to copy-paste. (I edited your post to fix this.) See [PSA: how to quote code with backticks](https://discourse.julialang.org/t/psa-how-to-quote-code-with-backticks/7530)

---

_[View the full topic](https://discourse.julialang.org/t/overflow-with-iterators-product/116532)._