Way to efficiently view lists of lists as a read-only flattened list?

Let’s say you have a list of lists that you want to view as a single list:

super_list = [
    [ 1 , 2 , ... , 1001234 ],
    [ 6 , 1 , ... , 3 ],
    [ 42 ]

# _squash is some function that does read-only vcat'ing
squashed_list = _squash(super_list)

@assert in(42, squashed_list)

What is an efficient way to make a view over these lists, with indexing as if they were all vcat’ed together?

// this treats the squashed_list as a read-only array

Iterators.flatten(list) should be your solution, it does not allocate.

If you need indexing you could use IterTools.nth function over the flattened vector.

It looks like Iterators.jl is deprecated, and that functionality isn’t replaced. So I dont think this resolves the issue.

It got deprecated because a lot of the functionality moved into Base. See here for the flatten function.

The old packages README doesn’t really make that clear, maybe a PR indicating the change would be good.

Thanks. I was confused because neither flatten nor Iterators are exported for me. I have to use using Base.Iterators to get it. That’s expected, right?

No, the module Iterators is in the namespace, so just use Iterators.flatten directly.

thanks. I’ll look into what’s going on, then.

Like @djsegal, I’ve been struggling to find an efficient idiomatic way to reduce a function over the individual elements of an Array{Array{T}}. Even when the reducing function doesn’t need to allocate, say something like max, I still see quite a bit of allocations.

x = [rand(100) for i in 1:50]
# assuming f itself doesn't allocate - e.g. max
f1(x) = reduce(max, Iterators.flatten(x))
@benchmark f1($x)
#  memory estimate:  312.52 KiB
#  allocs estimate:  10001

f2(x) = reduce(max, (z for y=x for z=y))
@benchmark f2($x)
#  memory estimate:  314.88 KiB
#  allocs estimate:  10102

I would appreciate it if anyone could suggest a way to achieve this without allocations. It would be a shame to have to write a for loop.