# \[ANN\] View common generators as arrays
**URL:** https://discourse.julialang.org/t/ann-view-common-generators-as-arrays/39707
**Category:** Package Announcements
**Tags:** array, generator
**Created:** [May 18, 2020, 5:30pm UTC](https://discourse.julialang.org/t/ann-view-common-generators-as-arrays/39707 "2020-05-18T17:30:23Z")
**Posts on this page:** 9
**Page:** 1
### Author:  [@klacru](https://discourse.julialang.org/u/klacru)
#### Post date: [May 18, 2020, 5:30pm UTC](https://discourse.julialang.org/t/ann-view-common-generators-as-arrays/39707/1 "2020-05-18T17:30:23Z")
The new package [GeneratorArrays](https://github.com/KlausC/GeneratorArrays.jl) effectively converts a generator into a lazy form of a read-only abstract array.
The object `array(f(i) for i in a)` can be used like `[f(i) for i in a]` without using an extra array for storage. If `a` is an `AbstractArray` it is equivalent to `MappedArrays.mappedarray(f, a)`.
Example:
```julia
julia> g = ( i - j for i in 1:5, j in 2:4)
Base.Generator{Base.Iterators.ProductIterator{Tuple{UnitRange{Int64},UnitRange{Int64}}},var"#56#57"}(var"#56#57"(), Base.Iterators.ProductIterator{Tuple{UnitRange{Int64},UnitRange{Int64}}}((1:5, 2:4)))
julia> array(g)
5×3 GeneratorArrays.GeneratorArray{Int64,2,Base.Generator{Base.Iterators.ProductIterator{Tuple{UnitRange{Int64},UnitRange{Int64}}},var"#56#57"}}:
-1 -2 -3
0 -1 -2
1 0 -1
2 1 0
3 2 1
julia> array(g)[5,2]
2
```
---
### Author:  [@oschulz](https://discourse.julialang.org/u/oschulz)
#### Post date: [May 18, 2020, 6:04pm UTC](https://discourse.julialang.org/t/ann-view-common-generators-as-arrays/39707/2 "2020-05-18T18:04:57Z")
Hm, wouldn’t this fit perfectly into LazyArrays.jl, as another subtype of `LazyArray`?
---
### Author:  [@jlapeyre](https://discourse.julialang.org/u/jlapeyre)
#### Post date: [May 18, 2020, 6:37pm UTC](https://discourse.julialang.org/t/ann-view-common-generators-as-arrays/39707/3 "2020-05-18T18:37:36Z")
100 thumbs up
```julia
julia> using StatsBase; using GeneratorArrays
julia> countmap(array(rand(1:3) for _ in 1:10^5))
Dict{Int64,Int64} with 3 entries:
2 => 33428
3 => 33235
1 => 33337
```
This really fits well with the philosophy of doing the conversion at the call site.
---
### Author:  [@tbeason](https://discourse.julialang.org/u/tbeason)
#### Post date: [May 18, 2020, 6:43pm UTC](https://discourse.julialang.org/t/ann-view-common-generators-as-arrays/39707/4 "2020-05-18T18:43:31Z")
How does element access happen? Am I correct that this might only be faster if you are only accessing each element a handful of times?
---
### Author:  [@mbauman](https://discourse.julialang.org/u/mbauman)
#### Post date: [May 18, 2020, 6:52pm UTC](https://discourse.julialang.org/t/ann-view-common-generators-as-arrays/39707/5 "2020-05-18T18:52:59Z")
Generators are really just an anonymous function + an iteration space. They (currently) only support iteration, and folks generally assume that they’ll iterate sequentially, but this package reaches into their internals to compute the axes of the iteration space and appropriately call the anonymous function on demand upon indexing.
We’re slowly working our way along this path in base itself, and I’d love to see us keep sliding down this slope.
---
### Author:  [@stevengj](https://discourse.julialang.org/u/stevengj)
#### Post date: [May 18, 2020, 8:51pm UTC](https://discourse.julialang.org/t/ann-view-common-generators-as-arrays/39707/6 "2020-05-18T20:51:20Z")
> [@klacru](#):
>
> The new package [GeneratorArrays](https://github.com/KlausC/GeneratorArrays.jl) effectively converts a generator into a lazy form a´of an read-only abstract array.
Couldn’t this be implemented as a thin wrapper around [MappedArrays](https://github.com/JuliaArrays/MappedArrays.jl), i.e. just by a constructor for a `MappedArray` from a `Generator`?
---
### Author:  [@klacru](https://discourse.julialang.org/u/klacru)
#### Post date: [May 20, 2020, 7:38am UTC](https://discourse.julialang.org/t/ann-view-common-generators-as-arrays/39707/7 "2020-05-20T07:38:58Z")
> Am I correct that this might only be faster if you are only accessing each element a handful of times?
Like many “lazy” approaches, this implementation trades space for speed. But the main design goal was not to save space but to make the generator syntax available for all the methods expecting abstract arrays, for a wide range of generators.
You will see speed advantages, if the saved space is huge and/or you want only to access some of the indices. There is room for performance improvements specially for the product iterator case.
---
### Author:  [@klacru](https://discourse.julialang.org/u/klacru)
#### Post date: [May 20, 2020, 7:50am UTC](https://discourse.julialang.org/t/ann-view-common-generators-as-arrays/39707/8 "2020-05-20T07:50:17Z")
> wouldn’t this fit perfectly into LazyArrays.jl?
> Couldn’t this be implemented as a thin wrapper around MappedArrays?
There are now `MappedArrays`, `LazyArray`, and `GeneratorArrays`, which seem to overlap. But I think that is only the case in the most elementary situations and each of the packages adds another dimension of features.
In the base case, these are more or less equivalent:
```julia
map(f, A) => LazyArrays.applied(f, A)
map(f, A) => MappedArrays.mappedarray(f, A)
[x for x in A] => GeneratorArrays.array(x for x in A)
f.(A) => non-materializing form for broadcasting - new syntax?
```
In contrast to the other approaches, `GeneratorArray` adds two stand-alone features.
1. It makes generators and iterators usable as arguments of methods, which accept `AbstractArray`; at least for some widely used forms of generators.
2. It re-establishes or creates (array-) structure which has been lost by the generator notation. The `ProductIterator` forms higher dimensions as space-product of lower dimensions.
For these reasons I think, the package provides enough orthogonality to justify its independence and should be maybe moved to `JuliaArrays` besides the other two.
---
### Author:  [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)
#### Post date: [May 21, 2020, 7:12am UTC](https://discourse.julialang.org/t/ann-view-common-generators-as-arrays/39707/9 "2020-05-21T07:12:32Z")
> [@mbauman](#):
>
> folks generally assume that they’ll iterate sequentially
If that should not be assumed, the manual should mention that (sorry if I missed it). TBH, I was under the impression that generators _do_ iterate sequentially, as they are closely related to `collect`.