# Memory Comprehensions

**URL:** <https://discourse.julialang.org/t/memory-comprehensions/129033>\
**Category:** General Usage\
**Tags:** array, memory, arrays, comprehension\
**Created:** [May 15, 2025, 12:46pm UTC](https://discourse.julialang.org/t/memory-comprehensions/129033 "2025-05-15T12:46:05Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![PatrickHaecker](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/patrickhaecker/32/222891_2.png) [@PatrickHaecker](https://discourse.julialang.org/u/PatrickHaecker)\
**Post date:** [May 15, 2025, 12:46pm UTC](https://discourse.julialang.org/t/memory-comprehensions/129033/1 "2025-05-15T12:46:05Z")

</div>

Is there a `Memory` comprehension in Julia, which I missed so far?

We have [`Array` comprehensions](https://docs.julialang.org/en/v1/manual/arrays/#man-comprehensions), but when you don’t need the multiple dimensions and resizing properties of an `Array` and a `Memory` is enough, it would be nice to construct a Memory directly without going through a (non-public) field access of an `Array` or an uninitialized `Memory` and a loop.

`[… for …]` and `(… for …)` are already used. `{… for …}` might be free: But are the cases so common that a dedicated syntax is necessary? On the one hand, Julia’s performance would probably benefit from everyone thinking about whether the functionalities of `Array` are really needed and make sense. On the other hand, it only gets rid of some inefficiencies, so not too much to gain here.

Would a `Memory` constructor accepting a [generator](https://docs.julialang.org/en/v1/manual/arrays/#man-generators) make sense? This sounds very useful in general and could be a good compromise between code verbosity and syntax complexity.

Or is there any other already existing elegant solution to the problem?

---

<div class="post-metadata">

**Author:** ![Oscar\_Smith](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oscar_smith/32/25343_2.png) [@Oscar\_Smith](https://discourse.julialang.org/u/Oscar_Smith)\
**Post date:** [May 15, 2025, 12:59pm UTC](https://discourse.julialang.org/t/memory-comprehensions/129033/2 "2025-05-15T12:59:13Z")

</div>

I think the generator answer is probably or best option at this point

---

<div class="post-metadata">

**Author:** ![PatrickHaecker](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/patrickhaecker/32/222891_2.png) [@PatrickHaecker](https://discourse.julialang.org/u/PatrickHaecker)\
**Post date:** [May 16, 2025, 4:36am UTC](https://discourse.julialang.org/t/memory-comprehensions/129033/3 "2025-05-16T04:36:29Z")

</div>

It turned out, that I didn’t need a generator as a function was enough. So I could mirror `ntuple`, which makes sense, when you think about it.

```julia
function memory(f::Function, n::Integer)
    n >= 1 || lazy"Creation of empty `Memory` is not supported using `memory`." |> ArgumentError |> throw
    temp = f(1)
    mem = Memory{typeof(temp)}(undef, n)
    mem[1] = temp
    for i in 2 : n
        mem[i] = f(i)
    end
    return mem
end

julia> memory(i -> 4-i, 4)
4-element Memory{Int64}:
 3
 2
 1
 0

```

However, as `Memory` needs the element type before construction (in contrast to a `Tuple`), this is either a little bit less efficient (as shown) or needs the type as an additional parameter. Using the additional parameter supports the empty `Memory`, too. However, the user effectively then needs to provide the type twice for a non-empty `Memory` (as `f`’s return type and as a separate argument to `memory`).

Improvements welcome!

Should I hand this in to `Base` (or `Core`)?.

---

<div class="post-metadata">

**Author:** ![DNF](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dnf/32/10191_2.png) [@DNF](https://discourse.julialang.org/u/DNF)\
**Post date:** [May 16, 2025, 5:22am UTC](https://discourse.julialang.org/t/memory-comprehensions/129033/4 "2025-05-16T05:22:11Z")

</div>

> [@PatrickHaecker](#):
>
> this is either a little bit less efficient (as shown)

I cannot see where you show this. Could you add a benchmark?

For performance, you could try

```julia-auto
function memory(f::F, n::Integer) where {F} 

```

in order to specialize on the function type (also, it doesn’t have to be a `Function` at all).

---

<div class="post-metadata">

**Author:** ![PatrickHaecker](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/patrickhaecker/32/222891_2.png) [@PatrickHaecker](https://discourse.julialang.org/u/PatrickHaecker)\
**Post date:** [May 16, 2025, 8:09am UTC](https://discourse.julialang.org/t/memory-comprehensions/129033/5 "2025-05-16T08:09:32Z")

</div>

> [@DNF](#):
>
> For performance, you could try
> 
> ```julia
> function memory(f::F, n::Integer) where {F} 
> 
> ```
> 
> in order to specialize on the function type (also, it doesn’t have to be a `Function` at all).

Thanks! This is indeed both slightly faster and more generic.

> [@DNF](#):
>
> I cannot see where you show this. Could you add a benchmark?

Sure. The effect is small, but seems to be consistent here.

```julia
function memory(f::F, n::Integer) where F # use a parameter to enforce specialization
    n >= 1 || lazy"Creation of empty `Memory` is not supported using `memory`." |> ArgumentError |> throw
    temp = f(1)
    mem = Memory{temp |> typeof}(undef, n)
    mem[1] = temp
    for i in 2 : n
        mem[i] = f(i)
    end
    return mem
end

function memory(f::F, T::DataType, n::Integer) where F # use a parameter to enforce specialization
    mem = Memory{T}(undef, n)
    for i in 1 : n
        mem[i] = f(i)
    end
    return mem
end

julia> @b memory(zero, $7)
9.653 ns (1 allocs: 80 bytes)

julia> @b memory(zero, Int, $7)
9.539 ns (1 allocs: 80 bytes)

```

---

<div class="post-metadata">

**Author:** ![Benny](https://avatars.discourse-cdn.com/v4/letter/b/49beb7/32.png) [@Benny](https://discourse.julialang.org/u/Benny)\
**Post date:** [May 16, 2025, 8:17am UTC](https://discourse.julialang.org/t/memory-comprehensions/129033/6 "2025-05-16T08:17:57Z")

</div>

I wouldn’t put much stock in \<1ns or \<10% differences like that, it is just as likely up to random chance or some artifact of your evaluation order.

```julia
julia> @benchmark memory(zero, Int, $7)
BenchmarkTools.Trial: 10000 samples with 998 evaluations per sample.
 Range (min … max): 16.232 ns … 719.639 ns ┊ GC (min … max): 0.00% … 0.00%
 Time (median): 17.836 ns ┊ GC (median): 0.00%
 Time (mean ± σ): 22.084 ns ± 26.904 ns ┊ GC (mean ± σ): 6.76% ± 6.16%

  ▆█▆▅▄▃▂▁ ▁ ▁
  █████████▆▅▆▆▇▇▆██▇▇▇▆▄▅▄▃▃▃▃▄▅▅▆▇▇▆▆▇▆▆▇▆▇▆▇▇▆▆▆▆▅▅▅▃▄▄▂▃▃▂ █
  16.2 ns Histogram: log(frequency) by time 68.4 ns <

 Memory estimate: 80 bytes, allocs estimate: 1.

julia> @benchmark memory(zero, $7)
BenchmarkTools.Trial: 10000 samples with 999 evaluations per sample.
 Range (min … max): 16.316 ns … 1.192 μs ┊ GC (min … max): 0.00% … 95.13%
 Time (median): 17.518 ns ┊ GC (median): 0.00%
 Time (mean ± σ): 23.684 ns ± 41.628 ns ┊ GC (mean ± σ): 12.41% ± 7.46%

  ██▅▃▁ ▁▂▂▁▁ ▂
  ████████▅▅▄▆▆▅▅▅█▇▅▆█▇▅▄▇███████▇█▆▆▆▆▆▆▅▆▆▆▅▅▄▅▅▅▅▅▃▅▃▃▄▁▄ █
  16.3 ns Histogram: log(frequency) by time 77.5 ns <

 Memory estimate: 80 bytes, allocs estimate: 1.

```

The distributions are noticeably different even with 10000 samples each, those small differences in statistics are to be expected. If I just ran `@btime`, I’d see that the extra `Int` input makes it a bit faster, but the actual stats usually went the other way.

---

<div class="post-metadata">

**Author:** ![PatrickHaecker](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/patrickhaecker/32/222891_2.png) [@PatrickHaecker](https://discourse.julialang.org/u/PatrickHaecker)\
**Post date:** [May 16, 2025, 8:33am UTC](https://discourse.julialang.org/t/memory-comprehensions/129033/7 "2025-05-16T08:33:37Z")

</div>

Yes, the difference is small, indeed. That’s why I selected the user friendly approach here (only two parameters – although you could argue that it’s not user friendly when it fails for `n = 0`).

However, the effect was noticeable for different sizes. Additionally, the three argument version seems to be easier/better to optimize in general. The three argument version could use SIMD from the start (although I think that it doesn’t currently), while the two argument version needs some really clever compiler tricks to do so.

If we want to have it as a PR to Julia, I am open which version to prepare. We could even provide both methods.

---

<div class="post-metadata">

**Author:** ![nsajko](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nsajko/32/221187_2.png) [@nsajko](https://discourse.julialang.org/u/nsajko)\
**Post date:** [May 25, 2025, 10:51am UTC](https://discourse.julialang.org/t/memory-comprehensions/129033/8 "2025-05-25T10:51:04Z")

</div>

A new relevant package, not yet registered, is CollectAs.jl. Use it like:

```julia
using CollectAs: collect_as
collect_as(Memory, any_iterator)
collect_as(Memory{Float32}, any_iterator)

```

However `collect_as` is not just for `Memory`, it’s basically for _any_ collection type (as long as there’s an implementation). In that way it generalizes `collect`, which only allows specifying the _element_ type.

The package currently [resides](https://github.com/nsajko/CollectAs.jl) on my personal Github namespace, but I hope to be able to move it to [JuliaCollections](https://github.com/JuliaCollections) before registering it.

EDIT: it’s still not registered, still in the process, currently it’s named Collects.jl, and there’s a dedicated topic here on the forum: [New package: Collects.jl - meant to improve upon and generalize the interface of collect](https://discourse.julialang.org/t/new-package-collects-jl-meant-to-improve-upon-and-generalize-the-interface-of-collect/129468)