# Type-stable Array of NamedTuples

**URL:** <https://discourse.julialang.org/t/type-stable-array-of-namedtuples/32743>\
**Category:** Performance\
**Tags:** question, array, namedtuple\
**Created:** [December 27, 2019, 5:23pm UTC](https://discourse.julialang.org/t/type-stable-array-of-namedtuples/32743 "2019-12-27T17:23:39Z")\
**Posts on this page:** 13\
**Page:** 1

<div class="post-metadata">

**Author:** ![robsmith11](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/robsmith11/32/29641_2.png) [@robsmith11](https://discourse.julialang.org/u/robsmith11)\
**Post date:** [December 27, 2019, 5:23pm UTC](https://discourse.julialang.org/t/type-stable-array-of-namedtuples/32743/1 "2019-12-27T17:23:39Z")

</div>

I often want to do the following for a large `n` where I perform some calculations and return a tuple of values with the result of each iteration:

```julia
function f(n::Int)
  v = Vector()
  for i in 1:n 
    ...
    push!(v, (a=i, b=1.23, c=false, d=:foo, ...))
  end
  v
end

```

As written, the code is less efficient than it could be because `v` will have type `Array{Any,1}`. The type instability can be fixed by explicitly specifying the type of the `NamedTuple`, but that is cumbersome as the tuple’s size grows.

Is there any way to tell Julia that I only want to push a single type to the `Array` so that it can either

- automatically infer the type from the `push!`, or
- wait to initiate the `Array` until the first `push!` call so that the type will be known at that time?

---

<div class="post-metadata">

**Author:** ![bashonubuntu](https://avatars.discourse-cdn.com/v4/letter/b/f19dbf/32.png) [@bashonubuntu](https://discourse.julialang.org/u/bashonubuntu)\
**Post date:** [December 27, 2019, 5:26pm UTC](https://discourse.julialang.org/t/type-stable-array-of-namedtuples/32743/2 "2019-12-27T17:26:46Z")

</div>

Check out

[https://github.com/JuliaComputing/IndexedTables.jl](https://github.com/JuliaComputing/IndexedTables.jl)

IndexedTables provide the backend for JuliaDB which you can also check out

[https://github.com/JuliaComputing/JuliaDB.jl](https://github.com/JuliaComputing/JuliaDB.jl)

There are some type-instabilities you can’t fully resolve. That’s fine, just put that piece of code inside a function-barrier. See below

[https://docs.julialang.org/en/v1/manual/performance-tips/#kernel-functions-1](https://docs.julialang.org/en/v1/manual/performance-tips/#kernel-functions-1)

---

<div class="post-metadata">

**Author:** ![ericphanson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ericphanson/32/215186_2.png) [@ericphanson](https://discourse.julialang.org/u/ericphanson)\
**Post date:** [December 27, 2019, 5:36pm UTC](https://discourse.julialang.org/t/type-stable-array-of-namedtuples/32743/3 "2019-12-27T17:36:29Z")

</div>

Maybe `push!!` from [BangBang.jl](https://github.com/tkf/BangBang.jl/) could help? Not sure though.

---

<div class="post-metadata">

**Author:** ![sairus7](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/sairus7/32/10816_2.png) [@sairus7](https://discourse.julialang.org/u/sairus7)\
**Post date:** [December 27, 2019, 5:39pm UTC](https://discourse.julialang.org/t/type-stable-array-of-namedtuples/32743/4 "2019-12-27T17:39:45Z")

</div>

You can use `typeof` to get a type of a variable and construct vector of that type:

```julia
function foo(n::Int)
  v = nothing
  for i in 1:n 
    if v === nothing
        v = Vector{typeof((a=i, b=1.23, c=false, d=:foo))}(undef, n)
    end
    # push!(v, (a=i, b=1.23, c=false, d=:foo))
    v[i] = (a=i, b=1.23, c=false, d=:foo)
  end
  v
end

```

But better if you know variable type at initialization.

You can use [StructArrays.jl](https://github.com/JuliaArrays/StructArrays.jl) if you need to work on vectors of individual fields.

There is also `map` syntax:

```julia
map(v -> f(v), 1:n)

```

and list comprehension:  
`[f(v) for v in 1:n]`

---

<div class="post-metadata">

**Author:** ![mohamed82008](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mohamed82008/32/18171_2.png) [@mohamed82008](https://discourse.julialang.org/u/mohamed82008)\
**Post date:** [December 27, 2019, 5:41pm UTC](https://discourse.julialang.org/t/type-stable-array-of-namedtuples/32743/5 "2019-12-27T17:41:29Z")

</div>

See this discussion [Ridiculous idea: types from the future](https://discourse.julialang.org/t/ridiculous-idea-types-from-the-future/5457).

---

<div class="post-metadata">

**Author:** ![bashonubuntu](https://avatars.discourse-cdn.com/v4/letter/b/f19dbf/32.png) [@bashonubuntu](https://discourse.julialang.org/u/bashonubuntu)\
**Post date:** [December 27, 2019, 5:57pm UTC](https://discourse.julialang.org/t/type-stable-array-of-namedtuples/32743/6 "2019-12-27T17:57:55Z")

</div>

Does this scale if the tuple size grows?

---

<div class="post-metadata">

**Author:** ![sairus7](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/sairus7/32/10816_2.png) [@sairus7](https://discourse.julialang.org/u/sairus7)\
**Post date:** [December 27, 2019, 6:04pm UTC](https://discourse.julialang.org/t/type-stable-array-of-namedtuples/32743/7 "2019-12-27T18:04:16Z")

</div>

Do you mean that:  
a) Tuple can be of different size between iterations inside one function call, or  
b) Tuple size is big for some particular functions, but remain constant within one call?

---

<div class="post-metadata">

**Author:** ![bashonubuntu](https://avatars.discourse-cdn.com/v4/letter/b/f19dbf/32.png) [@bashonubuntu](https://discourse.julialang.org/u/bashonubuntu)\
**Post date:** [December 27, 2019, 6:07pm UTC](https://discourse.julialang.org/t/type-stable-array-of-namedtuples/32743/8 "2019-12-27T18:07:42Z")

</div>

> [@sairus7](#):
>
> v = Vector{typeof((a=i, b=1.23, c=false, d=:foo))}(undef, n)

I mean b), if I understood correctly. Do you think that if I had a large number of elements in the tuple, say 100, initializing a vector with `typeof()` will be cumbersome? Maybe there is an automatic way to do this too.

---

<div class="post-metadata">

**Author:** ![sairus7](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/sairus7/32/10816_2.png) [@sairus7](https://discourse.julialang.org/u/sairus7)\
**Post date:** [December 27, 2019, 6:22pm UTC](https://discourse.julialang.org/t/type-stable-array-of-namedtuples/32743/9 "2019-12-27T18:22:47Z")

</div>

Consider splitting one big tuple in a group of smaller tuples or structures, like

```julia
t1 = (a, b, c, d, e)
s1 = MyStruct(f,g,h)
t = (t1, s1)

```

Or split your data and processing into different vectors and functions.

Also tuples are not very efficient for very large number of elements.

If you need to fill one big table, there are such packages as DataFrames or JuliaDB.

---

<div class="post-metadata">

**Author:** ![bashonubuntu](https://avatars.discourse-cdn.com/v4/letter/b/f19dbf/32.png) [@bashonubuntu](https://discourse.julialang.org/u/bashonubuntu)\
**Post date:** [December 27, 2019, 6:23pm UTC](https://discourse.julialang.org/t/type-stable-array-of-namedtuples/32743/10 "2019-12-27T18:23:49Z")

</div>

Perfect, that’s exactly why I linked to IndexedTables.jl above 🙂

---

<div class="post-metadata">

**Author:** ![rdeits](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rdeits/32/286_2.png) [@rdeits](https://discourse.julialang.org/u/rdeits)\
**Post date:** [December 27, 2019, 7:44pm UTC](https://discourse.julialang.org/t/type-stable-array-of-namedtuples/32743/11 "2019-12-27T19:44:25Z")

</div>

As discussed over in [Ridiculous idea: types from the future](https://discourse.julialang.org/t/ridiculous-idea-types-from-the-future/5457) (as @mohamed82008 mentioned), there is a pretty nice way to do this with `map`:

```julia
function f(n)
  map(1:n) do i
    (a=i, b=1.23, c=false, d=:foo)
  end
end

```

This is a pretty nice solution, since it’s shorter than the original code while being type-stable:

```julia
julia> f(5)
5-element Array{NamedTuple{(:a, :b, :c, :d),Tuple{Int64,Float64,Bool,Symbol}},1}:
 (a = 1, b = 1.23, c = 0, d = :foo)
 (a = 2, b = 1.23, c = 0, d = :foo)
 (a = 3, b = 1.23, c = 0, d = :foo)
 (a = 4, b = 1.23, c = 0, d = :foo)
 (a = 5, b = 1.23, c = 0, d = :foo)

julia> @code_warntype f(5)
Variables
  #self#::Core.Compiler.Const(f, false)
  n::Int64
  #3::getfield(Main, Symbol("##3#4"))

Body::Array{NamedTuple{(:a, :b, :c, :d),Tuple{Int64,Float64,Bool,Symbol}},1}

```

---

<div class="post-metadata">

**Author:** ![robsmith11](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/robsmith11/32/29641_2.png) [@robsmith11](https://discourse.julialang.org/u/robsmith11)\
**Post date:** [December 27, 2019, 8:04pm UTC](https://discourse.julialang.org/t/type-stable-array-of-namedtuples/32743/12 "2019-12-27T20:04:44Z")

</div>

`map` is indeed a good solution. I hadn’t realized that `map` would allow updating of state as it iterates over a vector, but it does work fine.

BTW, I’d mark this topic as solved, but for some reason the button no longer shows up for me.

EDIT:  
However, `map` restricts one to returning a single result for each iteration. It’s not possible to push multiple or zero results per iteration (without using nested arrays).

---

<div class="post-metadata">

**Author:** ![Mason](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mason/32/2423_2.png) [@Mason](https://discourse.julialang.org/u/Mason)\
**Post date:** [December 27, 2019, 9:21pm UTC](https://discourse.julialang.org/t/type-stable-array-of-namedtuples/32743/13 "2019-12-27T21:21:57Z")

</div>

> [@robsmith11](#):
>
> However, `map` restricts one to returning a single result for each iteration. It’s not possible to push multiple or zero results per iteration (without using nested arrays).

You can return a tuple of multiple (or zero) return results and then call `Iterators.flatten` on the result.