ArraysOfArrays with named fields

Let’s say I have a vector a = [x1, x2, x3, q1, q2, q3, q4]. What’s a nice way to use it in algorithms that expect a flat vector like that, but at the same time have a nice interface like a.x, a.q to extract [x1, x2, x3] and [q1, q2, q3, q4]?

I thought of

a = NamedArray(VectorOfVectors([[1,2,3], [1,2,3,4]]), [:x, :q])

but that requires writing a[:x] instead of a.x, and flatview(a.array)[i] instead of a[i].

There are packages that convert back and forth between struct and vector.

ParameterHandling.jl, TransformVariables.jl, Bijectors.jl, Flatten.jl, etc


Thanks. It seems there’s room for another small package to provide named views to sections of an existing vector…

This is sort of what StructArrays.jl does?

StructArrays is for a quite different use case I think:

  • each column with the same number of elements
  • columns can be of different types
  • data collected in StructArray columns
  • convenient constructors for vector of named tuples and list of columns

I’m looking for an interface to different sections of a vector:

  • sections can have different lengths
  • all values of the same type
  • all data stored together as a single vector
  • convenient constructor for named sections

For example, it could be a lightweight wrapper that works like this:

a = [x1, x2, x3, q1, q2, q3, q4]

a2 = Sections(a, x=1:3, q=4:7)

a2[1]  # a[1] (getindex simply forwards to `a`)
a2.x   # view(a, 1:3)
a2.q   # view(a, 4:7)
1 Like

This is what ComponentArrays was designed for.

You can do

using ComponentArrays
a = ComponentArray(x = [1, 2, 3], q = [1, 2, 3, 4])

# Access position component

# Access quaternion component

See this example for how you can use ComponentArrays for simulating hierarchical models.


Are you looking for GitHub - JuliaHEP/AwkwardArray.jl: Awkward Array in Julia mirrors the Python library, enabling effortless zero-copy data exchange between Julia and Python ? it’s nowhere near completion yet but maybe you come from Python’s GitHub - scikit-hep/awkward: Manipulate JSON-like data with NumPy-like idioms.

In [8]: array = ak.Array([
   ...:            [{"x": 1.1, "y": [1]}, {"x": 2.2, "y": [2, 2]}],
   ...:            [],
   ...:            [{"x": 3.3, "y": [3, 3, 3]}]
   ...:         ])

In [9]: array.x
Out[9]: <Array [[1.1, 2.2], [], [3.3]] type='3 * var * float64'>

In [10]: array.y
Out[10]: <Array [[[1], [2, 2]], [], [[3, 3, 3]]] type='3 * var * var * int64'>
1 Like

Thanks, ComponentArrays is exactly what I had in mind. You can even do the following:

a = ComponentArray([1,2,3,1,2,3,4], Axis(x=1:3, q=4:7))

which is very close to the API I had imagined :slight_smile:

1 Like