ArraysOfArrays with named fields

General Usage
,
1

Let’s say I have a vector a = [x1, x2, x3, q1, q2, q3, q4]. What’s a nice way to use it in algorithms that expect a flat vector like that, but at the same time have a nice interface like a.x, a.q to extract [x1, x2, x3] and [q1, q2, q3, q4]?

I thought of

a = NamedArray(VectorOfVectors([[1,2,3], [1,2,3,4]]), [:x, :q])

but that requires writing a[:x] instead of a.x, and flatview(a.array)[i] instead of a[i].

2

There are packages that convert back and forth between struct and vector.

ParameterHandling.jl, TransformVariables.jl, Bijectors.jl, Flatten.jl, etc

3

Thanks. It seems there’s room for another small package to provide named views to sections of an existing vector…

4

This is sort of what StructArrays.jl does?

5

StructArrays is for a quite different use case I think:

  • each column with the same number of elements
  • columns can be of different types
  • data collected in StructArray columns
  • convenient constructors for vector of named tuples and list of columns

I’m looking for an interface to different sections of a vector:

  • sections can have different lengths
  • all values of the same type
  • all data stored together as a single vector
  • convenient constructor for named sections

For example, it could be a lightweight wrapper that works like this:

a = [x1, x2, x3, q1, q2, q3, q4]

a2 = Sections(a, x=1:3, q=4:7)

a2[1]  # a[1] (getindex simply forwards to `a`)
a2.x   # view(a, 1:3)
a2.q   # view(a, 4:7)
6

This is what ComponentArrays was designed for.

You can do

using ComponentArrays
a = ComponentArray(x = [1, 2, 3], q = [1, 2, 3, 4])

# Access position component
a.x

# Access quaternion component
a.q

See this example for how you can use ComponentArrays for simulating hierarchical models.

7

Are you looking for GitHub - JuliaHEP/AwkwardArray.jl: Awkward Array in Julia mirrors the Python library, enabling effortless zero-copy data exchange between Julia and Python ? it’s nowhere near completion yet but maybe you come from Python’s GitHub - scikit-hep/awkward: Manipulate JSON-like data with NumPy-like idioms.

In [8]: array = ak.Array([
   ...:            [{"x": 1.1, "y": [1]}, {"x": 2.2, "y": [2, 2]}],
   ...:            [],
   ...:            [{"x": 3.3, "y": [3, 3, 3]}]
   ...:         ])

In [9]: array.x
Out[9]: <Array [[1.1, 2.2], [], [3.3]] type='3 * var * float64'>

In [10]: array.y
Out[10]: <Array [[[1], [2, 2]], [], [[3, 3, 3]]] type='3 * var * var * int64'>
8

Thanks, ComponentArrays is exactly what I had in mind. You can even do the following:

a = ComponentArray([1,2,3,1,2,3,4], Axis(x=1:3, q=4:7))

which is very close to the API I had imagined :slight_smile: