How to change/reduce vector of Structure to be a structure of aggregated fields?

General Usage
1

I have a file that I can read from it a specific structure Rs and its instance R (a simple MWE of it is below)

using LinearAlgebra, Parameters, Base
Base.@kwdef mutable struct Rs
    I::Vector{Float64} = [2,2,2]
    Dot::Array{Float64, 2} = [1 1 1; 2 2 2; 3 3 3]
end
R = Rs[];
push!(R, Rs());
push!(R, Rs()); 
julia> R
2-element Vector{Rs}:
 Rs([2.0, 2.0, 2.0], [1.0 1.0 1.0; 2.0 2.0 2.0; 3.0 3.0 3.0])
 Rs([2.0, 2.0, 2.0], [1.0 1.0 1.0; 2.0 2.0 2.0; 3.0 3.0 3.0])

I want to create RR to be of Rs, in which each field of RR is the aggregated vector/array of the field rows of R, i.e.,

RR = Rs(reduce(vcat, getproperty(r, :I) for r in R), reduce(vcat, getproperty(r, :Dot) for r in R));
julia> RR
Rs([2.0, 2.0, 2.0, 2.0, 2.0, 2.0], [1.0 1.0 1.0; 2.0 2.0 2.0; 3.0 3.0 3.0; 1.0 1.0 1.0; 2.0 2.0 2.0; 3.0 3.0 3.0])

The previous way is working. However, the fields of Rs are a lot. So, is there an alternative easier way to build RR rather than long command i.e., RR = Rs(reduce(..),reduce(..), reduce(..)..)?

Another issue is; if I want to do some operations (such as below), working on RR or R would be faster?

for i in eachindex(R)
    for j in 1:3
         R[i].I[j] = R[1].Dot [j,1];
    end
end

for i in 1:length(R.I)/3
    for j in 1:3
         R.I[3*i+j-3] = R.Dot [3*i+j-3,1];
    end
end
2

You would probably get better help faster if you choose a more descriptive title of your post.

3

Thank you for your advice. I followed.

4

I am not entirely sure I understood the question but are you aware of propertynames and the splat operator ...? You can probably assemble a Tuple or NamedTuple which you can splat in the Rs constructor.

6

No, I did believe Tuple or NamedTuple would be necessary intermediaries but I was wrong. This works:

RR = Rs((reduce(vcat, getfield.(R, pn)) for pn in fieldnames(Rs))...)

Of course, this requires that the reduction that you wanted to apply must be vcat for all fields.

About your loops, I suggest you benchmark both alternatives. Often details as how large are the elements, or if the compiler was able to employ vectorized assignment, impact what is the best alternative. Unless you know this to be a bottleneck I would go with the option that is most readable and less prone to break with future changes.

8

StaticArrays are recommended for small arrays (i.e., at most 100 elements), so I think that if you stick to your many structs with small arrays it may work, but if your plan is to combine them maybe StaticArrays are not adequate. Especially because then you cannot fix their length in the struct definition and therefore the fields will be abstract (more specifically UnionAll) throwing away any performance improvement. Only use StaticArrays if your arrays are small and always the same size as, for example, sets of three-dimensional coordinates or something like that.

About your code, please post an equivalent MWE, in which RRs has default values for fields and there is a vector if two of them already created, also node that you did not change fieldnames(Rs) in RR = ... to fieldnames(RRs).

1 Like
9

You may find StructArrays useful: they allow an efficient access to both individual structs (RR[1]), and to a field of all structs in the array (RR.I):

julia> using StructArrays

julia> RR = StructArray(R)
2-element StructArray(::Vector{Vector{Float64}}, ::Vector{Matrix{Float64}}) with eltype Rs:
 Rs([2.0, 2.0, 2.0], [1.0 1.0 1.0; 2.0 2.0 2.0; 3.0 3.0 3.0])
 Rs([2.0, 2.0, 2.0], [1.0 1.0 1.0; 2.0 2.0 2.0; 3.0 3.0 3.0])

julia> RR[1]
Rs([2.0, 2.0, 2.0], [1.0 1.0 1.0; 2.0 2.0 2.0; 3.0 3.0 3.0])

julia> RR.I
2-element Vector{Vector{Float64}}:
 [2.0, 2.0, 2.0]
 [2.0, 2.0, 2.0]
3 Likes
12

Why do you import Parameters if you use the non-exported Base.@kwdef that should be avoided?

You did not understood me, I offered to help after you sorted out your struct definition. Check the StaticArrays QuickStart for the notation to be used. Create a MWE with the struct already converted and state your difficulty, but again, I am not sure if StaticArrays are what you want, in the case you want to keep aggregating many small structs in a single large struct.

14

How is the format of the data file? It is possible that you need to parse it by hand (i.e., write your custom parsing function).