Conversion of properties of a NamedTuple or struct

General Usage Performance
1

I want write a piece of code that will receive large arrays of “state variables” from finite element codes, and do a conversion on all the e.g. Float64 appearing in this data structure. There are many element types, I do not want to specialize my conversion code to one specific type of “state variable”.

The following is an example of state data from one element, and its conversion

state = [(a=[1.,2.],b=3.),(a=[4.,5.],b=6.)]
cvt(a)                    = a
cvt(a::Float64)           = Float32(a) # Float32 is just a MWE, not my true purpose
cvt(a::Array{T}) where{T} = cvt.(a)
cvt(a::NamedTuple)        = (; zip(keys(a),cvt.(values(a)))...)
state32   = cvt(state)

This works as intended, but the performance is not good, because constructing a NamedTuple like that is not typestable. Converting the same amount of data stored in a Vector, is much faster (x60).

stateflat = randn(6)
using BenchmarkTools
let
    @btime state32 = cvt($state) 
    @btime a   = cvt($stateflat)
end
@code_warntype cvt(state)
@code_warntype cvt(stateflat)

Let us say I extracted typeof(state) and from that computed the type of state32 once and for all (@generated function, maybe), is there any way I could exploit that to construct a NamedTuple given type and properties (in a typestable manner, probably using a type-barrier function?)

It does not have to be NamedTuples. To have state data as struct is an option too. My half attempt at this does not compile

abstract type Data end
struct Matdata{R} <:Data
    a::Vector{R}
    b::R
end
state = [Matdata([1.,2.],3.),Matdata([4.,5.],6.)]

cvt(a)                    = a
cvt(a::Float64)           = Float32(a)
cvt(a::Array{T}) where{T} = cvt.(a)
function cvt(a::T{R}) where {T<:Data,R<:Real} # error here
    val = getfield(a) # error here how do I obtain a Tuple containing the data in a
    return T{Float32}(cvt.(val))
end
state32   = cvt(state)

If all else fails, I will not write a unique code to do the conversion, but see how I cant sweeten the pill for the conversion code that will have to be called in each element.

1 Like
2

There may be room to optimize further, but I get a pretty big pickup specializing just on the names of the NamedTuple

# original implementation
julia> @btime cvt($state)
  2.078 μs (38 allocations: 2.31 KiB)

cvt(a::NamedTuple{N, T}) where {N, T} = NamedTuple{N}(cvt.(values(a)))

julia> @btime cvt($state)
  126.710 ns (5 allocations: 352 bytes)
2 Likes
3

Thanks Chris,
Using a constructor - I did not even think of looking up the constructor !!! :smile:

Maybe as you say one could optimise further. But you popped the bottleneck, and I will probably optimise elsewhere in the code first!

Philippe

1 Like