Dynamically create Struct

General Usage
,
1

Hello everyone :grin:!

I am working on a compartment model for a Markov chain simulation. The parameters of the compartment model (name of each compartment and transition matrix) are defined in an excel file. A Compartment_Model is composed of several Compartment’s defined with the following Struct’s :

mutable struct Compartment
    name::String
    id::Int16

    initial_particle_number::Float64
    mean_residence_time::Float64
    connected_compartments_id:: Vector{Int16}
    transition_probabilities:: Vector{Float16}
end

Base.@kwdef struct Compartment_Model
    c1::Compartment
    c2::Compartment
    c3::Compartment
    c4::Compartment
    c5::Compartment

    compartment_list::Vector{Compartment} = [c1, c2,c3, c4, c5]
    correspondance_dict::Dict{Int16, Compartment} = Dict([compartment.id => compartment for compartment in compartment_list])
    total_number_of_particle::Float64  = sum([compartment.initial_particle_number for compartment in compartment_list])
    initial_particle_distribution::Vector{Float64} = [compartment.initial_particle_number/total_number_of_particle for compartment in compartment_list]
end

However, this method is not flexible because I have to manually rewrite Compartment_Model in the code if I change the input model from the Excel file. My question is: Is it possible to dynamically create Compartments from the data in the Excel file and create a Compartment_Model with them?
It might be possible to do something like that with macros, but I can’t see exactly how.

Thanks ! :grin:
fdekerm

1 Like
2

Rewrite and change how? It’s not readily apparent how you would rewrite Compartment_Model and dynamically create Compartment, or how that differs from the structure in the provided example. Describe what can vary and what can stay the same, the latter could lend to optimizations.

3

In the interest of maintainability, code generation is usually the last resort. Consider using Named Tuples or even dictionaries as alternatives to a generated struct.

1 Like
4

NamedTuple behaves like a dynamically created Struct, so that’s what you’re after! You just need to make your Compartment_Model parametric instead of hard coding the name ::Compartment for the field types.

1 Like
5

NamedTuple is also immutable, though, and Compartment is mutable; a closer alternative would be MutableNamedTuples.jl or a Dict for structural mutation. Making many different NamedTuple types also requires compilation of methods for each one; we don’t know enough about the situation to tell what would be better. Right now it’s entirely possible that the only thing that changes are the values, in which case it’s a matter of reading a file and instantiation, or the only thing that changes is the number of Compartments and very often, in which case removing the c1 to c5 fields is enough.

1 Like
6

My read of the situation is that there is an Excel file with headers that serves as input to the model, but that those headers are static for than run. But you’re absolutely right that that was an unchecked assumption – my bad!

Re: mutability, I also just missed that. Thank you for the catch. A side question for me: do we know if there is a significant difference in performance between a mutable struct with concete types for its fields an vs a NamedTuple of Ref{T} values (where T is concrete)?

7

Yeah, the performance characteristics could differ for sure. One could sort of say that a mutable struct is like a pointer to an immutable struct, while an immutable tuple of Refs is like an immutable tuple of pointers. So there’s extra inefficiency in the representation with the latter option, if nothing else.

1 Like
8

Why not use a normal NamedTuple and update it with setproperties from ConstructionBase.jl or @set from Setfield.jl or Accessors.jl?

That will be much faster than the other two options, doesn’t allocate, works on GPUs, yada yada.

5 Likes
9

I have used XLSX.jl to read an Excel file, and then used Julia metaprogramming to build code after parsing the Excel cells using Meta.parse(). This was a straightforward use of metaprogramming.

I didn’t follow enough of the discussion to have an opinion about what sort of Julia code you should generate.

10

On my side, I’ve been generating structs using a macro from XML files. This allows me to create structs subtyping an AbstractObject with some specific values coming from the XML files. I have no clue if it is better than using a Dict or else, but it sure looks like what you are trying to do.

11

Thanks for the answers !
As some of you have suggested, I tried to use a NamedTuple.
Here a simplified code if someone else is interested (with the Compartment structure defined in the original post)` :

function load_compartment_model(compartment_models_data_path)
    transition_probability_matrix, compartment_names = load_markov_chain_data(compartment_models_data_path)
    compartment_names_symbol = Vector{Symbol}(undef, length(compartment_names))
    compartment_list = Vector{Compartment}(undef, length(compartment_names))

    for (id, name) in enumerate(compartment_names)
        compartment_list[id] = Compartment(name, transition_probability_matrix)
        compartment_names_symbol[id] = Symbol(name)
    end

    compartment_model = NamedTuple{Tuple(compartment_names_symbol)}(compartment_list)
    return compartment_model
end

It seems to work as I want!
Could you elaborate a little bit about setproperties ? I don’t understand how it works.

Thanks !
fdekerm

12

Constructing a Dict, and converting it to NamedTuple (if the dictionary is not already OK for you), would be nicer IMO. Something like this:

function load_compartment_model(transition_probability_matrix, compartment_names)
    d = Dict{Symbol,Compartment}()
    for name ∈ compartment_names
        d[Symbol(name)] = Compartment(name, transition_probability_matrix)
    end
    (; d...)
end
1 Like
13

ConstructionBase.setproperties is a function that lets you update most immutable objects in-place by passing in a “patch” NamedTuple - fields in the patch will be updated in the target object and the whole thing is rebuilt. This normally has no runtime cost.

@set from Accessors.jl or Setfiled.jl (and most functions from packages that do rebuilding of immutable things) use Constructionbase.setproperties under the hood. @set is just a convenience macro to make the syntax easier.

The use-case is if you need to modify your model after construction - don’t use a mutable object, it will be much slower, use an immutable object and rebuild it as needed.

It seems counter intuitive that rebuilding is faster than setting a field. But in julia no actual rebuild happens in practice because immutable objects usually don’t really exist as you think they do. Its just registers being moved around. Setting a mutable field has to actually write to some specific bit memory.

2 Likes