How to define a struct programmatically?

General Usage
1

suppose I have parametric struct Mystruct{N} where N depends on the contents of data.

Now, I need a “super-struct” that contains variable numbers of of Mystructs, e.g. depends on data I may need:

struct Superstruct{N1, N2}
    obj1::Mystruct{N1}
    obj2::Mystruct{N2}
end

while another time I may need:

struct Superstruct{N1, N2, N3}
    obj1::Mystruct{N1}
    obj2::Mystruct{N2}
    obj3::Mystruct{N3}
end

and so on…

the problem is: how do I define a struct “on the fly”, i.e. depends on data?

what I need is, after processing of the data, the followings are determined:

  1. the Ns
  2. the number of objects needed

then an appropriate struct would be created (for in-place operations afterwards).

thanks.

2

I am not sure of what you are asking. Nonetheless, here are two approaches that, between them, should let you do what you want to do (probably). If you are doing many of these there are alternate approaches which would be more performant – and more complicated.

struct Mystruct{T}
    value::T
end

function typedstructs(name, the_ns...)
    howmany = length(the_ns)
    str = string("struct ",name,"\n")
    theobjs = collect(string("    obj", i, "::", "Mystruct{",the_ns[i],"}") for i=1:howmany)
    objs = join(nobjs,"\n")
    str = string(str, objs, "\n", "end")
    return eval(Meta.parse(str))
 end

# typedstructs(firstone, Int, String) gives you
struct firstone
    obj1::Mystruct{Int}
    obj2::Mystruct{String}
end

function paramstructs(name, howmany)
    nstrs = collect(string("N",i) for i=1:howmany)
    ns = string("{", join(nstrs,","), "}")
    str = string("struct ",name,ns,"\n")
    nobjs = collect(string("    obj", i, "::", "Mystruct{",nstrs[i],"}") for i=1:howmany)
    objs = join(nobjs,"\n")
    str = string(str, objs, "\n", "end")
    return eval(Meta.parse(str))
end

# paramstructs(firstone, Int, String) gives you
struct firstone{N1,N2}
    obj1::Mystruct{N1}
    obj2::Mystruct{N2}
end
1 Like
3

so… is using strings a good idea? or, should we use metaprogramming (which I’m very poor at :cold_sweat:)?

4

A good rule of thumb is “prefer functions to macros”, Macros really exist for those situations where functions cannot provide the transformation required – also remember that macros are evaluated differently than functions – so when working with dynamic information, using functions is often more appropriate. Unless you are calling this function thousands of times in a time-sensitive portion of your code there is absolutely no downside to using a function. And clarity is worth a great deal!

1 Like
5

Using Strings in this way is not recommended, but the main problem with using Strings (that I know of) is that this is much slower than it could be. You should be using quote ... end and interpolation. You can use quote inside functions. I do not understand Jeffrey’s point here.

6

Sounds like a tuple (or an array)? Generating struct types on the fly seems like the wrong approach here.

(You want to be careful about putting information into the type, like the parameter N, that depend on runtime data and change frequently. This may trigger a lot of recompilation and dynamic dispatch, and be slow. In general, represent runtime data as values, not types.)

8 Likes
7

@tomtom I agree with @stevengj.

fyi the choice of strings rather than direct expression manipulation was guided by a desire to provide a very easy to follow solution, while trying better to understand the real intent – in general, I agree that expressions are preferred.

8

Array not work because of heterogenous type parameters, i.e., the array could not be concrete.

Tuple…? is a tuple of mixed types a fast way of computing in Julia?

9

It wasn’t clear from your description if the type parameters were known at compile time; if they are only known at runtime you might be stuck with dynamic dispatch anyway, in which case you might as well use a non-concrete Array. (In that case, perhaps better to not put the data into the type at all.)

Yes. A tuple is basically equivalent to an anonymous struct.

(And a named tuple is even closer to an unnamed struct, but you only need that if you want to refer to the entries by name rather than by index.)

3 Likes
10

Isn’t Dict is also an alternative if one needs a struct in runtime?

I recently faced having (well, wanting, noone forced me :slight_smile: ) to decode a bytestream of (C) structs where the struct definition was part of the stream. I ended up with something like the below:

struct DynamicStruct
    name::Symbol
    type::Symbol
    fields::Vector{Symbol}
    fieldtypes::Dict{Symbol, Symbol}
    ints::Dict{Symbol, Int}
    structs::Dict{Symbol, DynamicStruct}
    intarrays::Dict{Symbol, Vector{Int}}
    etc...
end

where fieldtypes map names to the right member dict which is handled in the implementation of getproperty.

I haven’t benchmarked if trying to maintain types is really worth the effort or if everything could just have been in a big Dict{Symbol, Any} instead.

Is it fair to say that a struct or a (named) tuple is generally best for performance if you have the struct definition at compile time and dict is if you don’t?

1 Like
11

It depends on what (and how many) operations you want to perform on the data — it’s really hard to choose a data structure without knowing that!

1 Like