Create struct from YAML

General Usage
1

I have the following code:

using YAML, StructTypes, Parameters

 @with_kw struct Test
    can_log::String
    sys_log::String
    version::String
    info::String
end

data = YAML.load_file("data/config.yaml")
test = data["tests"][1]
println(test)

and data (config.yaml):

tests:
    - can_log:   data/27-08-2022/logfile.csv
      sys_log:   data/27-08-2022/syslog
      version:   1.7.1
      info:      first test
    - can_log:   data/28-08-2022/logfile.csv
      sys_log:   data/28-08-2022/syslog
      version:   1.7.2
      info:      second test

If I run the script the output is:

julia> test
Dict{Any, Any} with 4 entries:
  "sys_log" => "data/27-08-2022/syslog.csv"
  "can_log" => "data/27-08-2022/logfile.csv"
  "version" => "1.7.1"
  "info"    => "first test"

How can I convert this dict into a struct of the type Test ?

2

The StructTypes.jl package has a StructTypes.constructfrom function that will do this for you (assuming you have a StructType defined for your type).

3

It is not clear for me - after reading the documentation - how to do that.

Not working:

using YAML, StructTypes

mutable struct Test
    can_log::String
    sys_log::String
    version::String
    info::String
    Test() = new()
end

data = YAML.load_file("data/config.yaml")
test = data["tests"][1]
println(test)

StructTypes.StructType(::Type{Test}) = StructTypes.Mutable()
StructTypes.constructfrom(Test, test)

Error message:

julia> include("src/read.jl")
Dict{Any, Any}("sys_log" => "data/27-08-2022/syslog", "can_log" => "data/27-08-2022/logfile.csv", "version" => "1.7.1", "info" => "first test")
ERROR: LoadError: MethodError: no method matching applyfield!(::StructTypes.Closure{String}, ::Test, ::String)
Closest candidates are:
  applyfield!(::Any, ::T, ::Symbol) where T at ~/.julia/packages/StructTypes/Cw0JM/src/StructTypes.jl:780
Stacktrace:
 [1] constructfrom!(#unused#::StructTypes.Mutable, x::Test, #unused#::StructTypes.DictType, obj::Dict{Any, Any})
   @ StructTypes ~/.julia/packages/StructTypes/Cw0JM/src/StructTypes.jl:1025
 [2] constructfrom!
   @ ~/.julia/packages/StructTypes/Cw0JM/src/StructTypes.jl:1016 [inlined]
 [3] constructfrom
   @ ~/.julia/packages/StructTypes/Cw0JM/src/StructTypes.jl:1006 [inlined]
 [4] constructfrom(#unused#::Type{Test}, obj::Dict{Any, Any})
   @ StructTypes ~/.julia/packages/StructTypes/Cw0JM/src/StructTypes.jl:919
 [5] top-level scope
   @ ~/repos/config/src/read.jl:16
 [6] include(fname::String)
   @ Base.MainInclude ./client.jl:476
 [7] top-level scope
   @ REPL[1]:1
in expression starting at /home/ufechner/repos/config/src/read.jl:16
4
  1. StructTypes.Mutable() requires an empty constructor in the type, Test() = new(). StructTypes.Struct() doesn’t require it, and is also an option here, if the YAML files can be relied upon to always provide these four fields.

Though slightly less performant than StructTypes.Struct, Mutable is a much more robust method for mapping Julia struct fields for serialization. …
This flow has the nice properties of: allowing object construction success even if fields are missing in the input, and if “extra” fields exist in the input that aren’t apart of the Julia struct’s fields, they will automatically be ignored. This allows for maximum robustness when mapping Julia types to arbitrary data foramts that may be generated via web services, databases, other language libraries, etc.

(in short, StructTypes.Struct is slightly more performant, but StructTypes.Mutable is more robust to data issues in the input YAML file.)

  1. It seems the keys have to be Symbols for the construction to work. Either convert them after reading:
test = Dict(Symbol(k) => v for (k, v) in data["tests"][1])

or read them in as Symbols in the first place:

julia> data = YAML.load_file("filename.yml";  dicttype=Dict{Symbol,Any})
1 Like
5

Thanks a lot!

This works:

using YAML, StructTypes

 mutable struct Test
    can_log::String
    sys_log::String
    version::String
    info::String
    Test() = new()
end

data = YAML.load_file("data/config.yaml"; dicttype=Dict{Symbol,Any})
test_dict = data[:tests][1]
println(test_dict)

StructTypes.StructType(::Type{Test}) = StructTypes.Mutable()
test = StructTypes.constructfrom(Test, test_dict)
6

Or the complete code that converts an array of dicts from YAML to an array of structs in Julia:

using YAML, StructTypes

 mutable struct Test
    can_log::String
    sys_log::String
    version::String
    info::String
    Test() = new()
end
StructTypes.StructType(::Type{Test}) = StructTypes.Mutable()

const tests = Test[]

data = YAML.load_file("data/config.yaml"; dicttype=Dict{Symbol,Any})

for test_dict in data[:tests]
    test = StructTypes.constructfrom(Test, test_dict)
    push!(tests, test)
end

tests
1 Like