# Trouble understanding macros: creating a struct on the fly

**URL:** <https://discourse.julialang.org/t/trouble-understanding-macros-creating-a-struct-on-the-fly/4576>\
**Category:** General Usage\
**Tags:** macros\
**Created:** [June 30, 2017, 9:34pm UTC](https://discourse.julialang.org/t/trouble-understanding-macros-creating-a-struct-on-the-fly/4576 "2017-06-30T21:34:41Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![JonathanAnderson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jonathananderson/32/890_2.png) [@JonathanAnderson](https://discourse.julialang.org/u/JonathanAnderson)\
**Post date:** [June 30, 2017, 9:34pm UTC](https://discourse.julialang.org/t/trouble-understanding-macros-creating-a-struct-on-the-fly/4576/1 "2017-06-30T21:34:41Z")

</div>

I’m trying to create a macro that will help me build structs from a schema defined in a JSON document and while I have something that works, I would love a code review so I can learn how to do it better…

```julia
macro make_struct(struct_name, schema)
    fields=[:($(Symbol(entry[1])) :: $(Symbol(titlecase(entry[2])))) for entry in eval(schema)]
    return Expr(:type, false, eval(struct_name), Expr(:block, fields...))
end

```

And then I can verify how everything works here.

```julia
println(@macroexpand @make_struct :STRUCT_NAME [["x","int32"],["y","float32"]])
@make_struct :STRUCT_NAME [["x","int32"],["y","float32"]]
dump(STRUCT_NAME(1,2))

```

But my usage of `eval` and building my expression with the `Expr(...)` syntax feels like I’m missing a key point. Can anyone help me out?

---

<div class="post-metadata">

**Author:** ![jkroso](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jkroso/32/220909_2.png) [@jkroso](https://discourse.julialang.org/u/jkroso)\
**Post date:** [June 30, 2017, 11:18pm UTC](https://discourse.julialang.org/t/trouble-understanding-macros-creating-a-struct-on-the-fly/4576/2 "2017-06-30T23:18:15Z")

</div>

```julia
macro make_struct(name, schema)
  fields=[:($(Symbol(entry[1]))::$(Symbol(titlecase(entry[2])))) for entry in eval(schema)]
  :(struct $(esc(name))
    $(map(esc,fields)...)
   end)
end
@make_struct STRUCT_NAME [["x","int32"]]

```

I’d say the thing you missed was adding the newline immediately after the struct name. It needs this for some reason.

Also you should probably escape the name of the struct and it’s fields otherwise the structs will be defined in the module the macro is defined in and it will look for `DataType`’s used in the fields there too.

---

<div class="post-metadata">

**Author:** ![cstjean](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cstjean/32/1444_2.png) [@cstjean](https://discourse.julialang.org/u/cstjean)\
**Post date:** [June 30, 2017, 11:23pm UTC](https://discourse.julialang.org/t/trouble-understanding-macros-creating-a-struct-on-the-fly/4576/3 "2017-06-30T23:23:22Z")

</div>

The vanilla way of defining your macro is like this:

```julia
macro make_struct(struct_name, schema...)
    fields=[:($(entry.args[1])::$(entry.args[2])) for entry in schema]
    esc(quote struct $struct_name
        $(fields...)
        end
    end)
end
println(@macroexpand @make_struct STRUCT_NAME (x,int32) (y, float32))
@make_struct STRUCT_NAME (x, int32) (y, float32)

```

> But my usage of eval and building my expression with the Expr(…) syntax feels like I’m missing a key point.

You should essentially never call `eval` inside a Macro. A macro is a source code transformation. It accepts code (`Expr/Symbol` objects), and outputs code, which will take the place of the macro invocation.

The Julia section on [metaprogramming](https://docs.julialang.org/en/stable/manual/metaprogramming/) is pretty good. There are also beginner-level tutorials on Lisp macros on the web.

Once you’re more comfortable with macros, have a look at [MacroTools](https://github.com/MikeInnes/MacroTools.jl). It makes writing them a lot easier.

---

<div class="post-metadata">

**Author:** ![JonathanAnderson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jonathananderson/32/890_2.png) [@JonathanAnderson](https://discourse.julialang.org/u/JonathanAnderson)\
**Post date:** [July 1, 2017, 2:42pm UTC](https://discourse.julialang.org/t/trouble-understanding-macros-creating-a-struct-on-the-fly/4576/4 "2017-07-01T14:42:41Z")

</div>

Thanks. I think the part I’m missing is how to go from my array of strings into the evaluated tuple of symbols.

My use case is I have a schema file `S1.json` on disk with the following structure `[["x","int32"],["y","float32"]]` and I’m trying to evaluate this into a struct so I can slurp in an array of these structs with `open(f->read(f, S1, Int64(stat(filepath).size/sizeof(S1))), filepath, "r")`

If I type out the fields in the structure, then this macro works just fine

```julia
function parse_schema(schema)
    function schema_helper(element)
        :($(Symbol(element[1]))::$(Symbol(titlecase(element[2]))))
    end
    tuple(map(schema_helper, schema)...)
end

z1=Any[Any["x","int32"],Any["y","float32"]] # from a JSON.parse(...)
println(parse_schema(z1))

macro make_struct(struct_name, schema...)
    esc(
        quote
            struct $(struct_name)
            $(schema...)
            end
        end
    )
end
println(@macroexpand @make_struct S1 parse_schema(z1))

```

I just get the following

```julia
begin
    struct S1
        parse_schema(z1)
    end
end

```

I understand that this is just parsing my expression and making a new one. Every time I think about how I would put a for loop on the inside of the struct, I come to the solution i should be writing a function that builds an `Expr` and then just `eval`ing the new expression in my repl. Should I be avoiding a macro all together?

---

<div class="post-metadata">

**Author:** ![cstjean](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cstjean/32/1444_2.png) [@cstjean](https://discourse.julialang.org/u/cstjean)\
**Post date:** [July 1, 2017, 3:09pm UTC](https://discourse.julialang.org/t/trouble-understanding-macros-creating-a-struct-on-the-fly/4576/5 "2017-07-01T15:09:44Z")

</div>

> [@JonathanAnderson](#):
>
> Every time I think about how I would put a for loop on the inside of the struct, I come to the solution i should be writing a function that builds an Expr and then just evaling the new expression in my repl. Should I be avoiding a macro all together?

Yeah, I think so. You can `eval` your structs into existence while parsing the file. You might run into world-age issues, though, but they are solvable with more `eval`.

Also consider that you can essentially define new data types at runtime with tuples and parametric types, without `eval`:

```julia
> struct JSONStruct{NAME, T<:Tuple}
    fields::T
    JSONStruct{NAME}(fields...) where NAME = new{NAME, typeof(fields)}(fields)
end

> JSONStruct{:apple}(1,2,"hey")
JSONStruct{:apple,Tuple{Int64,Int64,String}}((1, 2, "hey"))

> Apple = JSONStruct{:apple,Tuple{Int64,Int64,String}}

```

but you need to be very comfortable with parametric types, and it can get very messy. See eg. [TypedTables.jl](https://github.com/FugroRoames/TypedTables.jl)

---

<div class="post-metadata">

**Author:** ![JonathanAnderson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jonathananderson/32/890_2.png) [@JonathanAnderson](https://discourse.julialang.org/u/JonathanAnderson)\
**Post date:** [July 2, 2017, 3:48pm UTC](https://discourse.julialang.org/t/trouble-understanding-macros-creating-a-struct-on-the-fly/4576/6 "2017-07-02T15:48:46Z")

</div>

So I ended up writing the following functions to make my datatype and to slurp in my array of datatypes from disk.

```julia
function make_schema_struct_expr(dataset::String)
    function schema_helper(element)
        :($(Symbol(element[1]))::$(Symbol(titlecase(element[2]))))
    end
    schema=open(JSON.parse, "/path/to/data/$dataset/schema.json", "r")
    Expr(:type, false
        , Symbol(dataset)
        , Expr(:block, map(schema_helper, schema)...))
end
    
function load_dataset{T}(dataset::Type{T}, datafile_name)
    filepath = "/path/to/data/$dataset/$datafile_name"
    num_records::Integer = stat(filepath).size / sizeof(dataset)
    open(f->read(f, dataset, num_records), filepath, "r")
end

```

and then I just make sure I eval the struct and then load in my data.

```julia
eval(make_schema_struct_expr("MyDataset"))
dat = load_dataset(MyDataset, "file_0.dat")

```

I bet I can do this even more cleanly but this feels like a win. Thank you @cstjean for your help! =D

---

<div class="post-metadata">

**Author:** ![Ralph\_Smith](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ralph_smith/32/10344_2.png) [@Ralph\_Smith](https://discourse.julialang.org/u/Ralph_Smith)\
**Post date:** [July 3, 2017, 4:00am UTC](https://discourse.julialang.org/t/trouble-understanding-macros-creating-a-struct-on-the-fly/4576/7 "2017-07-03T04:00:36Z")

</div>

Well, since the thread is titled “Trouble understanding macros…”, let’s use @cstjean’s first pattern:

```julia
module M
using JSON
export make_schema_struct

macro make_struct(struct_name, schema...)
    fields = [:($(x[1])::$(x[2])) for x in schema...]
    esc(quote struct $struct_name
        $(fields...)
        end
    end)
end

function make_schema_struct(datadir,dataset::String)
    schema = open(JSON.parse, joinpath(datadir,"$dataset/schema.json"), "r")
    schema = [(Symbol(x),Symbol(titlecase(y))) for (x,y) in schema]
    eval(current_module(),:(@M.make_struct $(Symbol(dataset)) $schema))
end
end

using M
make_schema_struct("/path/to/data","MyDataset")

```

Pretty much the same as yours, but a bit less expression-wrangling.

---

<div class="post-metadata">

**Author:** ![JonathanAnderson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jonathananderson/32/890_2.png) [@JonathanAnderson](https://discourse.julialang.org/u/JonathanAnderson)\
**Post date:** [July 3, 2017, 3:21pm UTC](https://discourse.julialang.org/t/trouble-understanding-macros-creating-a-struct-on-the-fly/4576/8 "2017-07-03T15:21:39Z")

</div>

Thanks @Ralph_Smith . This is great!