Save code generated by macro to file

General Usage
1

I have a string macro that generates a lot of code, and I would like to store this code in a Julia file such that I can just include it the next time it is required. How can I achieve this? I assume it has to be realized using @macroexpand somehow, but I’m wondering how to capture the output such that I get exactly the same behavior when including it as if I’d just left the macro in there.

To be clear, I’d like to turn

module MyModule
  c"""
    // here be dragons
  """
end

into

module MyModule
  include("herebedragons.jl")
end

What would be the Julian way to achieve this?

2

the julian way is not to save it :wink:

just generate the MyModule inside your package scope. code generation will happen once per precompilation

3

Unfortunately, that’s not a option, otherwise that would’ve been my first choice. There are technical reasons due to which I cannot run this at precompilation time but need to pre-generate and store the macro output.

4

You could probably serialize the AST you get from @macroexpand and then to include it you deserialize and eval it.

julia> expr = @macroexpand @time 1+1;

julia> using Serialization

julia> serialize("ast.jls", expr);

julia> ast = deserialize("ast.jls");

julia> eval(ast)
  0.000000 seconds
2
2 Likes
5

Using Serialization.jl is more robust, but you can also use a combination of write("foo.jl", string(expr)) and include("foo.jl"). However, this method can only be used if Meta.parse(string(expr)) is equivalent to expr.

expr = @macroexpand g(x) = @time (sleep(1); sin(x))
write("foo.jl", string(Base.remove_linenums!(expr)))
println(read("def.jl", String)); println()
include("foo.jl")
@show g(π/6);

Output:

g(x) = begin
        begin
            while false
            end
            local var"#30#stats" = Base.gc_num()
            local var"#33#compile_elapsedtime" = Base.cumulative_compile_time_ns_before()
            local var"#32#elapsedtime" = Base.time_ns()
            local var"#31#val" = begin
                        sleep(1)
                        sin(x)
                    end
            var"#32#elapsedtime" = Base.time_ns() - var"#32#elapsedtime"
            var"#33#compile_elapsedtime" = Base.cumulative_compile_time_ns_after() - var"#33#compile_elapsedtime"
            local var"#34#diff" = Base.GC_Diff(Base.gc_num(), var"#30#stats")
            Base.time_print(var"#32#elapsedtime", (var"#34#diff").allocd, (var"#34#diff").total_time, Base.gc_alloc_count(var"#34#diff"), var"#33#compile_elapsedtime", true)
            var"#31#val"
        end
    end

  1.010495 seconds (145 allocations: 13.766 KiB)
g(π / 6) = 0.49999999999999994

Edit: remove the redundant Base.remove_linenums!

6

This sounds like a good approach, since I also need to sanitize the macro output using regular expressions (change some hard-coded paths etc.). How can I verify that the equivalence you postulated as a requirement is indeed fulfilled?

7

Also, why do you need to call remove line numbers twice?

8

Define string_remove_linenums!(expr) = string(Base.remove_linenums!(expr)), set str = string_remove_linenums!(expr), and then we can check whether string_remove_linenums!(Meta.parse(str)) == str is true or not.

Example 1:

string_remove_linenums!(expr) = string(Base.remove_linenums!(expr))
expr = @macroexpand g(x) = @time (sleep(1); sin(x))
str = string_remove_linenums!(expr)
@show string_remove_linenums!(Meta.parse(str)) == str;

Result: true

Example 2:

macro h(x, p...)
    ex = :($(p[end]))
    for i in length(p)-1:-1:1
        ex = :(muladd($(esc(x)), $ex, $(p[i])))
    end
    ex
end

string_remove_linenums!(expr) = string(Base.remove_linenums!(expr))
expr = @macroexpand f(x) = (x2 = x^2; x * @h(x2, 1.0, -0.16666666666666666, 0.008333333333333333))
str = string_remove_linenums!(expr)
@show string_remove_linenums!(Meta.parse(str)) == str;

Result: true

Simple mistake. I edited and fixed it. :wink:

1 Like