# "LoadError: Evaluation into the closed module" with Macro that generates new types and methods

**URL:** <https://discourse.julialang.org/t/loaderror-evaluation-into-the-closed-module-with-macro-that-generates-new-types-and-methods/94994>\
**Category:** General Usage\
**Tags:** metaprogramming\
**Created:** [February 22, 2023, 3:24am UTC](https://discourse.julialang.org/t/loaderror-evaluation-into-the-closed-module-with-macro-that-generates-new-types-and-methods/94994 "2023-02-22T03:24:13Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![mrufsvold](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mrufsvold/32/31600_2.png) [@mrufsvold](https://discourse.julialang.org/u/mrufsvold)\
**Post date:** [February 22, 2023, 3:24am UTC](https://discourse.julialang.org/t/loaderror-evaluation-into-the-closed-module-with-macro-that-generates-new-types-and-methods/94994/1 "2023-02-22T03:24:13Z")

</div>

I’m working on a package that allows users to define pipeline tasks in an ergonomic way and then schedules jobs and their dependencies with `Dagger.jl`.

I have a macro that takes a user input like this:

```julia
@Job begin
    name = MyJob
    parameters = (a,b)
    dependencies = AnotherJob(a)
    process = f(a,b)
end

```

The macro then takes this information, makes a new struct called `MyJob` and splices the user’s input into to `get_dependencies(j::MyJob)`, `run_process(j::MyJob)`, and a few others. My test suite works perfectly from the Pkg test environment.

> **The macro's code looks like this:**
>
> ```julia
> """
> Unpack the parameters of the job, then splice the body of the function provided by the user
> """
> function unpack_input_function(function_name, job_name, parameter_names, function_body, additional_inputs=())
> return quote
> function Waluigi.$function_name(job::$job_name, $(additional_inputs...))
> let $((:($name = job.$name) for name in parameter_names)...)
> $function_body
> end
> end
> end
> end
> 
> macro Job(job_description)
> # This returns a Dict of job parameters and their values
> job_features = extract_job_features(job_description)
>    
> #Truncating some complicated code here that just extracts the parameters
> paramter_names = complicated_code(job_features)
> job_name = job_features[:name]
>     
> dependency_ex = unpack_input_function(:get_dependencies, job_name, parameter_names, job_features[:dependencies])
> process_ex = unpack_input_function(:run_process, job_name, parameter_names, job_features[:process])
>     
> struct_def = :(struct $job_name <: AbstractJob end)
> push!(struct_def.args[3].args, parameter_names...)
> return quote
> $struct_def
> eval($(esc(dependency_ex)))
> eval($(esc(process_ex)))
> end
> end
> 
> ```

However, I just tried to add the package to another project. When I try to define a Job, I get the following error when trying to precompile:

```julia
LoadError: Evaluation into the closed module `Waluigi` breaks incremental compilation because the side
effects will not be permanent. This is likely due to some other module mutating `Waluigi` with `eval` 
during precompilation - don't do this.

```

I think I understand that adding new type definitions and/or function dispatches within the dependency module doesn’t work, but I’m not sure how to approach changing the way the macro defines things to get it to work properly. By that I mean, I want the generic pipeline code I’ve written that calls `get_dependencies` to see the new dispatches.

Or it’s possible I’m approaching this in a completely wrong way. Open to that feedback too 🙂

---

<div class="post-metadata">

**Author:** ![mrufsvold](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mrufsvold/32/31600_2.png) [@mrufsvold](https://discourse.julialang.org/u/mrufsvold)\
**Post date:** [February 22, 2023, 3:26am UTC](https://discourse.julialang.org/t/loaderror-evaluation-into-the-closed-module-with-macro-that-generates-new-types-and-methods/94994/2 "2023-02-22T03:26:35Z")

</div>

FWIW: [This post](https://discourse.julialang.org/t/use-of-eval-in-closed-module-and-incremental-compilation-may-be-broken/30028/6) might be the solution, but since it was using `eval` in a function, I’m not sure how to apply it to my situation.

---

<div class="post-metadata">

**Author:** ![Sukera](https://avatars.discourse-cdn.com/v4/letter/s/ce7236/32.png) [@Sukera](https://discourse.julialang.org/u/Sukera)\
**Post date:** [February 22, 2023, 2:46pm UTC](https://discourse.julialang.org/t/loaderror-evaluation-into-the-closed-module-with-macro-that-generates-new-types-and-methods/94994/3 "2023-02-22T14:46:20Z")

</div>

> [@mrufsvold](#):
>
> ```julia
> return quote
> $struct_def
> eval($(esc(dependency_ex)))
> eval($(esc(process_ex)))
> end
> 
> ```

A macro already returns code that is then evaluated - using `eval` in a macro is likely going to be the issue here (even though the `eval` is part of the returned expression, there’s no need for it). Do you have an example of the code you would have written/that you want the macro to expand to? Also, what does your macro return without those `eval` (use `@macroexpand` to find out without actually running the resulting code)? Do the two match?

---

<div class="post-metadata">

**Author:** ![mrufsvold](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mrufsvold/32/31600_2.png) [@mrufsvold](https://discourse.julialang.org/u/mrufsvold)\
**Post date:** [February 22, 2023, 3:02pm UTC](https://discourse.julialang.org/t/loaderror-evaluation-into-the-closed-module-with-macro-that-generates-new-types-and-methods/94994/4 "2023-02-22T15:02:58Z")

</div>

Yes, `@macroexpand` matches my expected example.

Truncating a bit:

```julia
struct MyJob <: Waluigi.AbstractTarget
    a
    b
end

function Waluigi.get_dependencies(job::MyJob)
    let a = job.a, b = job.b
        AnotherJob(a)
    end
end

function Waluigi.run_process(job::MyJob, dependencies)
    let a = job.a, b = job.b
        f(a, b)
    end
end

```

You’re right that, because I added `eval` each of the function definitions, they are wrapped in `Walugi.eval(...)`. However, if I don’t do this, then when I later try to run the code that calls `run_process`, it doesn’t find the methods that the macro defines. If I add the `eval`, it works.

After writing all this out, I’m realizing my actual question is:

While still evaluating the definition of a method in the user’s module space, how do I get my package to pick up new the methods? When I want to extend a function from `Base`, it’s sufficient to just add the source module’s name `Base.sum(::MyType) = ...`, and everything seems to pick up on it.

---

<div class="post-metadata">

**Author:** ![Sukera](https://avatars.discourse-cdn.com/v4/letter/s/ce7236/32.png) [@Sukera](https://discourse.julialang.org/u/Sukera)\
**Post date:** [February 22, 2023, 3:17pm UTC](https://discourse.julialang.org/t/loaderror-evaluation-into-the-closed-module-with-macro-that-generates-new-types-and-methods/94994/5 "2023-02-22T15:17:46Z")

</div>

> [@mrufsvold](#):
>
> However, if I don’t do this, then when I later try to run the code that calls `run_process`, it doesn’t find the methods that the macro defines.

What error message do you get? I suspect your use of just putting `esc` on the whole returned expression and not on something like `Waluigi.run_process` ends up breaking [hygiene](https://docs.julialang.org/en/v1/manual/metaprogramming/#Hygiene). I can’t verify that though because your MWE is not complete - I don’t have access to `extract_job_features`, for example.

---

<div class="post-metadata">

**Author:** ![mrufsvold](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mrufsvold/32/31600_2.png) [@mrufsvold](https://discourse.julialang.org/u/mrufsvold)\
**Post date:** [February 22, 2023, 4:08pm UTC](https://discourse.julialang.org/t/loaderror-evaluation-into-the-closed-module-with-macro-that-generates-new-types-and-methods/94994/6 "2023-02-22T16:08:14Z")

</div>

Ah, thank you so much, @Sukera! I was being too broad with `exc`ing

Here is an actually working MWE for what I was trying to do for future googlers:

```julia
module A
abstract type Abstract end
f(::Abstract) = error("Not Implemented")

macro make_type(type_name)
    quote
        struct $type_name <: $(esc(A.Abstract)) end
        function A.f(::$(esc(type_name)))
            println("It worked!")
        end
    end
end

end # mod A

using .A 

module B
using ..A

A.@make_type NewType

A.f(NewType())
# prints "it works"!
end

```