# Nested Macros Bypass Package Dependencies

**URL:** <https://discourse.julialang.org/t/nested-macros-bypass-package-dependencies/2683>\
**Category:** Internals & Design\
**Created:** [March 15, 2017, 6:34pm UTC](https://discourse.julialang.org/t/nested-macros-bypass-package-dependencies/2683 "2017-03-15T18:34:44Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [March 15, 2017, 6:34pm UTC](https://discourse.julialang.org/t/nested-macros-bypass-package-dependencies/2683/1 "2017-03-15T18:34:44Z")

</div>

Just ran into an issue in the DiffEq-sphere that seems like it might be a more systemic problem. I am wondering if there is a proper way to address it, or if this needs to be addressed in Julia itself. The issue is here:

[https://github.com/JuliaDiffEq/StochasticDiffEq.jl/issues/16#issuecomment-286812619](https://github.com/JuliaDiffEq/StochasticDiffEq.jl/issues/16#issuecomment-286812619)

but the problem is summed up as this. To define a type, I am using Parameters.jl. For example:

```julia
@with_kw immutable EM{RSType} <: StochasticDiffEqAlgorithm
  rswm::RSType = RSWM(adaptivealg=:RSwM1)
end

```

the `@with_kw` makes the constructor have a keyword argument as you’d expect from this definition, and all is well. However, last night a release was put in to Parameters.jl which updated it to use v0.6. Now, users suddenly get an error:

```julia
ERROR: LoadError: LoadError: UndefVarError: @compat not defined
 in macro expansion; at ./none:2 [inlined]
 in anonymous at ./<missing>:?
while loading /home/blegat/.julia/v0.5/StochasticDiffEq/src/algorithms.jl, in expression starting on line 5
while loading /home/blegat/.julia/v0.5/StochasticDiffEq/src/StochasticDiffEq.jl, in expression starting on line 21
ERROR: Failed to precompile StochasticDiffEq to /home/blegat/.julia/lib/v0.5/StochasticDiffEq.ji.
 in eval_user_input(::Any, ::Base.REPL.REPLBackend) at ./REPL.jl:64
 in macro expansion at ./REPL.jl:95 [inlined]
 in (::Base.REPL.##3#4{Base.REPL.REPLBackend})() at ./event.jl:68

```

That means that `Compat` isn’t in the modules scope. But it was added to Parameters.jl’s module scope in the recent change. The issue is that the requirement for the dependency leaked out:

[https://github.com/mauro3/Parameters.jl/compare/v0.6.0...master#diff-ec1155cc07b8ae5fc6787c9cb23a7986R359](https://github.com/mauro3/Parameters.jl/compare/v0.6.0...master#diff-ec1155cc07b8ae5fc6787c9cb23a7986R359)

The `@with_kw` macro now puts in an `@compat`. So when that expands, it ends up with an expression which has `@compat` in it. So it seems that StochasticDiffEq.jl now needs a requirement and needs to pull `@compat` into its scope.

Someone probably has a good minimum example for this, I’m mostly frantically trying to fix this right now. I am sure that it’s a leaking of the Compat dependency since I added `using Compat` to StochasticDiffEq.jl master, but also I can fix release’s “you need Compat” bug by pinning Parameters.jl to v0.6.0.

Is there a way that we should be using nested macros to avoid this problem? Or should Julia be using the scope of where the macro was defined to handle nested macros?

---

<div class="post-metadata">

**Author:** ![yuyichao](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yuyichao/32/20_2.png) [@yuyichao](https://discourse.julialang.org/u/yuyichao)\
**Post date:** [March 15, 2017, 6:40pm UTC](https://discourse.julialang.org/t/nested-macros-bypass-package-dependencies/2683/2 "2017-03-15T18:40:50Z")

</div>

I didn’t look at the source but the macro must not `esc`ape anything that it wants to be resolved in the callee (macro) module.

---

<div class="post-metadata">

**Author:** ![mbauman](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mbauman/32/31082_2.png) [@mbauman](https://discourse.julialang.org/u/mbauman)\
**Post date:** [March 15, 2017, 6:43pm UTC](https://discourse.julialang.org/t/nested-macros-bypass-package-dependencies/2683/3 "2017-03-15T18:43:55Z")

</div>

Yes, this is likely a misplaced `esc`.

```julia
julia> module M
       using Compat
       macro foo(x)
           :(@compat $(esc(x)))
       end
       macro bar(x)
           esc(:(@compat $x))
       end
       end
M

julia> M.@foo 1
1

julia> M.@bar 1
ERROR: UndefVarError: @compat not defined

```

---

<div class="post-metadata">

**Author:** ![yuyichao](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yuyichao/32/20_2.png) [@yuyichao](https://discourse.julialang.org/u/yuyichao)\
**Post date:** [March 15, 2017, 6:49pm UTC](https://discourse.julialang.org/t/nested-macros-bypass-package-dependencies/2683/4 "2017-03-15T18:49:14Z")

</div>

A quick search also return other wrong escapes.

[https://github.com/mauro3/Parameters.jl/blob/082972d2886165353a9ee6ff845c9cb1e73d872d/src/Parameters.jl#L212](https://github.com/mauro3/Parameters.jl/blob/082972d2886165353a9ee6ff845c9cb1e73d872d/src/Parameters.jl#L212)

is wrong.

The simplest solution if you want to escape everything would be to splice in the module i.e. `$Parameters` instead of `Main.Parameters`. You can do the same with `Compat`.

---

<div class="post-metadata">

**Author:** ![yuyichao](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yuyichao/32/20_2.png) [@yuyichao](https://discourse.julialang.org/u/yuyichao)\
**Post date:** [March 15, 2017, 6:50pm UTC](https://discourse.julialang.org/t/nested-macros-bypass-package-dependencies/2683/5 "2017-03-15T18:50:53Z")

</div>

And yeah, [this](https://github.com/mauro3/Parameters.jl/compare/v0.6.0...master#diff-ec1155cc07b8ae5fc6787c9cb23a7986L466) escape is wrong.

---

<div class="post-metadata">

**Author:** ![mauro3](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mauro3/32/292_2.png) [@mauro3](https://discourse.julialang.org/u/mauro3)\
**Post date:** [March 15, 2017, 6:51pm UTC](https://discourse.julialang.org/t/nested-macros-bypass-package-dependencies/2683/6 "2017-03-15T18:51:19Z")

</div>

Thanks to all for looking into this! I find the escaping is not the easiest to graps… I’ll fix it tomorrow.

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [March 15, 2017, 6:56pm UTC](https://discourse.julialang.org/t/nested-macros-bypass-package-dependencies/2683/7 "2017-03-15T18:56:43Z")

</div>

Thank you so much for explaining the issue with a nice small example of how to fix it. I was scratching my head as to what to do other than add Compat to each package! I’m putting in some version bounds on the DiffEq dependencies of Parameters as a temporary fix, and will help @mauro3 fix those escapes.

As for other dependencies. This is likely causing an issue in CMakeWrapper.jl (@rdeits) and SciKitLearn.jl (@cstjean). You might want to test and add version bounds.

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [March 15, 2017, 7:33pm UTC](https://discourse.julialang.org/t/nested-macros-bypass-package-dependencies/2683/8 "2017-03-15T19:33:39Z")

</div>

Thanks for the help everyone. We’ll work on Parameters.jl. In the meantime, I put in a PR for upper bounds which will fix the released versions of packages.

For DiffEq:

[https://github.com/JuliaLang/METADATA.jl/pull/8343](https://github.com/JuliaLang/METADATA.jl/pull/8343)

For others (separated so that way the authors can be notified and delay if needed):

[https://github.com/JuliaLang/METADATA.jl/pull/8344](https://github.com/JuliaLang/METADATA.jl/pull/8344)

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [March 15, 2017, 8:30pm UTC](https://discourse.julialang.org/t/nested-macros-bypass-package-dependencies/2683/9 "2017-03-15T20:30:53Z")

</div>

> [@yuyichao](#):
>
> You can do the same with Compat.

You mean, change `@compat` to `$Compat.@compat`?

> [@yuyichao](#):
>
> And yeah, this escape is wrong.

Which line are you referring to?

---

<div class="post-metadata">

**Author:** ![yuyichao](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yuyichao/32/20_2.png) [@yuyichao](https://discourse.julialang.org/u/yuyichao)\
**Post date:** [March 15, 2017, 9:17pm UTC](https://discourse.julialang.org/t/nested-macros-bypass-package-dependencies/2683/10 "2017-03-15T21:17:25Z")

</div>

> [@ChrisRackauckas](#):
>
> change @compat to $Compat.@compat?

Yes.

> [@ChrisRackauckas](#):
>
> Which line are you referring to?

Line 466, which you can see in the URL… in `macro with_kw`. Apparently github doesn’t automatically expand enough content when the link is to a line that’s collapsed by default…

---

<div class="post-metadata">

**Author:** ![mauro3](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mauro3/32/292_2.png) [@mauro3](https://discourse.julialang.org/u/mauro3)\
**Post date:** [March 16, 2017, 8:28am UTC](https://discourse.julialang.org/t/nested-macros-bypass-package-dependencies/2683/11 "2017-03-16T08:28:31Z")

</div>

[https://github.com/JuliaLang/METADATA.jl/pull/8348](https://github.com/JuliaLang/METADATA.jl/pull/8348)

Merged within 5min by @tkelman!

Thanks to all!

---

<div class="post-metadata">

**Author:** ![mauro3](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mauro3/32/292_2.png) [@mauro3](https://discourse.julialang.org/u/mauro3)\
**Post date:** [March 17, 2017, 10:13am UTC](https://discourse.julialang.org/t/nested-macros-bypass-package-dependencies/2683/12 "2017-03-17T10:13:53Z")

</div>

For those following along at home: Turns out that the `@compat` was not needed at all as the inner-constructor syntax I used

```julia
julia> type A{T}
       a::T
       (::Type{A{T}}){T}(a::T) = new{T}(a)
       end

```

is supported on 0.5 anyway. Also, carelessly adding the `@compat` lead to this issue: [https://github.com/mauro3/Parameters.jl/issues/29](https://github.com/mauro3/Parameters.jl/issues/29). Tagged here: [https://github.com/JuliaLang/METADATA.jl/pull/8363](https://github.com/JuliaLang/METADATA.jl/pull/8363)