# Register and specialize a function in a different module/package (Julia segfaults)

**URL:** <https://discourse.julialang.org/t/register-and-specialize-a-function-in-a-different-module-package-julia-segfaults/35591>\
**Category:** General Usage\
**Created:** [March 5, 2020, 3:51pm UTC](https://discourse.julialang.org/t/register-and-specialize-a-function-in-a-different-module-package-julia-segfaults/35591 "2020-03-05T15:51:18Z")\
**Posts on this page:** 13\
**Page:** 1

<div class="post-metadata">

**Author:** ![essenciary](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/essenciary/32/210469_2.png) [@essenciary](https://discourse.julialang.org/u/essenciary)\
**Post date:** [March 5, 2020, 3:51pm UTC](https://discourse.julialang.org/t/register-and-specialize-a-function-in-a-different-module-package-julia-segfaults/35591/1 "2020-03-05T15:51:18Z")

</div>

I am working on a DSL for creating web pages. Something like this (Julia like pseudo-code):

```julia
module Html

function table(f::Function; args...)
# actual implementation here
end

function tr(f::Function; args...)
# actual implementation here
end

function td(f::Function; args...)
# actual implementation here
end

end

```

A user would produce code as:

```julia
using Html

table(border=0) do 
  tr(style="border: 1px solid red") do 
    td()
    td()
  end
end

```

The `Html` module provides an API for all HTML5 elements, which is a well defined set (so function are created for each standard element). This allows other packages to extend the `Html` API, for instance to provide more advanced `table` APIs:

```julia
module HtmlX

using Html

function Html.table(rows::Vector, columns::Vector)
# actual implementation here
end

function Html.table(df::DataFrames.DataFrame)
# actual implementation here
end

end

```

So all goes well for APIs for working with the standard HTML5 elements. However, the goal is to use `web components`, which extend the HTML5 standard with random elements. Like say a `paginated_table`. The problem is that `Html` can not define a `paginated_table` function in advance.

So I tried something like this:

```julia
module Html

function register_element(elem)
  Core.eval(@ __MODULE__ , Meta.parse("function $elem end"))
end

end

```

But when running:

```julia
module HtmlX

Html.register_element("paginated_table")

end

```

Julia complains that I’m evaling in a closed module.

The objective is to have a rich ecosystem of 3rd party packages where users install and use a variety of such external elements. I don’t like the idea of having the elements defined in say 10 modules within a project, as this would be very confusing for the users. An alternative would be for the packages to export their functions but this would lead to an inconsistent API as `Html` does not export the functions.

Any ideas or best practices? Thanks!

---

<div class="post-metadata">

**Author:** ![simeonschaub](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/simeonschaub/32/216566_2.png) [@simeonschaub](https://discourse.julialang.org/u/simeonschaub)\
**Post date:** [March 5, 2020, 4:20pm UTC](https://discourse.julialang.org/t/register-and-specialize-a-function-in-a-different-module-package-julia-segfaults/35591/2 "2020-03-05T16:20:30Z")

</div>

You don’t want to use `Core.eval`, but `<module>.eval` instead. The following works for me:

```julia
julia> module M
       f(s) = @ __MODULE__ ().eval(:(function $s end))
       end
Main.M

julia> M.f(:test)
test (generic function with 0 methods)

```

Although I’m not so sure, the lost transparency of this approach over having packages export their own modules/functions is really worth it.  
Edit:  
Or just call `Html.eval(...)` from outside the Html Module.

---

<div class="post-metadata">

**Author:** ![essenciary](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/essenciary/32/210469_2.png) [@essenciary](https://discourse.julialang.org/u/essenciary)\
**Post date:** [March 5, 2020, 5:41pm UTC](https://discourse.julialang.org/t/register-and-specialize-a-function-in-a-different-module-package-julia-segfaults/35591/3 "2020-03-05T17:41:00Z")

</div>

@simeonschaub Thank you! This is surprising, I expected that `eval module code` would have the same behaviour as `module.eval code`. I mean, hmmm, why different `eval` behaviours and why having restrictions on one if there’s another way of modifying a closed module.

I think that having all the HTML elements defined within `Html` would respect the principle of least surprise - users would just know where to find them. Otherwise, for consistency, I’d have to export over 100 functions defined within the `Html` module itself.

* * *

Related, I love the way functions in other modules can be specialised with Julia, it’s fantastic! But something that I’m missing and would make sense in this case (adding new functions) would be to allow reopening modules like Ruby does. Might be interesting for Julia v2:

```julia
# PaginatedTable.jl
opened module Html
function paginated_table end
end

module PaginatedTable

using Html

function Html.paginated_table()
# implementation
end

end

```

---

<div class="post-metadata">

**Author:** ![simeonschaub](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/simeonschaub/32/216566_2.png) [@simeonschaub](https://discourse.julialang.org/u/simeonschaub)\
**Post date:** [March 5, 2020, 5:54pm UTC](https://discourse.julialang.org/t/register-and-specialize-a-function-in-a-different-module-package-julia-segfaults/35591/4 "2020-03-05T17:54:49Z")

</div>

Are you talking about something similar to a header file? It seems to me like this is something that could probably be solved by tooling, maybe combined with some metaprogramming

---

<div class="post-metadata">

**Author:** ![yuyichao](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yuyichao/32/20_2.png) [@yuyichao](https://discourse.julialang.org/u/yuyichao)\
**Post date:** [March 5, 2020, 6:55pm UTC](https://discourse.julialang.org/t/register-and-specialize-a-function-in-a-different-module-package-julia-segfaults/35591/5 "2020-03-05T18:55:02Z")

</div>

FWIW, `Core.eval(module)` is better than `module.eval()`

---

<div class="post-metadata">

**Author:** ![essenciary](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/essenciary/32/210469_2.png) [@essenciary](https://discourse.julialang.org/u/essenciary)\
**Post date:** [March 6, 2020, 8:25am UTC](https://discourse.julialang.org/t/register-and-specialize-a-function-in-a-different-module-package-julia-segfaults/35591/6 "2020-03-06T08:25:20Z")

</div>

It would produce a similar result, but less sucky and more powerful. In Ruby, one can simply reopen a module and operate in it as if it was yours (and add and remove methods for example).

Here, example from the Ruby docs:  
[![](https://global.discourse-cdn.com/julialang/original/3X/6/7/677e59d33b9401ec216456f4b9086d052694249a.jpeg) ](https://dl.dropboxusercontent.com/s/iallob6tkwsuyzm/last%20image.jpg)

---

<div class="post-metadata">

**Author:** ![essenciary](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/essenciary/32/210469_2.png) [@essenciary](https://discourse.julialang.org/u/essenciary)\
**Post date:** [March 6, 2020, 8:26am UTC](https://discourse.julialang.org/t/register-and-specialize-a-function-in-a-different-module-package-julia-segfaults/35591/7 "2020-03-06T08:26:30Z")

</div>

Would you be so kind to elaborate in which way is it better and why? Always curious to learn about Julia’s internals. Thanks!

---

<div class="post-metadata">

**Author:** ![yuyichao](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yuyichao/32/20_2.png) [@yuyichao](https://discourse.julialang.org/u/yuyichao)\
**Post date:** [March 6, 2020, 12:42pm UTC](https://discourse.julialang.org/t/register-and-specialize-a-function-in-a-different-module-package-julia-segfaults/35591/8 "2020-03-06T12:42:21Z")

</div>

Because `module.eval` is not internal. It may or may not exist and may or may not do anything you want.

---

<div class="post-metadata">

**Author:** ![rdeits](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rdeits/32/286_2.png) [@rdeits](https://discourse.julialang.org/u/rdeits)\
**Post date:** [March 6, 2020, 1:49pm UTC](https://discourse.julialang.org/t/register-and-specialize-a-function-in-a-different-module-package-julia-segfaults/35591/9 "2020-03-06T13:49:42Z")

</div>

> [@yuyichao](#):
>
> It may or may not exist and may or may not do anything you want.

For example:

```julia
julia> module BadlyBehaved
         function eval(x)
           println("nope")
         end
       end
Main.BadlyBehaved

julia> BadlyBehaved.eval(:(x = 1))
nope

julia> Core.eval(BadlyBehaved, :(x = 1))
1

julia> BadlyBehaved.x
1

```

---

<div class="post-metadata">

**Author:** ![essenciary](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/essenciary/32/210469_2.png) [@essenciary](https://discourse.julialang.org/u/essenciary)\
**Post date:** [March 6, 2020, 1:52pm UTC](https://discourse.julialang.org/t/register-and-specialize-a-function-in-a-different-module-package-julia-segfaults/35591/10 "2020-03-06T13:52:42Z")

</div>

@yuyichao @rdeits Ah yes, thanks, makes total sense!

---

<div class="post-metadata">

**Author:** ![essenciary](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/essenciary/32/210469_2.png) [@essenciary](https://discourse.julialang.org/u/essenciary)\
**Post date:** [March 6, 2020, 2:12pm UTC](https://discourse.julialang.org/t/register-and-specialize-a-function-in-a-different-module-package-julia-segfaults/35591/11 "2020-03-06T14:12:35Z")

</div>

However, sadly, it does not work.

```julia
┌ Debug: 2020-03-06 14:38:53 Precompiling Stipple [c3245d6e-4e41-4672-accb-64620d5e5a71]
└ @ Base loading.jl:1273

WARNING: eval into closed module Html:
Expr(:function, :stylesheet)
  **incremental compilation may be fatally broken for this module**

```

And then Julia segfaults:

```julia
signal (11): Segmentation fault: 11

```

This is the offending code:

```julia
Core.eval(Genie.Renderer.Html, :(function stylesheet end))

function Genie.Renderer.Html.stylesheet(href::String; args...) :: String
  Genie.Renderer.Html.link(href=href, rel="stylesheet", args...)
end

```

Running various tests seems to confirm that the first part is successful (defining the function) but the second (specializing the function) causes Julia to crash. This behaviour was consistent throughout all my tests (always segfaults).

Leaving just the `eval` and running the function specialization code from the REPL works fine.

* * *

Update 1:  
Extracting away just the code, out of the package I’m developing into a minimal test file, works fine. So some combination of factors causes the segfault.

---

<div class="post-metadata">

**Author:** ![essenciary](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/essenciary/32/210469_2.png) [@essenciary](https://discourse.julialang.org/u/essenciary)\
**Post date:** [March 6, 2020, 3:04pm UTC](https://discourse.julialang.org/t/register-and-specialize-a-function-in-a-different-module-package-julia-segfaults/35591/12 "2020-03-06T15:04:13Z")

</div>

OK, pfff, so I managed to figure out when the segfault occurs. It seems to happen only when doing the eval and the specialization within a package.

Thus, if I place this code in a `test.jl` file:

```julia
module Stipple

using Genie

try
  Core.eval(Genie.Renderer.Html, :(function stylesheet end))
catch ex
  @error ex
end

function Genie.Renderer.Html.stylesheet(href::String; args...) :: String
  Genie.Renderer.Html.link(href=href, rel="stylesheet", args...)
end

end

```

and then run `$ julia test.jl` it works perfectly.

However, if I put the above code within a developed package called `Stipple` and then in `test.jl` I put `using Stipple`, Julia will segfault with:

```julia
$ julia test.jl                                                                                                                                                                                             
WARNING: eval into closed module Html:
Expr(:function, :stylesheet)
  **incremental compilation may be fatally broken for this module**

signal (11): Segmentation fault: 11
in expression starting at /Users/adrian/test.jl:25
jl_unwrap_unionall at /Users/julia/buildbot/worker/package_macos64/build/src/jltypes.c:953
jl_deserialize_value_any at /Users/julia/buildbot/worker/package_macos64/build/src/dump.c:1986 [inlined]
jl_deserialize_value at /Users/julia/buildbot/worker/package_macos64/build/src/dump.c:2212
jl_deserialize_datatype at /Users/julia/buildbot/worker/package_macos64/build/src/dump.c:1371 [inlined]
jl_deserialize_value at /Users/julia/buildbot/worker/package_macos64/build/src/dump.c:2209
jl_deserialize_value_svec at /Users/julia/buildbot/worker/package_macos64/build/src/dump.c:1508 [inlined]
jl_deserialize_value at /Users/julia/buildbot/worker/package_macos64/build/src/dump.c:2057
jl_deserialize_datatype at /Users/julia/buildbot/worker/package_macos64/build/src/dump.c:1485 [inlined]
jl_deserialize_value at /Users/julia/buildbot/worker/package_macos64/build/src/dump.c:2209
jl_deserialize_struct at /Users/julia/buildbot/worker/package_macos64/build/src/dump.c:1901
jl_deserialize_typemap_entry at /Users/julia/buildbot/worker/package_macos64/build/src/dump.c:1941 [inlined]
jl_deserialize_value at /Users/julia/buildbot/worker/package_macos64/build/src/dump.c:2195
jl_deserialize_struct at /Users/julia/buildbot/worker/package_macos64/build/src/dump.c:1901
jl_deserialize_value_any at /Users/julia/buildbot/worker/package_macos64/build/src/dump.c:2013 [inlined]
jl_deserialize_value at /Users/julia/buildbot/worker/package_macos64/build/src/dump.c:2212
jl_deserialize_value_any at /Users/julia/buildbot/worker/package_macos64/build/src/dump.c:1981 [inlined]
jl_deserialize_value at /Users/julia/buildbot/worker/package_macos64/build/src/dump.c:2212
jl_deserialize_datatype at /Users/julia/buildbot/worker/package_macos64/build/src/dump.c:1481 [inlined]
jl_deserialize_value at /Users/julia/buildbot/worker/package_macos64/build/src/dump.c:2209
jl_deserialize_value_module at /Users/julia/buildbot/worker/package_macos64/build/src/dump.c:1815 [inlined]
jl_deserialize_value at /Users/julia/buildbot/worker/package_macos64/build/src/dump.c:2134
jl_deserialize_value_array at /Users/julia/buildbot/worker/package_macos64/build/src/dump.c:1573
_jl_restore_incremental at /Users/julia/buildbot/worker/package_macos64/build/src/dump.c:3213
jl_restore_incremental at /Users/julia/buildbot/worker/package_macos64/build/src/dump.c:3272
_include_from_serialized at ./loading.jl:676
_require_from_serialized at ./loading.jl:743
_require at ./loading.jl:1034
require at ./loading.jl:922
require at ./loading.jl:917
jl_apply at /Users/julia/buildbot/worker/package_macos64/build/src/./julia.h:1631 [inlined]
call_require at /Users/julia/buildbot/worker/package_macos64/build/src/toplevel.c:399 [inlined]
eval_import_path at /Users/julia/buildbot/worker/package_macos64/build/src/toplevel.c:436
jl_toplevel_eval_flex at /Users/julia/buildbot/worker/package_macos64/build/src/toplevel.c:656
jl_parse_eval_all at /Users/julia/buildbot/worker/package_macos64/build/src/ast.c:873
jl_load at /Users/julia/buildbot/worker/package_macos64/build/src/toplevel.c:878 [inlined]
jl_load_ at /Users/julia/buildbot/worker/package_macos64/build/src/toplevel.c:885
include at ./boot.jl:328 [inlined]
include_relative at ./loading.jl:1105
include at ./Base.jl:31
exec_options at ./client.jl:287
_start at ./client.jl:460
jfptr__start_2084.clone_1 at /Applications/Julia-1.3.app/Contents/Resources/julia/lib/julia/sys.dylib (unknown line)
true_main at /usr/local/bin/julia (unknown line)
main at /usr/local/bin/julia (unknown line)
Allocations: 5898322 (Pool: 5896950; Big: 1372); GC: 7
fish: 'julia test.jl' terminated by signal SIGSEGV (Address boundary error)

```

Any clues?

---

<div class="post-metadata">

**Author:** ![essenciary](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/essenciary/32/210469_2.png) [@essenciary](https://discourse.julialang.org/u/essenciary)\
**Post date:** [March 6, 2020, 3:33pm UTC](https://discourse.julialang.org/t/register-and-specialize-a-function-in-a-different-module-package-julia-segfaults/35591/13 "2020-03-06T15:33:27Z")

</div>

100% confirm (sorry for spamming but this might be important). The issue occurs when evaling and specializing a function within a package from another package. Evaling leads to a warning (`eval into closed module`) while specializing results in segfault.

This is the code:

```julia
module EvalPkgTest

using Revise

Core.eval(Revise, :(function foobar end))

function Revise.foobar() :: String
  "foo bar"
end

end # module

```

This is the package:  
[https://www.dropbox.com/s/4y9qp4epqgbidb7/EvalPkgTest.zip?dl=0](https://www.dropbox.com/s/4y9qp4epqgbidb7/EvalPkgTest.zip?dl=0)  
(just a regular package created with `PkgTemplates`).

I think this info should go as an issue in Julia? Or Pkg?