Keeping (AOT-compiling) runtime-generated functions

General Usage
1

I’m using code generation to generate efficient implementations of nested functions. That is, some functions are generated first and then used in another code-generated function. In a REPL-based workflow, this works fine. The generating code is JIT-compiled once, the functions are generated and then also JIT-compiled once. From then on, they can be called repeatedly with no further compilation. Visualisation:

(1) generator (static code) → (2) JIT → (3) generated function(s) → (4) JIT → (5) function calls

Now I’d like to “keep” them for later execution in another REPL session or in a one-off script invoked from the command line (i.e. AOT-compilation). If the functions were written as “static code” by hand, I could use, e.g., PackageCompiler to compile them into the sysimage or into a standalone executable. Is there a way of doing this for functions generated at runtime?

If I just compile the generating (no the generated) functions into the sysimage (steps 1-2), I still have the JIT-compilation of the latter in every run/call (step 4).

1 Like
2

Did you try putting precompile calls in your module (e.g., using SnoopCompile)? IIUC this works both with usual precompilation cache and system image.

(By the way, are you using generated function in the generator code of another generated function? FYI I don’t think it is OK, strictly speaking, because a generated function is a generic function.)

3

Ok, probably I’ll have to start one step before: Can I somehow compile methods into my sysimage that are defined by a script (or in the REPL), not in a module / package?

I just tested having a fully code-generated method defined in a module, e.g.

module Mytest

fun_name = :myfun
@eval export $fun_name

eval_code_gen(k) = :($(k[1]) + $(k[2])*x^2 + $(k[3])*x^3)
named_function_gen(code, name) = :( $name(x) = $code )

(1.2, 0.3, 0.54) |> eval_code_gen |> c -> named_function_gen(c, fun_name) |> eval

end # Mytest

and it works fine (which is nice per se) and myfun can be compiled into the system image (which is kinda obvious, since the module / package can be used as usual).

Now, the implementation above makes no sense; since everything is hardcoded, a plain function could be used. But my use case is more like if there was this method defined in the module:

export make_named_function
make_named_function(k, fun_name) = k |> eval_code_gen |> c -> named_function_gen(c, fun_name)

It’s a “generator” for a function for which I can specify (some) of its parameters. In the REPL or in a script it could be used like so:

make_named_function((1.4, -0.3, 0.33), :mycustomfun) |> eval
mycustomfun(0.9)

So my question is whether I can compile mycustomfun, defined in a plain script or in a REPL, into the system image? The use case is to have some external automation (think CMake / CTest) using several of those functions (with different “fixed” parameters k) in a vast amount of tests (parameter sweeps etc.) during which only x needs to be changed. Or, let the function be an entire mathematical model of a physical process PLUS its simulation for some scenario… and this simulation is used in the backend of an interactive “simulation web app”.

It would be nice if the to-be-compiled functions could be generated “on-the-fly” / dynamically, not in a module. And no, closures are not an option in the actual application, since “structural” parameters are included in k, which, for example, are used to construct tuples to collect / reorder values without dynamic memory allocation, greatly speeding up things…

4

Short version:

  • Can export a function that is created by eval in module A and use it in a REPL/script, an compile it into the system image.
  • Cannot use a function that is created by eval in module A in another module B.
  • Can pass eval method from module B to module A, then do everything (export, use, compile in sysimage).

Why is this? Why can I use something in the scope of the REPL, but not in another module? I guess I’m not experienced in Julia’s internals enough to see this…

Would it help to attach / wrap such dynamically generated (anonymous) functions to / in an instance of a struct with a Function-typed field?

Long version now. So far the following works:

  • I can generate and eval code within a module
  • I can re-use such code-generated and eval-ed functions in other such functions, within the same module
  • I can export such a function from the module
  • I can use this exported function in the global scope of a REPL session or a script
  • I can compile such an exported function into the system image

Now, what I can not do is:

  • Use such a function (code-gen and eval to define it) in another module

When I just do using this other module, I get the

WARNING: eval into closed module XXX

followed by a:

signal (11): Segmentation fault
in expression starting at /path/to/script/with/using.jl:1
unknown function (ip: 0x7fd86aac81ff)

Now, what I can do then is:

  • Modify the “generating” functions in the inner module to receive an eval method
  • In the outer / other module, pass the module’s eval method to the functions of the inner module
  • Import the outer module, use the code-generated an eval-ed function from it
  • … and compile it into the system image

This is a pain since I’d have to recursively pass eval methods into (possibly nested) modules.

5

Ok, now I’m very confused. When I move the two modules inside another common parent module (which is a dev package, as were the two individual modules before), everything works! I just get the WARNING: eval into closed module ... with the note that ** incremental compilation may be broken for this module ** upon precompilation, but that’s not a problem for me.