Suppose I have embedded Julia in a C++ application so I can write plugins/extensions in Julia. The form of each plugin is (more or less) the following.
module <dynamically-generated>
import Main.bindings # Julia bindings for application API
function plugin(arg1, args)
...
end
end
The plugin code is dynamic and must be evaluated at runtime using jl_eval_string or an equivalent (i.e. using Julia packages is not an option). Is there any way I can cache the bytecode/binary the first time the plugin is JIT compiled so it can be used by subsequent executions of the C++ application?
When I looked at Julia a couple years ago it seemed like bytecode/binary caching was only possible for packages, not modules. Reading through some of the recent exchanges, it seems like caching is still not supported for modules, but maybe (hopefully!) I am wrong about that?
To provide some context, I did a performance comparison between embedded Python and embedded Julia a couple years ago. Ignoring quality of implementation issues (a big issue for third-party Julia packages), the average execution time of a Julia plugin, once JIT compiled, was substantially lower than that of a functionally equivalent Python plugin (as expected). The problem was that the cost to JIT compile the Julia plugin on first execution was so high that I would have had to execute the plugin hundreds or thousands of times before its cumulative execution time fell below that of its Python counterpart. If I could amortize the JIT compilation cost (via caching) across many executions of the C++ application it might bend the curve in Julia’s favor.
A package in its most basic form is just a compiled module, so I’m not sure why you are ruling that out so quickly. Normally, we would want things like a UUID, but we can also do away with that by manipulating the LOAD_PATH. Here’s an example.
julia> readdir() # current directory is empty
String[]
# Write a simple module to MyModule.jl in the current directory
julia> open("MyModule.jl", "w") do io
print(io,"""
module MyModule
greet() = println("Hello World")
end
""")
end
julia> push!(LOAD_PATH, pwd())
4-element Vector{String}:
"@"
"@v#.#"
"@stdlib"
"/home/mkitti/temptemp"
julia> using MyModule
[ Info: Precompiling MyModule [top-level]
julia> Base.compilecache(Base.PkgId(MyModule)) # this forces recompilation, and locates the cache
[ Info: Precompiling MyModule [top-level]
("/home/mkitti/.julia/compiled/v1.9/MyModule.ji", "/home/mkitti/.julia/compiled/v1.9/MyModule.so")
In a new Julia session, we manipulate the LOAD_PATH, and we can see that there is no precompilation. The module loads instantly.
julia> push!(LOAD_PATH, pwd())
4-element Vector{String}:
"@"
"@v#.#"
"@stdlib"
"/home/mkitti/temptemp"
julia> using MyModule
julia> MyModule.greet()
Hello World
julia>
Thank you, @mkitti, I wonder if the difference is between using using or import in native Julia code and using jl_eval_string to evaluate the code. This seems to suggest that might be the case. I will do some experimentation and report back.