Hello! I’m relatively new to Julia and have encountered some behavior I cannot explain.
In a MWE, I have a function that takes an ODEProblem and algorithm and just calls DiffEq.solve() with these arguments.
function myfun(prob, alg)
sol = solve(prob, alg)
end
When I put this function in a script (that I include in my main script), @time reports compilation time on the first run, but not when I paste the relevant line @time myscript(prob, alg) into the REPL or include my main script again. So far, so good.
But when I include the same function in a module (with Project.toml and Manifest.toml) and do using MyModule in my main script, something weird happens.
including main.jl in new Julia session: @time reports >99% compilation time, as expected
re-including main.jl from the same session: @time reports >99% compilation time, 100% of which was recompilation
if instead of re-including main.jl, I just paste the relevant line @time myfun(prob, alg) in the REPL, @time reports no compilation time (as I would expect)
So, why is re-including the main script different from just running the relevant line from the REPL? And why does this happen only when calling a function from a local module, and not when calling a function from a script?
Any help is appreciated!
(main.jl for reference, as I cannot upload files)
using DifferentialEquations
using MyModule
include("script.jl")
function simple_deriv!(du, u, p, t)
du[:] = -1.0 .* u
return nothing
end
t_0 = 0.0
t_end = 1.0
u_0 = [1.0]
alg = ImplicitEuler()
args = (; adaptive=false, dt=1 // 20)
prob = ODEProblem(simple_deriv!, u_0, (t_0, t_end); args...)
print("running from module:")
@time myfun(prob, alg)
print("running from script:")
@time myscript(prob, alg)
FTR, Julia features several startup options that make answering questions like this one much easier. I’m guessing they won’t be needed here, just FYI. These include:
--warn-overwrite=yes
--trace-compile=stderr
# only available on nightly
--trace-compile-timing
# only available on nightly
--trace-dispatch=stderr
What does this file contain? Does it contain a redefinition of myfun?
OK, now I realize I don’t really understand what issue are you describing, a more minimal reproducer could help clear things up.
Perhaps it’d help to explain some things about using and include:
using has multiple functionalities:
loading and precompiling packages: this happens only the first time you call using SomePackage
bringing the names of a module (possibly the top-level module of a package) into scope
The include function is a lower-level tool. What it does is literally include the contents of the file given as argument into the place where it was called. If you know C or C++, include in Julia is the equivalent of #include C preprocessor directive.
So if you do:
using SomePackage
using SomePackage
… the second using is a no-op.
If you do:
include("some_file")
include("some_file")
you will literally be including the contents of the file some_file into the caller file (or REPL) twice. So, for example, if some_file is just the definition of a function, calling include twice will first define the function, and then redefine it, overwriting the first definition.
Newcomers tend to misuse include a lot. In fact, include is only rarely actually necessary.
I was confused initially, some further investigation showed that defining the function in a module vs. in a script does not make a difference.
Here is a simplified example:
using DifferentialEquations
function simple_deriv!(du, u, p, t)
du[:] = -1.0 .* u
return nothing
end
t_0 = 0.0
t_end = 1.0
u_0 = [1.0]
alg = ImplicitEuler()
args = (; adaptive=false, dt=1 // 20)
prob = ODEProblem(simple_deriv!, u_0, (t_0, t_end); args...)
@time solve(prob, alg)
print("done")
My issue:
When including this file in a new session, @time reports >99% compilation time
running just @time solve(prob, alg) afterwards reports no compilation
re-defining simple_deriv! by pasting the same definition in the REPL, and then running @time solve(prob, alg) reports >99% compilation time, 100% of which was recompilation. The same happens when I re-include the file above from the REPL
I don’t understand why this is happening – if I re-define the function with the exact same expression, what changes? Why is recompilation necessary?
Julia doesn’t check that the body of the method is the same as before. Furthermore, I don’t think that condition is sufficient for the compiler to establish that the redefinition doesn’t require recompilation (invalidation) - it’d have to check that the behavior is completely the same, which means it’d have to check that for each mentioned name, called function, etc.
In short, redefining a method invalidates the compiled code, causing future calls to require recompilation.