Creating functions with ModelingToolkit for parallel usage

Hi,
after figuring out how to use ModelingToolkit to create functions/derivates etc. I would like to run code generated by ModelingToolkit in parallel. E.g. solving the Lorenz system for multiple initial states, see the following example:

using Distributed 
using ModelingToolkit
using DifferentialEquations

# create the Lorenz system
eqs_mt = []
push!(eqs_mt, dt(x) ~ lorenz_mt(x,y,z)[1])
push!(eqs_mt, dt(y) ~ lorenz_mt(x,y,z)[2])
push!(eqs_mt, dt(z) ~ lorenz_mt(x,y,z)[3])
de_mt = ODESystem(eqs_mt)
ode_func_mt = ODEFunction(de_mt, [x,y,z], [sigma,rho,beta])

u0 = [19.,20.,50.]
params = [16.,45.92,4]

ode_prob_mt = ODEProblem(ode_func_mt, u0, (0., 10.),params)

# add processes to workspace
addprocs(2)

@everywhere begin
    
    using DifferentialEquations
    using ModelingToolkit
    
    function solve_lorenz(ode_problem)
        print(solve(ode_problem))
    end
end


This code errors with

On worker 2:
UndefVarError: ###375 not defined
deserialize_datatype at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.1/Serialization/src/Serialization.jl:1115
handle_deserialize at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.1/Serialization/src/Serialization.jl:771
deserialize at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.1/Serialization/src/Serialization.jl:731
deserialize_datatype at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.1/Serialization/src/Serialization.jl:1139
handle_deserialize at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.1/Serialization/src/Serialization.jl:771
deserialize at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.1/Serialization/src/Serialization.jl:731
deserialize_datatype at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.1/Serialization/src/Serialization.jl:1139
handle_deserialize at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.1/Serialization/src/Serialization.jl:771
deserialize at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.1/Serialization/src/Serialization.jl:731
deserialize_datatype at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.1/Serialization/src/Serialization.jl:1139
handle_deserialize at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.1/Serialization/src/Serialization.jl:771
deserialize at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.1/Serialization/src/Serialization.jl:731
handle_deserialize at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.1/Serialization/src/Serialization.jl:778
deserialize at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.1/Serialization/src/Serialization.jl:731
#3 at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.1/Serialization/src/Serialization.jl:869
ntuple at ./tuple.jl:136
deserialize_tuple at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.1/Serialization/src/Serialization.jl:869
handle_deserialize at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.1/Serialization/src/Serialization.jl:761
deserialize_msg at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.1/Serialization/src/Serialization.jl:731
#invokelatest#1 at ./essentials.jl:742 [inlined]
invokelatest at ./essentials.jl:741 [inlined]
message_handler_loop at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.1/Distributed/src/process_messages.jl:160
process_tcp_streams at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.1/Distributed/src/process_messages.jl:117
#105 at ./task.jl:259

I assume that the generated function, with name ###375, is not available on the worker process as it was created on the main process. Is there a possibility to make this function available to worker processes without calling ODEFunction? I would like to avoid costly generation of Jacobians, etc. on each worker process, when they are already precalculated on the main process.

I need a better answer two for how to message pass a generic function. That’s really all it is. MWE: I already created f(x) = x locally, how to send it after?

Can you open an issue with a reproducible example? I think I have a way to fix this but I’m not sure if it’s the same as your issue.

I filed an issue on https://github.com/JuliaDiffEq/ModelingToolkit.jl/issues/174

1 Like

Fixed by https://github.com/JuliaDiffEq/ModelingToolkit.jl/pull/173 . Requires the new GG.jl to get registered!

1 Like