DiffEqParamEstim.jl on Ensemble problems: autodiff doesn't work

Specific Domains Modelling & Simulations
1

Hi!
Hi! I’m not sure if what I have highlighted below is a bug/missing design feature/necessary evil. Could anyone enlighten me? Thanks!

I’m using the excellent build_loss_objective() function from DiffEqParamEstim.jl, to build differentiable loss functions on the solutions of ODEs

build_loss_objective(…;mpg_autodiff=true) provides automatic differentiation of the loss, using ForwardDiff. The fallback is a finite-difference derivative. mpg_autodiff works great on individual ODEProblems.

Unfortunately, if I use build_loss_objective() on an EnsembleProblem, it only works if mpg_autodiff is set to false. Otherwise, I get the following bug:

ERROR: type EnsembleProblem has no field p
Stacktrace:
 [1] getproperty(::EnsembleProblem{ODEProblem{Array{Float64,1},Tuple{Float64,Float64},true,Array{Float64,1},ODEFunction{true,typeof(pf_func),UniformScaling{Bool},Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing},Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}},DiffEqBase.StandardODEProblem},typeof(prob_func),typeof(DiffEqBase.DEFAULT_OUTPUT_FUNC),typeof(DiffEqBase.DEFAULT_REDUCTION),Nothing}, ::Symbol) at ./Base.jl:33
 [2] top-level scope at REPL[182]:1
 [3] eval(::Module, ::Any) at ./boot.jl:331
 [4] eval_user_input(::Any, ::REPL.REPLBackend) at /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.4/REPL/src/REPL.jl:86
 [5] run_backend(::REPL.REPLBackend) at /Users/dhruvaraman/.julia/packages/Revise/MgvIv/src/Revise.jl:1023
 [6] top-level scope at none:0

Rather than provide a minimal working example of my own, I will hijack the tutorial here. If you add the kwarg ‘mpg_autodiff = true’ to the build_loss_objective() function, the same type of error occurs.

Thanks!

2

yeah that’s a bit difficult to implement in this form. I think DiffEqFlux.jl: High Level Scientific Machine Learning (SciML) Pre-Built Architectures · DiffEqFlux.jl, specifically https://diffeqflux.sciml.ai/dev/examples/LV-stochastic/ can be more helpful here because then you can define a loss over ensemble problems directly in a way where AD works over it.

1 Like
3

Thanks! Fair enough. Maybe another option is building an array of DiffEqObjectives, and then making a new DiffEqObjective instance that sums them in parallel. I made a (serial) implementation of this that works (below). I need to figure out a good way of parallelising it now. Which might be obvious/impossible, I’m not that familiar with parallel processing but I’ll have a go.

function sum_losses(lArray::Array{T,1}, p0) where T<:DiffEqObjective
    dummy_g = deepcopy(p0)
    function cost1(p)
        return reduce(+, [loss(p) for loss in lArray])
    end

    function cost2(p,g)
        fill!(g,0)
        c = 0
        for el in lArray
            c += el(p,dummy_g)
            g .+= dummy_g
        end
        return c
    end
    return DiffEqObjective(cost1,cost2)
end