LoadError when using interpolations as input for a neural ode

I am trying to train a simple nn with an external input that depends on time. The input is interpolated using:

broadband = LinearInterpolation(time, U)

The network runs without any problem for the first iteration of the optimization process using ADAM. After calculating the loss, the program stops to work and I get the following error message:

ERROR: LoadError: ArgumentError: unable to check bounds for indices of type Intertions.WeightedAdjIndex{2,Float64}

Does the backpropagation algorithm fail when interpolation is involved?

Please find an excerpt of my code below:

# Define the neural network and ODE part
nn_model = FastChain(FastDense(4, 8, tanh), FastDense(8, 4))
p_model = initial_params(nn_model)

function dudt(u, p, t)
    nn_model(vcat(u[1], broadband(t), u[3], u[4]), p)

prob = ODEProblem(dudt, u0, [time_training[1], time_training[end]])

function predict_neuralode(p)
    _prob = remake(prob, p = p)
    sol = Array(solve(_prob, Tsit5(), saveat = t_step, abstol = 1e-8, reltol = 1e-6))
    sol = sol[3, :]

# Loss function defined as elementwise distance bewteen pred. actual data
function loss(p) # loss function
    sol = predict_neuralode(p)
    loss = sum(abs2, sol - Q_training)
    print("Loss: $loss")
    return loss

# start optimization
res0 = DiffEqFlux.sciml_train(loss, p_model, ADAM(0.01), maxiters = 100)

Can you share code that reproduces the error? I can’t see what would cause the error from what you posted.

Yes, below is a working example to reproduce the error. I had to replace my input which comes from a local file with some arbitray function that is interpolated, but the error is the same.

using DifferentialEquations
using Plots
using Flux
using DiffEqFlux
using Interpolations
using HDF5
import Statistics
import LinearAlgebra

# Define all required functions
# Define the double pendulum
function linear_target(du, u, p, t)
    ϕ_1, σ_1, ϕ_2, σ_2 = u
    α, β, γ = p
    du[1] = σ_1
    du[2] = γ * broadband(t) - α * σ_1 - ϕ_1 + β * (ϕ_2 - ϕ_1)
    du[3] = σ_2
    du[4] = -α * σ_2 - ϕ_2 - β * (ϕ_2 - ϕ_1)

# Main
# Create some input data
t_step = 0.01125
time = 0:t_step:450
U = sin.(time)

# Interpolate signal in time for later usage in ODE
broadband = LinearInterpolation(time, U)

# Create target data for later optimization
p_ref = [1.1340, 0.5644, 3.8565]
tspan = (0.0, time[end])
u0 = [0.0, 0.0, 0.0, 0.0]
prob = ODEProblem(linear_target, u0, tspan, p_ref)
sol = solve(prob, AutoTsit5(Rosenbrock23()), p = p_ref, saveat = t_step)
Y_ref = sol[3, :] # Take only σ2 as target

# Define the neural network and ODE part
nn_model = FastChain(FastDense(4, 8, tanh), FastDense(8, 4))
p_model = initial_params(nn_model)

function dudt(u, p, t)
    nn_model(vcat(u[1], broadband(t), u[3], u[4]), p)

prob = ODEProblem(dudt, u0, [time[1], time[end]])

function predict_neuralode(p)
    _prob = remake(prob, p = p)
    sol = Array(solve(_prob, Tsit5(), saveat = t_step, abstol = 1e-8, reltol = 1e-6))
    sol = sol[3, :]

# Loss function defined as elementwise distance bewteen pred. actual data
function loss(p) # loss function
    sol = predict_neuralode(p)
    loss = sum(abs2, sol - Y_ref)
    print("Loss: $loss")
    return loss

# start optimization
res0 = DiffEqFlux.sciml_train(loss, p_model, ADAM(0.01), maxiters = 100)

Dear Community,
if anyone stumbles across this blog entry and may has an idea what the problem could be, please let me know.
Although this thread is old, the problem remains highly relevant to me!
Thank you very much in advance!

Okay, I can see how to get this working.

First of all, unsurprisingly, it seems like the error is as a result of trying to differentiate LinearInterpolation.

julia> gradient(broadband, 0.5)
ERROR: ArgumentError: unable to check bounds for indices of type Interpolations.WeightedAdjIndex{2,Float64}

Some part of that package apparently doesn’t play well with autodiff.

The implication of this is that you can’t compute d(du/dt)/dt.

Now, fortunately, you don’t need to! It just so happens that you only need d(du/dt)/dt to backpropagate through an ODE wrt the initial time point tspan[1], which in your case is fixed. (Not a fact that’s terribly obvious, I don’t think. I only know this because I’ve worked on problems of this exact type before, and had to write the gradient calculations from scratch.)

As a result, it should suffice to simply dummy the gradient wrt t, and the rest of the gradients should be calculated just fine:

import Zygote

function dudt(u, p, t)
    b = Zygote.ignore(()->broadband(t))
    nn_model(vcat(u[1], b, u[3], u[4]), p)

This is a bit of a hack, of course, but it seems to work.

This aside, neural differential equations of this type (with time-dependent input) are known as neural controlled differential equations. There’s been a line of work on this you might find interesting:
Neural Controlled Differential Equations for Irregular Time Series, NeurIPS 2020
Neural Rough Differential Equations for Long Time Series, ICML 2021
PyTorch library: torchcde.
(+ one more paper coming soon)

And more tangentially (kind of just for fun wrt this discussion), using a SDE+CDE as a generator-discriminator pair in a GAN:
Neural SDEs as Infinite-Dimensional GANs, ICML 2021.

I hope that helps.


Caveat emptor:

I do note that there is also Zygote.dropgrad. I’m not completely certain of the difference between dropgrad and ignore, but I found that dropgrad gave some other obscure error. Possibly dropgrad is the appropriate tool here, but the documentation is pretty sparse. I haven’t tried training the above model more than a few steps to see if it converges.

As an alternative, you can try implementing the interpolation yourself. (Such that the autodiff works with it.) Linear interpolation is pretty straightforward, after all.

Also have a think about using the tstops argument. If you’re using linear interpolation then your vector field has kinks (derivative discontinuities) wrt time, and the solver will have a slightly easier time of it if it doesn’t have to discover these for itelf.


Yeah no worries, feel free to bump it quicker than that. Sometimes threads just get lost in the text wave. Patrick’s answer is a good one. I’d also mention that GitHub - PumasAI/DataInterpolations.jl: A library of data interpolation and smoothing functions has a bunch of derivative overloads, which can be important for interpolations since straight differentiation of many interpolants is not necessarily numerically stable (I’m looking at you Lagrange interpolation). But in your case, it would be best to just ignore the derivative of that term.

I usually tend to do it via wrapping it in a quick @nograd function, but I think they all lead to the same pond. Maybe @dhairyagandhi96 could enlighten us with how they differ.

Interpolations.jl recently got Zygote integration in https://github.com/JuliaMath/Interpolations.jl/pull/414, that may be helpful to try out here

But in this case, is the fact that it needs to exist just an issue of missing DCE?

Dear @patrick-kidger, @ChrisRackauckas and @dhairyagandhi96,

thank you so much for your replies! For now I just tried to implement the “hack” suggested by @patrick-kidger and an initial loss is returned, but then Julia throws a new error stating that

" LoadError: ArgumentError: tuple must be non-empty in expression starting at…" and the location in my newly adapted file (simply replaced dudt). The line of code that seems to stop working is

res0 = DiffEqFlux.sciml_train(loss, p_model, ADAM(0.01), maxiters = 100)

Which confuses me quite a bit. @patrick-kidger you mentioned you had the code running for a few iterations so I assume you didn’t encounter this problem?

Update again. That’s a hiccup from yesterday’s ChainRules update stuff (oops…)

Yeah suspected so… I used Pkg.update() but it still doesn’t work… Should I wait and try again in a few days?

]st? It should be good now.

Hmmm - I restarted Julia and tried it agian, now im throwing compiling errors for the DiffEqFlux package right of the start.

ERROR: LoadError: Failed to precompile DiffEqFlux [aae7a2af-3d4f-5e19-a356-7da93b79d9d0] to C:\Users\grego\.julia\compiled\v1.5\DiffEqFlux\BdO4p_6n1vG.ji

I feel like that at this point it maybe simpler to just reinstall Julia clean…

What is it saying in that?

I wonder if the issue is v1.6. Additions of some compiler tools (Cassette) might be involved in something here.

Here’s the full error message (with my file path removed :slight_smile: )

LoadError: Failed to precompile DiffEqFlux [aae7a2af-3d4f-5e19-a356-7da93b79d9d0] to C:\Users\grego.julia\compiled\v1.5\DiffEqFlux\BdO4p_6n1vG.ji.

in expression starting at C:\Users\grego\ […] \min_working_example.jl:3

error(::String) at error.jl:33

compilecache(::Base.PkgId, ::String) at loading.jl:1305

_require(::Base.PkgId) at loading.jl:1030

require(::Base.PkgId) at loading.jl:928

require(::Module, ::Symbol) at loading.jl:923

include_string(::Function, ::Module, ::String, ::String) at loading.jl:1088

Try updating again. I found I made a version bounding issue that would cause a problem in v1.5 (we really need a new LTS :sweat_smile:).

I’m afraid it didn’t workt out, but at least the error message changed to some extend/got longer…

[ Info: Precompiling DiffEqFlux [aae7a2af-3d4f-5e19-a356-7da93b79d9d0]  
ERROR: LoadError: LoadError: UndefVarError: HasBranchingCtx not defined
 [1] getproperty(::Module, ::Symbol) at .\Base.jl:26
 [2] top-level scope at C:\Users\grego\.julia\packages\DiffEqFlux\W9Cwj\src\fast_layers.jl:195
 [3] include(::Function, ::Module, ::String) at .\Base.jl:380
 [4] include at .\Base.jl:368 [inlined]
 [5] include(::String) at C:\Users\grego\.julia\packages\DiffEqFlux\W9Cwj\src\DiffEqFlux.jl:1
 [6] top-level scope at C:\Users\grego\.julia\packages\DiffEqFlux\W9Cwj\src\DiffEqFlux.jl:84
 [7] include(::Function, ::Module, ::String) at .\Base.jl:380
 [8] include(::Module, ::String) at .\Base.jl:368
 [9] top-level scope at none:2
 [10] eval at .\boot.jl:331 [inlined]
 [11] eval(::Expr) at .\client.jl:467
 [12] top-level scope at .\none:3
in expression starting at C:\Users\grego\.julia\packages\DiffEqFlux\W9Cwj\src\fast_layers.jl:195
in expression starting at C:\Users\grego\.julia\packages\DiffEqFlux\W9Cwj\src\DiffEqFlux.jl:84
ERROR: LoadError: Failed to precompile DiffEqFlux [aae7a2af-3d4f-5e19-a356-7da93b79d9d0] to C:\Users\grego\.julia\compiled\v1.5\DiffEqFlux\BdO4p_6n1vG.ji.
 [1] error(::String) at .\error.jl:33
 [2] compilecache(::Base.PkgId, ::String) at .\loading.jl:1305
 [3] _require(::Base.PkgId) at .\loading.jl:1030
 [4] require(::Base.PkgId) at .\loading.jl:928
 [5] require(::Module, ::Symbol) at .\loading.jl:923
 [6] include_string(::Function, ::Module, ::String, ::String) at .\loading.jl:1088
in expression starting at C:\Users\grego\ [...] \min_working_example.jl:3

Could you please post ]st -m? That’s going to be a lot more helpful than going error-by-error. My guess is you don’t have DiffEqSensitivity v6.52.0, but that would be weird since there’s no clear bounds that would cause that?

Yes sure! Please find the console output below:

Status `C:\Users\grego\.julia\environments\v1.5\Manifest.toml`
  [c3fe647b] AbstractAlgebra v0.17.1
  [621f4979] AbstractFFTs v1.0.1
  [1520ce14] AbstractTrees v0.3.4
  [79e6a3ab] Adapt v3.3.1
  [27a7e980] Animations v0.4.1
  [ec485272] ArnoldiMethod v0.1.0
  [4fba245c] ArrayInterface v3.1.17
  [4c555306] ArrayLayouts v0.7.2
  [56f22d72] Artifacts v1.3.0
  [bf4720bc] AssetRegistry v0.1.0
  [c52e3926] Atom v0.12.30
  [13072b0f] AxisAlgorithms v1.0.0
  [ab4f0b2a] BFloat16s v0.1.0
  [aae01518] BandedMatrices v0.16.9
  [a74b3585] Blosc v0.7.0
  [0b7ba130] Blosc_jll v1.21.0+0
  [764a87c0] BoundaryValueDiffEq v2.7.1
  [6e34b625] Bzip2_jll v1.0.6+5
  [fa961155] CEnum v0.4.1
  [00ebfdb7] CSTParser v2.5.0
  [052768ef] CUDA v2.4.3
  [83423d85] Cairo_jll v1.16.0+6
  [7057c7e9] Cassette v0.3.7
  [082447d4] ChainRules v0.7.70
  [d360d2e6] ChainRulesCore v0.9.45
  [53a63b46] CodeTools v0.7.1
  [da1fd8a2] CodeTracking v1.0.5
  [944b1d66] CodecZlib v0.7.0
  [a2cac450] ColorBrewer v0.4.0
  [35d6a980] ColorSchemes v3.12.1
  [3da002f7] ColorTypes v0.10.12
  [c3611d14] ColorVectorSpace v0.8.7
  [5ae59095] Colors v0.12.8
  [861a8166] Combinatorics v1.0.2
  [a80b9123] CommonMark v0.6.4
  [38540f10] CommonSolve v0.2.0
  [bbf7d656] CommonSubexpressions v0.3.0
  [34da2185] Compat v3.31.0
  [e66e0078] CompilerSupportLibraries_jll v0.3.4+0
  [b152e2b5] CompositeTypes v0.1.2
  [88cd18e8] ConsoleProgressMonitor v0.1.2
  [187b0558] ConstructionBase v1.3.0
  [d38c429a] Contour v0.5.7
  [a8cc5b0e] Crayons v4.0.4
  [9a962f9c] DataAPI v1.6.0
  [82cc6244] DataInterpolations v3.4.1
  [864edb3b] DataStructures v0.18.9
  [e2d170a0] DataValueInterfaces v1.0.0
  [bcd4f6db] DelayDiffEq v5.31.1
  [2b5f629d] DiffEqBase v6.66.0
  [459566f4] DiffEqCallbacks v2.16.1
  [5a0ffddc] DiffEqFinancial v2.4.0
  [aae7a2af] DiffEqFlux v1.40.0
  [c894b116] DiffEqJump v6.14.2
  [77a26b50] DiffEqNoiseProcess v5.8.0
  [055956cb] DiffEqPhysics v3.9.0
  [41bf760c] DiffEqSensitivity v6.49.1
  [163ba53b] DiffResults v1.0.3
  [b552c78f] DiffRules v1.0.2
  [0c46a032] DifferentialEquations v6.17.1
  [c619ae07] DimensionalPlotRecipes v1.2.0
  [b4f34e82] Distances v0.9.2
  [31c24e10] Distributions v0.24.18
  [ced4e74d] DistributionsAD v0.6.28
  [33d173f1] DocSeeker v0.4.3
  [ffbed154] DocStringExtensions v0.8.5
  [e30172f5] Documenter v0.26.3
  [5b8099bc] DomainSets v0.5.2
  [5ae413db] EarCut_jll v2.1.5+1
  [da5c29d0] EllipsisNotation v1.1.0
  [2e619515] Expat_jll v2.2.7+6
  [d4d017d3] ExponentialUtilities v1.8.4
  [e2ba6199] ExprTools v0.1.3
  [c87230d0] FFMPEG v0.4.1
  [b22a6f82] FFMPEG_jll v4.3.1+4
  [7a1cc6ca] FFTW v1.3.2
  [f5851436] FFTW_jll v3.3.9+7
  [7034ab61] FastBroadcast v0.1.8
  [9aa1b823] FastClosures v0.3.2
  [5789e2e9] FileIO v1.10.1
  [1a297f60] FillArrays v0.11.7
  [6a86dc24] FiniteDiff v2.8.0
  [53c48c17] FixedPointNumbers v0.8.4
  [08572546] FlameGraphs v0.2.5
  [587475ba] Flux v0.12.1
  [a3f928ae] Fontconfig_jll v2.13.1+14
  [59287772] Formatting v0.4.2
  [f6369f11] ForwardDiff v0.10.18
  [b38be410] FreeType v4.0.0
  [d7e528f0] FreeType2_jll v2.10.1+5
  [663a7486] FreeTypeAbstraction v0.9.1
  [559328eb] FriBidi_jll v1.0.5+6
  [069b7b12] FunctionWrappers v1.1.2
  [de31a74c] FunctionalCollections v0.5.0
  [d9f16b24] Functors v0.2.1
  [fb4132e2] FuzzyCompletions v0.4.1
  [0656b61e] GLFW_jll v3.3.4+0
  [0c68f7d7] GPUArrays v6.4.1
  [61eb1bfa] GPUCompiler v0.8.3
  [28b8d3ca] GR v0.57.5
  [d2c73de3] GR_jll v0.57.3+0
  [a75be94c] GalacticOptim v1.3.0
  [5c1252a2] GeometryBasics v0.3.10
  [78b55507] Gettext_jll v0.20.1+7
  [7746bdde] Glib_jll v2.59.0+4
  [af5da776] GlobalSensitivity v1.1.0
  [a2bd30eb] Graphics v1.1.0
  [3955a311] GridLayoutBase v0.5.4
  [42e2da0e] Grisu v1.0.2
  [f67ccb44] HDF5 v0.15.5
  [0234f1f7] HDF5_jll v1.12.0+1
  [cd3eb016] HTTP v0.8.19
  [9fb69e20] Hiccup v0.2.2
  [0e44f5e4] Hwloc v2.0.0
  [e33a78d0] Hwloc_jll v2.4.1+0
  [b5f81e59] IOCapture v0.1.1
  [7869d1d1] IRTools v0.4.3
  [615f187c] IfElse v0.1.0
  [a09fc81d] ImageCore v0.8.22
  [82e4d734] ImageIO v0.4.1
  [9b13fd28] IndirectArrays v0.5.1
  [d25df0c9] Inflate v0.1.2
  [83e8ac13] IniFile v0.5.0
  [1d5cc7b8] IntelOpenMP_jll v2018.0.3+2
  [a98d9a8b] Interpolations v0.13.2
  [8197267c] IntervalSets v0.5.3
  [f1662d9f] Isoband v0.1.1
  [c8e1da08] IterTools v1.3.0
  [42fd0dbc] IterativeSolvers v0.9.1
  [82899510] IteratorInterfaceExtensions v1.0.0
  [692b3bcd] JLLWrappers v1.3.0
  [682c06a0] JSON v0.21.1
  [aacddb02] JpegTurbo_jll v2.0.1+3
  [98e50ef6] JuliaFormatter v0.12.3
  [aa1ae85d] JuliaInterpreter v0.8.18
  [e5e0dc1b] Juno v0.8.4
  [5ab0869b] KernelDensity v0.6.3
  [c1c5ebd0] LAME_jll v3.100.0+3
  [929cbde3] LLVM v3.9.0
  [7c4cb9fa] LNR v0.2.1
  [dd4b983a] LZO_jll v2.10.0+3
  [b964fa9f] LaTeXStrings v1.2.1
  [2ee39098] LabelledArrays v1.6.1
  [23fbe1c1] Latexify v0.15.6
  [a5e1c1ea] LatinHypercubeSampling v1.8.0
  [73f95e8e] LatticeRules v0.0.1
  [50d2b5c4] Lazy v0.15.1
  [4af54fe1] LazyArtifacts v1.3.0
  [1d6d02ad] LeftChildRightSiblingTrees v0.1.2
  [deac9b47] LibCURL_jll v7.70.0+2
  [29816b5a] LibSSH2_jll v1.9.0+3
  [dd192d2f] LibVPX_jll v1.9.0+1
  [e9f186c6] Libffi_jll v3.2.1+4
  [d4300ac3] Libgcrypt_jll v1.8.5+4
  [7e76a0d4] Libglvnd_jll v1.3.0+3
  [7add5ba3] Libgpg_error_jll v1.36.0+3
  [94ce4f54] Libiconv_jll v1.16.0+8
  [4b2f31a3] Libmount_jll v2.34.0+3
  [89763e89] Libtiff_jll v4.1.0+2
  [38a345b3] Libuuid_jll v2.34.0+7
  [093fc24a] LightGraphs v1.3.5
  [d3d80556] LineSearches v7.1.1
  [2ab3a3ac] LogExpFunctions v0.2.4
  [e6f89c97] LoggingExtras v0.4.7
  [bdcacae8] LoopVectorization v0.12.45
  [5ced341a] Lz4_jll v1.9.2+2
  [d00139f3] METIS_jll v5.1.0+5
  [856f044c] MKL_jll v2021.1.1+1
  [1914dd2f] MacroTools v0.5.6
  [ee78f7c6] Makie v0.13.14
  [dbb5928d] MappedArrays v0.4.0
  [7eb4fadd] Match v1.1.0
  [739be429] MbedTLS v1.0.3
  [c8ffd9c3] MbedTLS_jll v2.16.8+1
  [442fdcdd] Measures v0.3.1
  [e89f7d12] Media v0.5.0
  [e1d29d7a] Missings v1.0.0
  [961ee093] ModelingToolkit v5.20.0
  [e94cdb99] MosaicViews v0.3.3
  [46d2c3a1] MuladdMacro v0.2.2
  [f9640e96] MultiScaleArrays v1.8.1
  [d41bc354] NLSolversBase v7.8.0
  [2774e3e8] NLsolve v4.5.1
  [872c559c] NNlib v0.7.19
  [77ba4419] NaNMath v0.3.5
  [f09324ee] Netpbm v1.0.1
  [8913a72c] NonlinearSolve v0.3.8
  [510215fc] Observables v0.3.3
  [6fe1bfb0] OffsetArrays v1.10.0
  [e7412a2a] Ogg_jll v1.3.4+2
  [4536629a] OpenBLAS_jll v0.3.9+5
  [458c3c95] OpenSSL_jll v1.1.1+6
  [efe28fd5] OpenSpecFun_jll v0.5.3+4
  [429524aa] Optim v1.3.0
  [91d4177d] Opus_jll v1.3.1+3
  [bac558e1] OrderedCollections v1.4.1
  [1dea7af3] OrdinaryDiffEq v5.59.1
  [2f80f16e] PCRE_jll v8.42.0+4
  [90014a1f] PDMats v0.11.1
  [f57f5aa1] PNGFiles v0.3.7
  [19eb6ba3] Packing v0.4.1
  [5432bcbf] PaddedViews v0.5.8
  [65888b18] ParameterizedFunctions v5.10.0
  [d96e819e] Parameters v0.12.2
  [69de0a69] Parsers v1.1.0
  [fa939f87] Pidfile v1.2.0
  [30392449] Pixman_jll v0.40.0+0
  [ccf2f8ad] PlotThemes v2.0.1
  [995b91a9] PlotUtils v1.0.10
  [91a5bcdd] Plots v1.16.5
  [e409e4f3] PoissonRandom v0.4.0
  [f517fe37] Polyester v0.3.1
  [647866c9] PolygonOps v0.1.1
  [85a6dd25] PositiveFactorizations v0.2.4
  [21216c6a] Preferences v1.2.2
  [33c8b6b6] ProgressLogging v0.1.4
  [92933f4c] ProgressMeter v1.7.1
  [ea2cea3b] Qt5Base_jll v5.15.2+0
  [1fd47b50] QuadGK v2.4.1
  [8a4e6c94] QuasiMonteCarlo v0.2.3
  [74087812] Random123 v1.4.2
  [fb686558] RandomExtensions v0.4.3
  [e6cf234a] RandomNumbers v1.4.0
  [c84ed2f1] Ratios v0.4.0
  [3cdcf5f2] RecipesBase v1.1.1
  [01d81517] RecipesPipeline v0.3.3
  [731186ca] RecursiveArrayTools v2.11.4
  [f2c3362d] RecursiveFactorization v0.1.12
  [189a3867] Reexport v1.1.0
  [ae029012] Requires v1.1.3
  [ae5879a3] ResettableStacks v1.1.0
  [37e2e3b7] ReverseDiff v1.9.0
  [79098fc4] Rmath v0.6.1
  [f50d1b31] Rmath_jll v0.2.2+2
  [7e49a35a] RuntimeGeneratedFunctions v0.5.2
  [476501e8] SLEEFPirates v0.6.22
  [1bc83da4] SafeTestsets v0.0.1
  [0bca4576] SciMLBase v1.13.6
  [6c6a2e73] Scratch v1.1.0
  [efcf1570] Setfield v0.7.0
  [992d4aef] Showoff v1.0.3
  [73760f76] SignedDistanceFields v0.4.0
  [699a6c99] SimpleTraits v0.9.3
  [ed01d8cd] Sobol v1.5.0
  [a2af1166] SortingAlgorithms v1.0.0
  [47a9eef4] SparseDiffTools v1.13.2
  [276daf66] SpecialFunctions v1.5.1
  [860ef19b] StableRNGs v1.0.0
  [cae243ae] StackViews v0.1.1
  [aedffcd0] Static v0.2.5
  [90137ffa] StaticArrays v1.2.4
  [82ae8749] StatsAPI v1.0.0
  [2913bbd2] StatsBase v0.33.8
  [4c63d2b9] StatsFuns v0.9.8
  [9672c7b4] SteadyStateDiffEq v1.6.4
  [789caeaf] StochasticDiffEq v6.35.0
  [7792a7ef] StrideArraysCore v0.1.13
  [88034a9c] StringDistances v0.10.0
  [09ab397b] StructArrays v0.4.2
  [bea87d4a] SuiteSparse_jll v5.4.0+9
  [c3572dad] Sundials v4.5.0
  [fb77eaff] Sundials_jll v5.2.0+1
  [d1185830] SymbolicUtils v0.11.3
  [0c5d862f] Symbolics v0.1.32
  [fa267f1f] TOML v1.0.3
  [3783bdb8] TableTraits v1.0.1
  [bd369af6] Tables v1.4.4
  [5d786b92] TerminalLoggers v0.1.3
  [8290d209] ThreadingUtilities v0.4.4
  [a759f4b9] TimerOutputs v0.5.9
  [0796e94c] Tokenize v0.5.17
  [9f7883ad] Tracker v0.2.16
  [3bb67fe8] TranscodingStreams v0.9.5
  [592b5752] Trapz v2.0.2
  [a2a6695c] TreeViews v0.3.0
  [30578b45] URIParser v0.4.1
  [3a884ed6] UnPack v1.0.2
  [1cfade01] UnicodeFun v0.4.1
  [1986cc42] Unitful v1.8.0
  [3d5dd08c] VectorizationBase v0.20.18
  [19fa3120] VertexSafeGraphs v0.1.2
  [a2964d1f] Wayland_jll v1.17.0+4
  [2381bf8a] Wayland_protocols_jll v1.18.0+4
  [0f1e0344] WebIO v0.8.15
  [104b5d7c] WebSockets v1.5.9
  [cc8bc4a8] Widgets v0.6.3
  [efce3f68] WoodburyMatrices v0.5.3
  [02c8fc9c] XML2_jll v2.9.10+3
  [aed1982a] XSLT_jll v1.1.33+4
  [4f6342f7] Xorg_libX11_jll v1.6.9+4
  [0c0b7dd1] Xorg_libXau_jll v1.0.9+4
  [935fb764] Xorg_libXcursor_jll v1.2.0+4
  [a3789734] Xorg_libXdmcp_jll v1.1.3+4
  [1082639a] Xorg_libXext_jll v1.3.4+4
  [d091e8ba] Xorg_libXfixes_jll v5.0.3+4
  [a51aa0fd] Xorg_libXi_jll v1.7.10+4
  [d1454406] Xorg_libXinerama_jll v1.1.4+4
  [ec84b674] Xorg_libXrandr_jll v1.5.2+4
  [ea2f1a96] Xorg_libXrender_jll v0.9.10+4
  [14d82f49] Xorg_libpthread_stubs_jll v0.1.0+3
  [c7cfdc94] Xorg_libxcb_jll v1.13.0+3
  [cc61e674] Xorg_libxkbfile_jll v1.1.0+4
  [12413925] Xorg_xcb_util_image_jll v0.4.0+1
  [2def613f] Xorg_xcb_util_jll v0.4.0+1
  [975044d2] Xorg_xcb_util_keysyms_jll v0.4.0+1
  [0d47668e] Xorg_xcb_util_renderutil_jll v0.3.9+1
  [c22f9ab0] Xorg_xcb_util_wm_jll v0.4.1+1
  [35661453] Xorg_xkbcomp_jll v1.4.2+4
  [33bec58e] Xorg_xkeyboard_config_jll v2.27.0+4
  [c5fb5394] Xorg_xtrans_jll v1.4.0+3
  [a5390f91] ZipFile v0.9.3
  [83775a58] Zlib_jll v1.2.11+18
  [3161d3a3] Zstd_jll v1.4.8+0
  [e88e6eb3] Zygote v0.6.12
  [700de1a5] ZygoteRules v0.2.1
  [9a68df92] isoband_jll v0.2.2+0
  [0ac62f75] libass_jll v0.14.0+4
  [f638f0a6] libfdk_aac_jll v0.1.6+4
  [b53b4c65] libpng_jll v1.6.37+6
  [f27f6e37] libvorbis_jll v1.3.6+6
  [8e850ede] nghttp2_jll v1.40.0+2
  [1270edf5] x264_jll v2020.7.14+2
  [dfaa095f] x265_jll v3.0.0+3
  [d8fb68d0] xkbcommon_jll v0.9.1+5
  [2a0f44e3] Base64
  [ade2ca70] Dates
  [8bb1440f] DelimitedFiles
  [8ba89e20] Distributed
  [7b1f6079] FileWatching
  [9fa8497b] Future
  [b77e0a4c] InteractiveUtils
  [76f85450] LibGit2
  [8f399da3] Libdl
  [37e2e46d] LinearAlgebra
  [56ddb016] Logging
  [d6f4376e] Markdown
  [a63ad114] Mmap
  [44cfe95a] Pkg
  [de0858da] Printf
  [9abbd945] Profile
  [3fa0cd96] REPL
  [9a3f8284] Random
  [ea8e919c] SHA
  [9e88b42a] Serialization
  [1a1011a3] SharedArrays
  [6462fe0b] Sockets
  [2f01184e] SparseArrays
  [10745b16] Statistics
  [4607b0f0] SuiteSparse
  [8dfed614] Test
  [cf7118a7] UUIDs
  [4ec0a83e] Unicode