DiffEqFlux and Flux v0.10 (compatibility)

Specific Domains Machine Learning
1

I’d like to grind through the examples at the C. Rackaukas blog post on NN and ODE (DiffEqFlux.jl – A Julia Library for Neural Differential Equations). However, I’m struggling to get packages working together now.

If I add Flux first (v0.10) then trying to add DiffEqFlux gives the errors below (which I still find a little hard to parse).

If I add DiffEqFlux first then I can add Flux but it automatically reverts to v0.9 - I’m assuming this means DiffEqFlux needs some updates. If anybody can correct me, I’d be happy, and it would also be great to know when DiffEqFlux and Flux v0.10 will work together.

ERROR: Unsatisfiable requirements detected for package DiffEqBase [2b5f629d]:
 DiffEqBase [2b5f629d] log:
 ├─possible versions are: [3.13.2-3.13.3, 4.0.0-4.0.1, 4.1.0, 4.2.0, 4.3.0-4.3.1, 4.4.0, 4.5.0, 4.6.0, 4.7.0, 4.8.0, 4.9.0, 4.10.0-4.10.1, 4.11.0-4.11.1, 4.12.0, 4.13.0, 4.14.0-4.14.1, 4.15.0, 4.16.0, 4.17.0, 4.18.0, 4.19.0, 4.20.0-4.20.3, 4.21.0, 4.21.2-4.21.3, 4.22.0-4.22.2, 4.23.0, 4.23.2-4.23.4, 4.24.0-4.24.3, 4.25.0-4.25.1, 4.26.0-4.26.3, 4.27.0-4.27.1, 4.28.0-4.28.1, 4.29.0-4.29.2, 4.30.0-4.30.2, 4.31.0-4.31.2, 4.32.0, 5.0.0-5.0.1, 5.1.0, 5.2.0-5.2.3, 5.3.0-5.3.2, 5.4.0-5.4.1, 5.5.0-5.5.2, 5.6.0-5.6.4, 5.7.0, 5.8.0-5.8.1, 5.9.0, 5.10.0-5.10.3, 5.11.0-5.11.1, 5.12.0, 5.13.0, 5.14.0-5.14.2, 5.15.0, 5.16.0-5.16.5, 5.17.0-5.17.1, 5.18.0, 5.19.0, 5.20.0-5.20.1, 6.0.0, 6.1.0, 6.2.0-6.2.4, 6.3.0-6.3.6, 6.4.0-6.4.2, 6.5.0-6.5.1, 6.6.0, 6.7.0] or uninstalled
 ├─restricted by compatibility requirements with DifferentialEquations [0c46a032] to versions: [6.6.0, 6.7.0]
 │ └─DifferentialEquations [0c46a032] log:
 │   ├─possible versions are: [5.0.0, 5.1.0, 5.2.0-5.2.1, 5.3.0-5.3.1, 6.0.0, 6.1.0, 6.2.0, 6.3.0, 6.4.0, 6.6.0, 6.7.0, 6.8.0, 6.9.0] or uninstalled
 │   └─restricted to versions 6.9.0 by an explicit requirement, leaving only versions 6.9.0
 └─restricted by compatibility requirements with DiffEqFlux [aae7a2af] to versions: [3.13.2-3.13.3, 4.0.0-4.0.1, 4.1.0, 4.2.0, 4.3.0-4.3.1, 4.4.0, 4.5.0, 4.6.0, 4.7.0, 4.8.0, 4.9.0, 4.10.0-4.10.1, 4.11.0-4.11.1, 4.12.0, 4.13.0, 4.14.0-4.14.1, 4.15.0, 4.16.0, 4.17.0, 4.18.0, 4.19.0, 4.20.0-4.20.3, 4.21.0, 4.21.2-4.21.3, 4.22.0-4.22.2, 4.23.0, 4.23.2-4.23.4, 4.24.0-4.24.3, 4.25.0-4.25.1, 4.26.0-4.26.3, 4.27.0-4.27.1, 4.28.0-4.28.1, 4.29.0-4.29.2, 4.30.0-4.30.2, 4.31.0-4.31.2, 4.32.0, 5.0.0-5.0.1, 5.1.0, 5.2.0-5.2.3, 5.3.0-5.3.2, 5.4.0-5.4.1, 5.5.0-5.5.2, 5.6.0-5.6.4, 5.7.0, 5.8.0-5.8.1, 5.9.0, 5.10.0-5.10.3, 5.11.0-5.11.1, 5.12.0, 5.13.0, 5.14.0-5.14.2, 5.15.0, 5.16.0-5.16.5, 5.17.0-5.17.1, 5.18.0, 5.19.0, 5.20.0-5.20.1] — no versions left
   └─DiffEqFlux [aae7a2af] log:
     ├─possible versions are: [0.0.1, 0.1.0, 0.2.0, 0.3.0, 0.4.0, 0.5.0-0.5.2, 0.6.0-0.6.1, 0.7.0, 0.8.0-0.8.1] or uninstalled
     ├─restricted to versions * by an explicit requirement, leaving only versions [0.0.1, 0.1.0, 0.2.0, 0.3.0, 0.4.0, 0.5.0-0.5.2, 0.6.0-0.6.1, 0.7.0, 0.8.0-0.8.1]
     └─restricted by compatibility requirements with Flux [587475ba] to versions: [0.0.1, 0.1.0, 0.2.0, 0.3.0, 0.4.0] or uninstalled, leaving only versions: [0.0.1, 0.1.0, 0.2.0, 0.3.0, 0.4.0]
       └─Flux [587475ba] log:
         ├─possible versions are: [0.4.1, 0.5.0-0.5.4, 0.6.0-0.6.10, 0.7.0-0.7.3, 0.8.0-0.8.3, 0.9.0, 0.10.0] or uninstalled
         └─restricted to versions 0.10.0 by an explicit requirement, leaving only versions 0.10.0
2

It’s not compatible yet. It’s a huge change in Flux and we need to work on getting this updated. There are two WIP PRs on this:

https://github.com/JuliaDiffEq/DiffEqSensitivity.jl/pull/71

https://github.com/JuliaDiffEq/DiffEqFlux.jl/pull/52

It is one of our priorities right now.

3 Likes
3

Thanks, this is more or less what I expected.

Can you suggest a basic set up (I.e. versions) of Flux and other packages that should work with the code from the blog?

When it got the packages installed with Flux V0.9 I was getting error messages indicating problems with missing adjoints for a constructor only the first example with the Flux NN. I am away from my computer now so I can’t get the detailed messages until later (or test problems).

4

Everything should be fine on Flux v0.9. I was running it earlier today. Maybe the blog post has some old stuff: let me know if the README examples from https://github.com/JuliaDiffEq/DiffEqFlux.jl do better, and if they do, I’ll try to find where the two got desync’d.

@arnavsood mentioned something yesterday: what did you find?

5

I’ll give the README samples a try tomorrow morning and give an update then.

Thanks again!

6

Great news that big guys are working on this problem. I am looking forward for this to work.

Thanks for all the work.

Tomas

7

It is one of our priorities right now.

Sounds awesome!

8

I just looked at the code posted in the current README on GitHub, and I still get an error regarding adjoint. First I note that the blog implements predict_rd() using diffeq_rd, but the current README implements a function predict_adjoint using diffeq_adjoint.

Since the error message I got referenced a missing adjoint, I was hoping this would fix the issue. However, I get the same failure with a msg indicating a problem with missing adjoint.

First I’ll post my Pkg “status”, then I’ll copy in the error/stacktrace. I’m using Julia 1.3 with only a small number of packages installed:

(v1.3) pkg> st
    Status `~/.julia/environments/v1.3/Project.toml`
  [c52e3926] Atom v0.11.3
  [aae7a2af] DiffEqFlux v0.8.1
  [0c46a032] DifferentialEquations v6.9.0
  [587475ba] Flux v0.9.0
  [e5e0dc1b] Juno v0.7.2
  [91a5bcdd] Plots v0.28.2

Here’s the error info:

julia> Flux.train!(loss_adjoint, params, data, opt, cb = cb)
ERROR: Need an adjoint for constructor ODESolution{Float64,2,Array{Array{Float64,1},1},Nothing,Nothing,Array{Float64,1},Array{Array{Array{Float64,1},1},1},ODEProblem{Array{Float64,1},Tuple{Float64,Float64},true,Array{Float64,1},ODEFunction{true,typeof(lotka_volterra),LinearAlgebra.UniformScaling{Bool},Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing},Nothing,DiffEqBase.StandardODEProblem},Tsit5,OrdinaryDiffEq.InterpolationData{ODEFunction{true,typeof(lotka_volterra),LinearAlgebra.UniformScaling{Bool},Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing},Array{Array{Float64,1},1},Array{Float64,1},Array{Array{Array{Float64,1},1},1},OrdinaryDiffEq.Tsit5Cache{Array{Float64,1},Array{Float64,1},Array{Float64,1},OrdinaryDiffEq.Tsit5ConstantCache{Float64,Float64}}},DiffEqBase.DEStats}. Gradient is of type Array{Float64,2}
Stacktrace:
 [1] (::Zygote.Jnew{ODESolution{Float64,2,Array{Array{Float64,1},1},Nothing,Nothing,Array{Float64,1},Array{Array{Array{Float64,1},1},1},ODEProblem{Array{Float64,1},Tuple{Float64,Float64},true,Array{Float64,1},ODEFunction{true,typeof(lotka_volterra),LinearAlgebra.UniformScaling{Bool},Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing},Nothing,DiffEqBase.StandardODEProblem},Tsit5,OrdinaryDiffEq.InterpolationData{ODEFunction{true,typeof(lotka_volterra),LinearAlgebra.UniformScaling{Bool},Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing},Array{Array{Float64,1},1},Array{Float64,1},Array{Array{Array{Float64,1},1},1},OrdinaryDiffEq.Tsit5Cache{Array{Float64,1},Array{Float64,1},Array{Float64,1},OrdinaryDiffEq.Tsit5ConstantCache{Float64,Float64}}},DiffEqBase.DEStats},Nothing,false})(::Array{Float64,2}) at /Users/rick/.julia/packages/Zygote/8dVxG/src/lib/lib.jl:265
 [2] (::Zygote.var"#316#back#172"{Zygote.Jnew{ODESolution{Float64,2,Array{Array{Float64,1},1},Nothing,Nothing,Array{Float64,1},Array{Array{Array{Float64,1},1},1},ODEProblem{Array{Float64,1},Tuple{Float64,Float64},true,Array{Float64,1},ODEFunction{true,typeof(lotka_volterra),LinearAlgebra.UniformScaling{Bool},Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing},Nothing,DiffEqBase.StandardODEProblem},Tsit5,OrdinaryDiffEq.InterpolationData{ODEFunction{true,typeof(lotka_volterra),LinearAlgebra.UniformScaling{Bool},Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing},Array{Array{Float64,1},1},Array{Float64,1},Array{Array{Array{Float64,1},1},1},OrdinaryDiffEq.Tsit5Cache{Array{Float64,1},Array{Float64,1},Array{Float64,1},OrdinaryDiffEq.Tsit5ConstantCache{Float64,Float64}}},DiffEqBase.DEStats},Nothing,false}})(::Array{Float64,2}) at /Users/rick/.julia/packages/ZygoteRules/6nssF/src/adjoint.jl:49
 [3] ODESolution at /Users/rick/.julia/packages/DiffEqBase/DqkH4/src/solutions/ode_solutions.jl:2 [inlined]
 [4] (::typeof(∂(ODESolution{Float64,2,Array{Array{Float64,1},1},Nothing,Nothing,Array{Float64,1},Array{Array{Array{Float64,1},1},1},ODEProblem{Array{Float64,1},Tuple{Float64,Float64},true,Array{Float64,1},ODEFunction{true,typeof(lotka_volterra),LinearAlgebra.UniformScaling{Bool},Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing},Nothing,DiffEqBase.StandardODEProblem},Tsit5,OrdinaryDiffEq.InterpolationData{ODEFunction{true,typeof(lotka_volterra),LinearAlgebra.UniformScaling{Bool},Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing},Array{Array{Float64,1},1},Array{Float64,1},Array{Array{Array{Float64,1},1},1},OrdinaryDiffEq.Tsit5Cache{Array{Float64,1},Array{Float64,1},Array{Float64,1},OrdinaryDiffEq.Tsit5ConstantCache{Float64,Float64}}},DiffEqBase.DEStats})))(::Array{Float64,2}) at /Users/rick/.julia/packages/Zygote/8dVxG/src/compiler/interface2.jl:0
 [5] solution_new_retcode at /Users/rick/.julia/packages/DiffEqBase/DqkH4/src/solutions/ode_solutions.jl:93 [inlined]
 [6] (::typeof(∂(solution_new_retcode)))(::Array{Float64,2}) at /Users/rick/.julia/packages/Zygote/8dVxG/src/compiler/interface2.jl:0
 [7] solve! at /Users/rick/.julia/packages/OrdinaryDiffEq/BhP0W/src/solve.jl:372 [inlined]
 [8] (::typeof(∂(solve!)))(::Nothing) at /Users/rick/.julia/packages/Zygote/8dVxG/src/compiler/interface2.jl:0
 [9] #__solve#345 at /Users/rick/.julia/packages/OrdinaryDiffEq/BhP0W/src/solve.jl:5 [inlined]
 [10] (::typeof(∂(#__solve#345)))(::Array{Float64,2}) at /Users/rick/.julia/packages/Zygote/8dVxG/src/compiler/interface2.jl:0
 [11] #153 at /Users/rick/.julia/packages/Zygote/8dVxG/src/lib/lib.jl:142 [inlined]
 [12] #283#back at /Users/rick/.julia/packages/ZygoteRules/6nssF/src/adjoint.jl:49 [inlined]
 [13] #__solve at ./none:0 [inlined]
 [14] (::typeof(∂(#__solve)))(::Array{Float64,2}) at /Users/rick/.julia/packages/Zygote/8dVxG/src/compiler/interface2.jl:0
 [15] #153 at /Users/rick/.julia/packages/Zygote/8dVxG/src/lib/lib.jl:142 [inlined]
 [16] (::Zygote.var"#283#back#155"{Zygote.var"#153#154"{typeof(∂(#__solve)),Tuple{Tuple{Nothing,Nothing,Nothing},Tuple{Nothing}}}})(::Array{Float64,2}) at /Users/rick/.julia/packages/ZygoteRules/6nssF/src/adjoint.jl:49
 [17] #solve#381 at /Users/rick/.julia/packages/DiffEqBase/DqkH4/src/solve.jl:39 [inlined]
 [18] (::typeof(∂(#solve#381)))(::Array{Float64,2}) at /Users/rick/.julia/packages/Zygote/8dVxG/src/compiler/interface2.jl:0
 [19] (::Zygote.var"#153#154"{typeof(∂(#solve#381)),Tuple{Tuple{Nothing,Nothing,Nothing},Tuple{Nothing}}})(::Array{Float64,2}) at /Users/rick/.julia/packages/Zygote/8dVxG/src/lib/lib.jl:142
 [20] #283#back at /Users/rick/.julia/packages/ZygoteRules/6nssF/src/adjoint.jl:49 [inlined]
 [21] #solve at ./none:0 [inlined]
 [22] (::typeof(∂(#solve)))(::Array{Float64,2}) at /Users/rick/.julia/packages/Zygote/8dVxG/src/compiler/interface2.jl:0
 [23] #153 at /Users/rick/.julia/packages/Zygote/8dVxG/src/lib/lib.jl:142 [inlined]
 [24] (::Zygote.var"#283#back#155"{Zygote.var"#153#154"{typeof(∂(#solve)),Tuple{Tuple{Nothing,Nothing,Nothing},Tuple{Nothing}}}})(::Array{Float64,2}) at /Users/rick/.julia/packages/ZygoteRules/6nssF/src/adjoint.jl:49
 [25] #diffeq_adjoint#21 at /Users/rick/.julia/packages/DiffEqFlux/UcpUz/src/Flux/layers.jl:54 [inlined]
 [26] (::typeof(∂(#diffeq_adjoint#21)))(::Array{Float64,2}) at /Users/rick/.julia/packages/Zygote/8dVxG/src/compiler/interface2.jl:0
 [27] (::Zygote.var"#153#154"{typeof(∂(#diffeq_adjoint#21)),Tuple{NTuple{5,Nothing},Tuple{Nothing}}})(::Array{Float64,2}) at /Users/rick/.julia/packages/Zygote/8dVxG/src/lib/lib.jl:142
 [28] (::Zygote.var"#283#back#155"{Zygote.var"#153#154"{typeof(∂(#diffeq_adjoint#21)),Tuple{NTuple{5,Nothing},Tuple{Nothing}}}})(::Array{Float64,2}) at /Users/rick/.julia/packages/ZygoteRules/6nssF/src/adjoint.jl:49
 [29] #diffeq_adjoint at /Users/rick/.julia/packages/Zygote/8dVxG/src/compiler/interface2.jl:0 [inlined]
 [30] (::typeof(∂(#diffeq_adjoint)))(::Array{Float64,2}) at /Users/rick/.julia/packages/Zygote/8dVxG/src/compiler/interface2.jl:0
 [31] predict_adjoint at /Users/rick/src/julia_misc/ODE_from_blog/Tests_from_ReadMe.jl:23 [inlined]
 [32] (::typeof(∂(predict_adjoint)))(::Array{Float64,2}) at /Users/rick/.julia/packages/Zygote/8dVxG/src/compiler/interface2.jl:0
 [33] loss_adjoint at /Users/rick/src/julia_misc/ODE_from_blog/Tests_from_ReadMe.jl:26 [inlined]
 [34] (::typeof(∂(loss_adjoint)))(::Float64) at /Users/rick/.julia/packages/Zygote/8dVxG/src/compiler/interface2.jl:0
 [35] #153 at /Users/rick/.julia/packages/Zygote/8dVxG/src/lib/lib.jl:142 [inlined]
 [36] #283#back at /Users/rick/.julia/packages/ZygoteRules/6nssF/src/adjoint.jl:49 [inlined]
 [37] #14 at /Users/rick/.julia/packages/Flux/oX9Pi/src/optimise/train.jl:69 [inlined]
 [38] (::Zygote.var"#38#39"{Zygote.Params,Zygote.Context,typeof(∂(#14))})(::Float64) at /Users/rick/.julia/packages/Zygote/8dVxG/src/compiler/interface.jl:101
 [39] gradient(::Function, ::Zygote.Params) at /Users/rick/.julia/packages/Zygote/8dVxG/src/compiler/interface.jl:47
 [40] macro expansion at /Users/rick/.julia/packages/Flux/oX9Pi/src/optimise/train.jl:68 [inlined]
 [41] macro expansion at /Users/rick/.julia/packages/Juno/oLB1d/src/progress.jl:119 [inlined]
 [42] #train!#12(::var"#31#32", ::typeof(Flux.Optimise.train!), ::Function, ::Zygote.Params, ::Base.Iterators.Take{Base.Iterators.Repeated{Tuple{}}}, ::ADAM) at /Users/rick/.julia/packages/Flux/oX9Pi/src/optimise/train.jl:66
 [43] (::Flux.Optimise.var"#kw##train!")(::NamedTuple{(:cb,),Tuple{var"#31#32"}}, ::typeof(Flux.Optimise.train!), ::Function, ::Zygote.Params, ::Base.Iterators.Take{Base.Iterators.Repeated{Tuple{}}}, ::ADAM) at ./none:0
 [44] top-level scope at none:0