[Flux]Trouble running examples from the Flux Differentiable control problems blogpost

General Usage
1

Hey there,

Trying to replicate all the results and examples from the blogpost: https://fluxml.ai/2019/03/05/dp-vs-rl.html

These are the steps that I used, as described in the README. Zygote trows an error. Running Julia 1.1.1 on Linux, Ubuntu 18.04 LTS

doctoraat@doctoraat:~/Downloads/model-zoo-master/games/differentiable-programming/trebuchet$ ls
cuda  DDPG.jl  DiffRL.jl  Manifest.toml  Project.toml
doctoraat@doctoraat:~/Downloads/model-zoo-master/games/differentiable-programming/trebuchet$ julia
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.1.1 (2019-05-16)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

julia> using Pkg; Pkg.activate("."); Pkg.instantiate()
  Updating registry at `~/.julia/registries/General`
  Updating git-repo `https://github.com/JuliaRegistries/General.git`

julia> include("DiffRL.jl")
[ Info: Recompiling stale cache file /home/doctoraat/.julia/compiled/v1.1/Flux/QdkVy.ji for Flux [587475ba-b771-5e3f-ad9e-33799f191a9c]
[ Info: Recompiling stale cache file /home/doctoraat/.julia/compiled/v1.1/Trebuchet/nIiFf.ji for Trebuchet [98b73d46-197d-11e9-11eb-69a6ff759d3a]
[ Info: Recompiling stale cache file /home/doctoraat/.julia/compiled/v1.1/Zygote/4kbLI.ji for Zygote [e88e6eb3-aa80-5325-afca-941959d7151f]
ERROR: LoadError: LoadError: UndefVarError: Argument not defined
Stacktrace:
 [1] include at ./boot.jl:326 [inlined]
 [2] include_relative(::Module, ::String) at ./loading.jl:1038
 [3] include at ./sysimg.jl:29 [inlined]
 [4] include(::String) at /home/doctoraat/.julia/packages/Zygote/1Urp2/src/Zygote.jl:1
 [5] top-level scope at none:0
 [6] include at ./boot.jl:326 [inlined]
 [7] include_relative(::Module, ::String) at ./loading.jl:1038
 [8] include(::Module, ::String) at ./sysimg.jl:29
 [9] top-level scope at none:2
 [10] eval at ./boot.jl:328 [inlined]
 [11] eval(::Expr) at ./client.jl:404
 [12] top-level scope at ./none:3
in expression starting at /home/doctoraat/.julia/packages/Zygote/1Urp2/src/compiler/reverse.jl:1
in expression starting at /home/doctoraat/.julia/packages/Zygote/1Urp2/src/Zygote.jl:24
ERROR: LoadError: Failed to precompile Zygote [e88e6eb3-aa80-5325-afca-941959d7151f] to /home/doctoraat/.julia/compiled/v1.1/Zygote/4kbLI.ji.
Stacktrace:
 [1] error(::String) at ./error.jl:33
 [2] compilecache(::Base.PkgId, ::String) at ./loading.jl:1197
 [3] _require(::Base.PkgId) at ./loading.jl:960
 [4] require(::Base.PkgId) at ./loading.jl:858
 [5] require(::Module, ::Symbol) at ./loading.jl:853
 [6] include at ./boot.jl:326 [inlined]
 [7] include_relative(::Module, ::String) at ./loading.jl:1038
 [8] include(::Module, ::String) at ./sysimg.jl:29
 [9] include(::String) at ./client.jl:403
 [10] top-level scope at none:0
in expression starting at /home/doctoraat/Downloads/model-zoo-master/games/differentiable-programming/trebuchet/DiffRL.jl:2

Any help to replicate the results would be much appreciated, highly interested in this topic.

Bonus question: is there code available to also generate the nice graphics from the blog post?

2

I am on Julia v1.1.0 on OSX and I don’t have this issue. I do have a seperate issue though that you will probably face once you resolve this. For me, the catch in loss(wind, target) is always getting called during training. After removing it I get the following

MethodError: no method matching promote_tspan(::Tracker.TrackedTuple{Tuple{Float64,Float64}})
Closest candidates are:
  promote_tspan(!Matched::Nothing) at /Users/me/.julia/packages/DiffEqBase/3gIPB/src/problems/problem_utils.jl:13
  promote_tspan(!Matched::Tuple{T,T}) where T at /Users/me/.julia/packages/DiffEqBase/3gIPB/src/problems/problem_utils.jl:7
  promote_tspan(!Matched::Tuple{T1,T2}) where {T1, T2} at /Users/me/.julia/packages/DiffEqBase/3gIPB/src/problems/problem_utils.jl:9
  ...
(::getfield(DiffEqBase, Symbol("##call#297#300")))(::Nothing, ::Type{DiffEqBase.ODEProblem{true,tType,isinplace,P,F,C,PT} where PT where C where F where P where isinplace where tType}, ::DiffEqBase.ODEFunction{true,typeof(Trebuchet.stage1!),LinearAlgebra.UniformScaling{Bool},Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing}, ::TrackedArray{…,Array{Float64,1}}, ::Tracker.TrackedTuple{Tuple{Float64,Float64}}, ::TrebuchetState, ::DiffEqBase.StandardODEProblem) at ode_problems.jl:36
DiffEqBase.ODEProblem{true,tType,isinplace,P,F,C,PT} where PT where C where F where P where isinplace where tType(::DiffEqBase.ODEFunction{true,typeof(Trebuchet.stage1!),LinearAlgebra.UniformScaling{Bool},Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing}, ::TrackedArray{…,Array{Float64,1}}, ::Tracker.TrackedTuple{Tuple{Float64,Float64}}, ::TrebuchetState, ::DiffEqBase.StandardODEProblem) at ode_problems.jl:36
DiffEqBase.ODEProblem{true,tType,isinplace,P,F,C,PT} where PT where C where F where P where isinplace where tType(::DiffEqBase.ODEFunction{true,typeof(Trebuchet.stage1!),LinearAlgebra.UniformScaling{Bool},Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing}, ::TrackedArray{…,Array{Float64,1}}, ::Tracker.TrackedTuple{Tuple{Float64,Float64}}, ::TrebuchetState) at ode_problems.jl:36
#ODEProblem#303(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::Type, ::DiffEqBase.ODEFunction{true,typeof(Trebuchet.stage1!),LinearAlgebra.UniformScaling{Bool},Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing}, ::TrackedArray{…,Array{Float64,1}}, ::Tracker.TrackedTuple{Tuple{Float64,Float64}}, ::TrebuchetState) at ode_problems.jl:62
DiffEqBase.ODEProblem(::DiffEqBase.ODEFunction{true,typeof(Trebuchet.stage1!),LinearAlgebra.UniformScaling{Bool},Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing}, ::TrackedArray{…,Array{Float64,1}}, ::Tracker.TrackedTuple{Tuple{Float64,Float64}}, ::TrebuchetState) at ode_problems.jl:62
#ODEProblem#304(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::Type, ::Function, ::TrackedArray{…,Array{Float64,1}}, ::Tracker.TrackedTuple{Tuple{Float64,Float64}}, ::TrebuchetState) at ode_problems.jl:66
DiffEqBase.ODEProblem(::Function, ::TrackedArray{…,Array{Float64,1}}, ::Tracker.TrackedTuple{Tuple{Float64,Float64}}, ::TrebuchetState) at ode_problems.jl:66
simulate_(::TrebuchetState, ::Float64, ::Val{:Ground}) at simulate.jl:27
simulate_(::TrebuchetState, ::Float64) at simulate.jl:15
simulate(::TrebuchetState) at simulate.jl:7
shoot(::Tuple{Tracker.TrackedReal{Float64},Tracker.TrackedReal{Float64},Tracker.TrackedReal{Float64}}) at utils.jl:9
shoot(::Tracker.TrackedReal{Float64}, ::Tracker.TrackedReal{Float64}, ::Tracker.TrackedReal{Float64}) at DiffRL.jl:9
#59 at DiffRL.jl:12 [inlined]
forwarddiff at forward.jl:98 [inlined]
shoot(::TrackedArray{…,Array{Float64,1}}) at DiffRL.jl:12
distance(::Float64, ::Float64) at DiffRL.jl:30
loss(::Float64, ::Float64) at DiffRL.jl:35
#14 at train.jl:72 [inlined]
gradient_(::getfield(Flux.Optimise, Symbol("##14#20")){typeof(loss)}, ::Tracker.Params) at back.jl:97
#gradient#24(::Bool, ::Function, ::Function, ::Tracker.Params) at back.jl:164
gradient at back.jl:164 [inlined]
macro expansion at train.jl:71 [inlined]
macro expansion at progress.jl:119 [inlined]
#train!#12(::getfield(Flux, Symbol("#throttled#18")){getfield(Flux, Symbol("##throttled#10#14")){Bool,Bool,getfield(Main, Symbol("##65#66")),Int64}}, ::Function, ::Function, ::Tracker.Params, ::Base.Generator{UnitRange{Int64},getfield(Main, Symbol("##63#64"))}, ::ADAM) at train.jl:69
(::getfield(Flux.Optimise, Symbol("#kw##train!")))(::NamedTuple{(:cb,),Tuple{getfield(Flux, Symbol("#throttled#18")){getfield(Flux, Symbol("##throttled#10#14")){Bool,Bool,getfield(Main, Symbol("##65#66")),Int64}}}}, ::typeof(Flux.Optimise.train!), ::Function, ::Tracker.Params, ::Base.Generator{UnitRange{Int64},getfield(Main, Symbol("##63#64"))}, ::ADAM) at none:0
top-level scope at none:0

Use the following for making the animations. There’s not much documentation, but this may help https://github.com/MikeInnes/Trebuchet.jl

t = TrebuchetState(;wind_speed=3.0, release_angle=deg2rad(45), weight=200)
simulate(t)

if @isdefined(Blink)
  body!(Blink.Window(), visualise(t))
else
  visualise(t)
end
3

See https://github.com/FluxML/model-zoo/issues/161

This solutions presented removes the Zygote dependency. It fixed the issues I had as well.