Zygote error. "Only reference types can be differentiated with `Params`."

Specific Domains Machine Learning
1

I have been updating the Lotka-Volterra example in the model zoo. The call to train! fails with an error in Zygote src/compiler/interface.jl line 129 with the error in the title.

The reason for the error is a mystery to me.

My code is at https://github.com/Emmanuel-R8/model-zoo/blob/update-diffeq/contrib/diffeq/lotka-volterra.jl. Call to train! is on the very last line.

Has anybody seen that before?

2

That’s really annoying. I already put in 2 PRs to update that blog post and they somehow got reverted:

https://github.com/JuliaLang/www.julialang.org/pull/583
https://github.com/JuliaLang/www.julialang.org/pull/582

3

Should be updated now, but the TeX needs help: Update DiffEqFlux blog post by ChrisRackauckas · Pull Request #736 · JuliaLang/www.julialang.org · GitHub

4

Thanks Chris.

Looking at your diifs, I got the error to go away. I had forgotten to wrap the parameters using params = Flux.params(p).

This brings 2 points:

  • the error message is not really informative (to a novice like me);
  • I had read Flux.train!() to debug the error. The very first line is ps = Params(ps) and I had concluded the wrapping was not necessary before calling the function. Clearly I am still missing something.
5

The Flux API can be a bit difficult to use because of the globals which is why we recommend a sciml_train workflow these days, as demonstrated in the README: https://github.com/SciML/DiffEqFlux.jl

6

Can you clarify what you mean by “globals” in the Flux API?

7

Flux uses global references to “know” what the “parameters” are everywhere. DiffEqFlux’s sciml_train has a vector p, and you use the values in p, and those values get updated. Making everything explicit can make things more clear IMO.

8

Are you referring to the variables passed to params()?

I agree you are right that explicitly passing things is more clear and reliable,
but the params() scheme has the advantage I think that it can differentiate
through code that was not written for Flux.

9

So can explicit parameter setups. In fact, it’s probably moreso for things that are explicit in parameters, because lots of libraries want you to pass the values you use.