Please help with the very simplest 4-line Autodiff example: Flux0.9

Specific Domains Machine Learning
1

This is with Flux 0.9, Julia1.3

Meta comment: Julia looks amazing. I keep coming back to it, but everything I try does not seem to work. Bad beginner luck.

1a) This example I want to work. It should find [0,0] as the solution to the bottom of z=x^2 + y^2, but it give an error “back! was already used”

using Flux, Tracker
p = param([0.3, -0.7])
for i=1:5  
  global p
  theloss = sum(p .* p)
  Tracker.back!(theloss)   # `back!` was already used
  p = p - 0.05 * Tracker.grad(p)
  println(i," ",p)
end

1b) In this version gradient returns something that does not seem to be a gradient,
also nor a function that can be called to obtain a gradient. I also tried many other variations. But a solution to 1a) above is enough. This is just to show that other things were tried.

p = param([0.3, -0.7])
theloss(p) = sum(p .* p)
for i=1:5  
  global p
  p = p - 0.05 * gradient(theloss,p)
  println(i," ",p)
end
  1. Here, we want to get the gradient with respect to only the first parameter to the function, because the second parameter is not numeric.
f(x,path) = x^2   # path is some other variable, e.g. a string
gradient(f,1) 	# no method matching f with a single parameter
gradient(f,1,"foo")	# no method matching param(String)
2

My suggestion is, unless you really need to, just update to the Zygote-backed Flux 0.10. I find it’s much easier to debug.

2 Likes