[Flux] Issue while computing Hessian MethodError: objects of type Tracker.Grads are not callable

Specific Domains Machine Learning
1

I’m new to Flux and I’m trying to compute the hessian.
Here’s the code:

grad = Flux.gradient(() -> loss(x, y), Flux.params(model))
hess = Flux.gradient(grad, Flux.params(model))

But when running the line to compute hessian, it’s throwing the following error
MethodError: objects of type Tracker.Grads are not callable

2

Did you see the keyword nest or allownest = true?

1 Like
3

Nope. But I just tried adding nest=true to the grad and still it’s showing the same error.

4

You are trying to take the gradient of an object, not a function. Create a function that takes the gradient and take the gradient of that

grad = ()->Flux.gradient(() -> loss(x, y), Flux.params(model))
hess = Flux.gradient(grad, Flux.params(model))
1 Like
5

Your suggestion solved the issue partially. But after that I was getting the error outputs are not scalar

The bellow code is working, but I’m not sure if what I’m doing is correct.

grad() = Tracker.gradient(() -> loss(x, y), Flux.params(model), nest = true)
hess = forward(grad,Flux.params(model))

Thanks :slight_smile:

1 Like
6

You can also try the built in function
https://github.com/FluxML/Tracker.jl/blob/642bab12c9b37e7b29fd32925f63cdd9df107df0/src/back.jl#L180

7

Tried that and got the following error

ERROR: MethodError: no method matching jacobian(::TrackedArray{…,Array{Float64,2}}, ::Array{Float64,2}, ::Array{Float64,2})

8

@ChrisRackauckas, @MikeInnes Please help…

9

Hessian is the Jacobian of the gradient, and it’s already in the library:

https://github.com/FluxML/Tracker.jl/blob/67a6b35a1fec6b5f2ca0a5721a16feba95e58cdc/src/back.jl#L180

10

Got this error while trying that…

ERROR: MethodError: no method matching jacobian(::TrackedArray{…,Array{Float64,2}}, ::Array{Float64,2}, ::Array{Float64,2})
[/quote]

11

you need to be computing the Jacobian of a vector function on a vector.

12

But I think I have defined a generic function

loss(x,y) = -sum(y.*log.(model(x)) .- (1 .- y).*log.(1 .- model(x))) * 1 // size(y, 2)

This is my loss function.

I also tried the loss functions that came with Flux and the ones defined in the Model zoo. No luck!

13

@denizyuret Is it possible to compute hessian in Knet ? If so, how to do it?

Thanks in advance :slight_smile: