Zygote and TimerOutputs, derivative of timed function

Specific Domains Machine Learning
1

Hi,
sometimes I want to benchmark my code on my neural networks, especially their forward pass, using TimerOutputs.jl, but I don’t want to comment out all this when calculating gradients.
Currently it’s crashing, not being able to derivate through the timing macro.
Is there a way to define adjoint for this macro so the macro would effectively be ignored during backprop and just delegated gradient to the timed function?

Thanks.

2

For future readers: this is exactly what Zygote.ignore was designed for.

2 Likes
3

Since this got a bit of activity recently, ChainRulesCore.ignore_derivatives is a better, lighter and AD-agnostic option now.

2 Likes
4

Thanks, so the ignore_derivatives is the AD agnostic version of Zygote.ignore and can be used in the same way? Good to know, thanks.