ReverseDiff - differentiating with respect to some parameters but not others

Nathaniel
Sun, 17 Jul 2022 03:14:21 +0000
Machine Learning

I’m planning on training some neural networks using ReverseDiff.jl (which I think makes sense because I’ll be training lots of small networks on the CPU), but I have a basic question about how ReverseDiff is intended to be used.

In the examples we’re given a function with some parameters and shown how to differentiate with respect to all parameters. But the outputs of a neural network are a function of (1) the weights and biases, and (2) the inputs to the network. I only need to differentiate with respect to the weights and biases, not the inputs. So is there a way with ReverseDiff to differentiate with respect to some of the parameters to a function, but not others?

7 posts - 3 participants