How can I auto differentiate the function cd with respect to p? using ForwardDiff using NLsolve function cd(; p=[1.0, 1.0], i=100) function sys(sto, x; p=p, i=i) x1 = x[1] x2 = x[2] p1 = p[1] p2 = p[2] sto[1] = x1/p2 - x2/p1 sto[2] = i - p1*x1 - …

But isn’t the error I’m getting because at some point something gets converted Float64 so that ForwardDiff loses track of the derivatives? I was hoping to simply fix that, but I can’t spot where the problem happens.

[image] amrods: I was hoping to simply fix that Geneally, you won’t be able to AD through an iterative solver. There is no “simple fix”, other than defining the derivative from the equation(s) that implicitly characterize the solution, as suggested by @kristoffer.carlsson .

But doesn’t AD handle differentiation of algorithms (except when they are not written in Julia)? I thought that Optim outsources something to a library outside Julia, and so I though could write a version of some optimization algorithm in pure Julia and get away with it, no?

If I am not misunderstanding your question: In theory you can differentiate an iterative algorithm. For example, I think a lot of functions yielding trandendentials and irrationals might end up using an iterative algorithm (Newton Raphson, the Babylonian method, etc) depending on the precision. So…

Yes, I understand that. I guess I was surprised that ForwardDiff throws an error. The issue still stands: suppose I am not worried about efficiency, what can I do to get that differentiation to work? About registering the rule, I don’t know how to do it. I am more of a user than a developer of Juli…

[image] jlperla: In theory you can differentiate an iterative algorithm. This, unfortunately, isn’t true in general. At least not in the sense of being able to blindly AD through any iterative algorithm. As an example, consider bisection to find x such that x + a = 0, from a bracket of floor(…

I know very little about AD, but my impression is that you don’t have to be a “developer” to define rules, you just use ChainRulesCore and write a rule like you would write another function (or maybe a good comparison is a plot recipe for Plots.jl), see FAQ · ChainRules ?

Yes, I was looking at that, but it seems to me that I have to use DiffRules.jl, because I think that’s what ForwardDiff.jl uses.

That might be, and apologies if I’ve added to the confusion, my point was rather that you don’t need to develop a package (potentially involving tests, documentation, GitHub CI etc etc) but that writing a rule isn’t much different from writing a regular function in your script as far as I understand…

[image] nilshg: but that writing a rule isn’t much different from writing a regular function in your script as far as I understand. ForwardDiff doesn’t really have any vector rules so there isn’t much in place to easily do this. In order to get this correct you need to propagate the derivativ…

How to autodifferentiate the results of NLsolve?

General Usage

rveltz March 1, 2023, 7:14am 22

There is also ImplicitDifferentiation.jl which nicely takes care of registering the rules. It works with Zygote but support for ForwardDiff was recently added. See also this discourse post

Topic		Replies	Views
How to implement AD for this function? General Usage differentiation , autodiff	9	700	May 7, 2021
Autodiff-ing a function defined by the result of Roots.find_zero fails General Usage roots , forwarddiff , autodiff	12	868	September 29, 2022
Differentiating through optimization problem New to Julia question , autodiff , enzyme	17	1231	May 20, 2024
Obtain jacobian used by NLsolve General Usage question , nlsolve	2	355	July 12, 2023
Method Error for ForwardDiff.hessian when ForwardDiff.Jacobian and ForwardDiff.gradient works perfectly fine. How to resolve? New to Julia question , forwarddiff	18	483	June 7, 2024

How to autodifferentiate the results of NLsolve?

Related topics