# Automatic differentiation of an objective, which already uses AD tools
**URL:** https://discourse.julialang.org/t/automatic-differentiation-of-an-objective-which-already-uses-ad-tools/24864
**Category:** Optimization (Mathematical)
**Tags:** question
**Created:** [June 2, 2019, 9:17pm UTC](https://discourse.julialang.org/t/automatic-differentiation-of-an-objective-which-already-uses-ad-tools/24864 "2019-06-02T21:17:05Z")
**Posts on this page:** 4
**Page:** 1
### Author:  [@ArnoStrouwen](https://discourse.julialang.org/u/ArnoStrouwen)
#### Post date: [June 2, 2019, 9:17pm UTC](https://discourse.julialang.org/t/automatic-differentiation-of-an-objective-which-already-uses-ad-tools/24864/1 "2019-06-02T21:17:05Z")
I’m searching for parameters u to maximize a function obj(u). This objective function, however, depends on the derivatives of another function y(u,p), with regards to the parameters p.
An example:
```julia
abstract type S1 end
using ForwardDiff:Dual
function y(u,p1,p2)
p1_sen = Dual{S1}(p1,1.0,0.0)
p2_sen = Dual{S1}(p1,0.0,1.0)
n_u = size(u,2)
y = zeros(typeof(p1_sen*u[1,1]),n_u)
y[1] = 0.0
for i in 2:n_u
y[i] = y[i-1] + p1_sen*p2_sen*u[1,i] + p1_sen^2*u[2,i]
end
y
end
function obj(u)
p1 = 2.0
p2 = 3.0
sensitivities = y(u,p1,p2)
sensitivities[end].partials[1]*sensitivities[end].partials[2]
end
u = ones(2,100)
obj(u)
```
For efficient optimization I would like to calculate the gradient of obj(u) (preferably with reverse mode AD), and I’m not sure on how to do this for objectives already using other automatic differentiation tools.
I could calculate the Hessian of y(u,p), but this would be wasteful as not all second order derivatives of y are needed, only the mixed ones between u and p. In general there are many more parameters u than p.
Any advice would be appreciated.
---
### Author:  [@c42f](https://discourse.julialang.org/u/c42f)
#### Post date: [June 4, 2019, 4:15am UTC](https://discourse.julialang.org/t/automatic-differentiation-of-an-objective-which-already-uses-ad-tools/24864/2 "2019-06-04T04:15:10Z")
> [@ArnoStrouwen](#):
>
> ```julia
> function y(u,p1,p2)
> p1_sen = Dual{S1}(p1,1.0,0.0)
> p2_sen = Dual{S1}(p1,0.0,1.0)
>
> ```
Is `p2` meant to be used here in, it doesn’t seem to be used in function `y`? Are you using a manual tag `S1` on purpose? Why not use one of the higher level APIs?
By the way, it looks like you might be integrating an ODE in `y` and computing sensitivities wrt ODE parameters? In that case I think there’s already a lot of ecosystem support to do this at a high level of abstraction; I suggest reading [https://docs.juliadiffeq.org/latest/analysis/sensitivity.html](https://docs.juliadiffeq.org/latest/analysis/sensitivity.html)
Beyond those two comments, I _think_ you should be able to compose calls to `ForwardDiff.derivative` / `gradient` etc and things should work fine provided you use the higher level APIs rather than using `Dual` directly.
---
### Author:  [@oxinabox](https://discourse.julialang.org/u/oxinabox)
#### Post date: [June 4, 2019, 8:04am UTC](https://discourse.julialang.org/t/automatic-differentiation-of-an-objective-which-already-uses-ad-tools/24864/3 "2019-06-04T08:04:15Z")
I know you said
> For efficient optimization I would like to calculate the gradient of obj(u)
but consider giving Optim.jl’s gradient free optimizers.
Like the PSO a shot.
At least just while you are working out a better solution.
---
### Author:  [@ArnoStrouwen](https://discourse.julialang.org/u/ArnoStrouwen)
#### Post date: [June 4, 2019, 9:08am UTC](https://discourse.julialang.org/t/automatic-differentiation-of-an-objective-which-already-uses-ad-tools/24864/4 "2019-06-04T09:08:40Z")
> [@c42f](#):
>
> Is `p2` meant to be used here in, it doesn’t seem to be used in function `y` ?
Yes, I mistakenly used p1 twice.
> [@c42f](#):
>
> Are you using a manual tag `S1` on purpose? Why not use one of the higher level APIs?
>
> By the way, it looks like you might be integrating an ODE in `y` and computing sensitivities wrt ODE parameters? In that case I think there’s already a lot of ecosystem support to do this at a high level of abstraction; I suggest reading [https://docs.juliadiffeq.org/latest/analysis/sensitivity.html](https://docs.juliadiffeq.org/latest/analysis/sensitivity.html)
The problem is indeed related to ODE’s. I’m aware of the sensitivity options provided by DiffEq. Two issues: even in the link you gave they use `Dual` directly for the more complicated examples, and my problem is more complex as I require some (but not all) second order derivatives of y(u,p). So I’m not sure direct use of `Dual` can be avoided, but if it could it would save me a lot of trouble.
> [@oxinabox](#):
>
> but consider giving Optim.jl’s gradient free optimizers.
> Like the PSO a shot.
>
> At least just while you are working out a better solution.
Thanks, I’m currently using NLopt.jl to do something similar, but I’ll check out Optim.jl.