# Optimizing an objective function which contains auto-diifferentiation

**URL:** <https://discourse.julialang.org/t/optimizing-an-objective-function-which-contains-auto-diifferentiation/130395>\
**Category:** General Usage\
**Tags:** question, optim\
**Created:** [July 2, 2025, 3:50am UTC](https://discourse.julialang.org/t/optimizing-an-objective-function-which-contains-auto-diifferentiation/130395 "2025-07-02T03:50:27Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![sn248](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/sn248/32/210680_2.png) [@sn248](https://discourse.julialang.org/u/sn248)\
**Post date:** [July 2, 2025, 3:50am UTC](https://discourse.julialang.org/t/optimizing-an-objective-function-which-contains-auto-diifferentiation/130395/1 "2025-07-02T03:50:27Z")

</div>

Hi

I am trying to optimize an objective function which itself contains auto-differentiation, but the optimization fails, so I tried with a trivial example and it still fails. If anyone can suggest any approaches, that will be great.

Thanks!

See the trivial example below:

```julia
using Optim
using ForwardDiff

## Objective Function
function obj(x)
   return 2 * (x[1] - 1)^2 + 4 * (x[1] - 1) # this works
   ## return 2.0 * (x[1] - 1)^2 + ForwardDiff.derivative(x -> 2.0 * (x[1] - 1)^2, x) ## fails
end

opt_options = Optim.Options(
    g_abstol=1e-4, 
    f_abstol=1e-3, 
    iterations=1000, 
    f_calls_limit=10000, 
    store_trace=true,
    show_trace=true,
    show_every=10 
)

lower_bounds = [-10.0]
upper_bounds = [10.0]
initial_params = [5.0]

optimizer_to_use = Fminbox(BFGS())

solution = Optim.optimize(
    obj,
    lower_bounds,
    upper_bounds, # Required by Fminbox
    initial_params,
    optimizer_to_use,
    opt_options;
    autodiff = :finite,) # :finite or :forward, fails with using ForwardDiff within Obj

## minimum is -2.0 at x = 0.0
solution.minimizer, solution.minimum

```

---

<div class="post-metadata">

**Author:** ![Benny](https://avatars.discourse-cdn.com/v4/letter/b/49beb7/32.png) [@Benny](https://discourse.julialang.org/u/Benny)\
**Post date:** [July 2, 2025, 4:54am UTC](https://discourse.julialang.org/t/optimizing-an-objective-function-which-contains-auto-diifferentiation/130395/2 "2025-07-02T04:54:11Z")

</div>

> [@sn248](#):
>
> `ForwardDiff.derivative(x -> 2.0 * (x[1] - 1)^2, x) ## fails`

Haven’t ran it, but at first glance, shouldn’t this be `ForwardDiff.derivative(x -> 2.0 * (x - 1)^2, x[1])`? `x[1]` is your input, and the closure’s `x` is a separate variable for the function to differentiate.

---

<div class="post-metadata">

**Author:** ![gdalle](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gdalle/32/27854_2.png) [@gdalle](https://discourse.julialang.org/u/gdalle)\
**Post date:** [July 2, 2025, 7:10am UTC](https://discourse.julialang.org/t/optimizing-an-objective-function-which-contains-auto-diifferentiation/130395/3 "2025-07-02T07:10:25Z")

</div>

```julia
ForwardDiff.derivative(xvar -> 2.0 * (xvar - 1)^2, x[1])

```

This formulation might make it even cleaner that you’re taking the derivative of the function `f(xvar)` applied at `xvar = x[1]`

---

<div class="post-metadata">

**Author:** ![sn248](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/sn248/32/210680_2.png) [@sn248](https://discourse.julialang.org/u/sn248)\
**Post date:** [July 2, 2025, 12:22pm UTC](https://discourse.julialang.org/t/optimizing-an-objective-function-which-contains-auto-diifferentiation/130395/4 "2025-07-02T12:22:33Z")

</div>

That seems to have solved the issue!

Pasting the working code below.

```julia
using Optim
using ForwardDiff

## Objective Function
function obj(x)
   ##return 2 * (x[1] - 1)^2 + 4 * (x[1] - 1) # this works
   return 2.0 * (x[1] - 1)^2 + ForwardDiff.derivative(xvar -> 2.0 * (xvar - 1)^2, x[1]) ## fails
end

opt_options = Optim.Options(
    g_abstol=1e-4, 
    f_abstol=1e-3, 
    iterations=1000, 
    f_calls_limit=10000, 
    store_trace=true,
    show_trace=true,
    show_every=10 
)

lower_bounds = [-10.0]
upper_bounds = [10.0]
initial_params = [5.0]

optimizer_to_use = Fminbox(BFGS())

solution = Optim.optimize(
    obj,
    lower_bounds,
    upper_bounds, 
    initial_params,
    optimizer_to_use,
    opt_options;
    autodiff = :forward,)       
## minimum is -2.0 at x = 0.0

solution.minimizer, solution.minimum

```

[/quote]