# ForwardDiff.jl \\ Optim.jl and recompilation for any vector size

**URL:** <https://discourse.julialang.org/t/forwarddiff-jl-optim-jl-and-recompilation-for-any-vector-size/53104>\
**Category:** Optimization (Mathematical)\
**Created:** [January 10, 2021, 6:56am UTC](https://discourse.julialang.org/t/forwarddiff-jl-optim-jl-and-recompilation-for-any-vector-size/53104 "2021-01-10T06:56:40Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![PharmCat](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pharmcat/32/6953_2.png) [@PharmCat](https://discourse.julialang.org/u/PharmCat)\
**Post date:** [January 10, 2021, 6:56am UTC](https://discourse.julialang.org/t/forwarddiff-jl-optim-jl-and-recompilation-for-any-vector-size/53104/1 "2021-01-10T06:56:40Z")

</div>

I try to optimize function with different variable vector size. And I think that for each task type method is compilated again. Main function is type-stable as I think by @code\_warntype and @code\_typed. And probably each tag for Dual number lead to recompilation. May be I’m wrong… But how can i solve this problem?

---

<div class="post-metadata">

**Author:** ![dpsanders](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dpsanders/32/3573_2.png) [@dpsanders](https://discourse.julialang.org/u/dpsanders)\
**Post date:** [January 10, 2021, 8:24am UTC](https://discourse.julialang.org/t/forwarddiff-jl-optim-jl-and-recompilation-for-any-vector-size/53104/2 "2021-01-10T08:24:10Z")

</div>

Please provide a minimal working example that can be copied and pasted.

---

<div class="post-metadata">

**Author:** ![PharmCat](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pharmcat/32/6953_2.png) [@PharmCat](https://discourse.julialang.org/u/PharmCat)\
**Post date:** [January 10, 2021, 12:55pm UTC](https://discourse.julialang.org/t/forwarddiff-jl-optim-jl-and-recompilation-for-any-vector-size/53104/3 "2021-01-10T12:55:47Z")

</div>

@dpsanders , yep!

code:

```julia
using ForwardDiff
function f(x::Vector{T}) where T
    x2 = x .* x
    return log(sum(x2))
end
v1 = [1., 2.]
v2 = [1., 2., 3.]
@time ForwardDiff.hessian(f, v1)
@time ForwardDiff.hessian(f, v1)
@time ForwardDiff.hessian(f, v2)
@time ForwardDiff.hessian(f, v2)

```

out:

```julia
julia> @time ForwardDiff.hessian(f, v1)
  2.000931 seconds (4.09 M allocations: 232.179 MiB, 99.94% compilation time)
2×2 Matrix{Float64}:
  0.24 -0.32
 -0.32 -0.24

julia> 

julia> @time ForwardDiff.hessian(f, v1)
  0.000030 seconds (10 allocations: 1.359 KiB)
2×2 Matrix{Float64}:
  0.24 -0.32
 -0.32 -0.24

julia> 

julia> @time ForwardDiff.hessian(f, v2)
  2.122246 seconds (4.02 M allocations: 224.059 MiB, 7.06% gc time, 99.97% compilation time)
3×3 Matrix{Float64}:
  0.122449 -0.0408163 -0.0612245
 -0.0408163 0.0612245 -0.122449
 -0.0612245 -0.122449 -0.0408163

julia> 

julia> @time ForwardDiff.hessian(f, v2)
  0.000030 seconds (10 allocations: 2.828 KiB)
3×3 Matrix{Float64}:
  0.122449 -0.0408163 -0.0612245
 -0.0408163 0.0612245 -0.122449
 -0.0612245 -0.122449 -0.0408163

julia> 

```

first call f(x) with v2 - 2.122246 seconds

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [January 10, 2021, 1:30pm UTC](https://discourse.julialang.org/t/forwarddiff-jl-optim-jl-and-recompilation-for-any-vector-size/53104/4 "2021-01-10T13:30:02Z")

</div>

That’s because the chunk size is changing. Use a fixed chunk size if you don’t want recompilation.

---

<div class="post-metadata">

**Author:** ![PharmCat](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pharmcat/32/6953_2.png) [@PharmCat](https://discourse.julialang.org/u/PharmCat)\
**Post date:** [January 10, 2021, 5:54pm UTC](https://discourse.julialang.org/t/forwarddiff-jl-optim-jl-and-recompilation-for-any-vector-size/53104/5 "2021-01-10T17:54:21Z")

</div>

@ChrisRackauckas , thank you!

This example works fine with Chunk{1}()

How to implement this to Optim.jl ?

Previous code:

```julia
td = TwiceDifferentiable(optfunc, θ; autodiff = :forward)
optmethod = Optim.Newton(;alphaguess = LineSearches.InitialStatic(), linesearch = LineSearches.HagerZhang())
        opt = Optim.optimize(td, θ, optmethod)

```

I make this, it works, but it works with recompilation:

```julia
chunk = ForwardDiff.Chunk{1}()
    gcfg = ForwardDiff.GradientConfig(optfunc, θ, chunk)
    hcfg = ForwardDiff.HessianConfig(optfunc, θ, chunk)
    gfunc!(g, x) = ForwardDiff.gradient!(g, optfunc, x, gcfg)
    hfunc!(h, x) = ForwardDiff.hessian!(h, optfunc, x, hcfg)
    td = TwiceDifferentiable(optfunc, gfunc!, hfunc!, θ)
optmethod = Optim.Newton(;alphaguess = LineSearches.InitialStatic(), linesearch = LineSearches.HagerZhang())
        opt = Optim.optimize(td, θ, optmethod)

```

Edit: fix.