# Type instability in ForwardDiff.gradient
**URL:** https://discourse.julialang.org/t/type-instability-in-forwarddiff-gradient/68973
**Category:** Numerics
**Tags:** performance, forwarddiff, type-stability
**Created:** [September 30, 2021, 2:06am UTC](https://discourse.julialang.org/t/type-instability-in-forwarddiff-gradient/68973 "2021-09-30T02:06:41Z")
**Posts on this page:** 9
**Page:** 1
### Author:  [@ElOceanografo](https://discourse.julialang.org/u/ElOceanografo)
#### Post date: [September 30, 2021, 2:06am UTC](https://discourse.julialang.org/t/type-instability-in-forwarddiff-gradient/68973/1 "2021-09-30T02:06:42Z")
Working on a nested optimization problem, I noticed that a lot of time was being spent in gradient calls, and a little digging found that doing out-of-place gradients was introducing a type-instability. I followed the advice [here](https://juliadiff.org/ForwardDiff.jl/latest/user/advanced.html#Configuring-Chunk-Size-1), but it does not appear to have helped…see MWE below:
```julia
using ForwardDiff, LinearAlgebra
f(x) = dot(x, x)
x = randn(100)
gconfig = ForwardDiff.GradientConfig(f, x)
# all of these are type-unstable
g1(x) = ForwardDiff.gradient(f, x)
g2(x) = ForwardDiff.gradient(f, x, gconfig)
g3(x::T) where T = ForwardDiff.gradient(f, x)::T
g4(x::T) where T = ForwardDiff.gradient(f, x, gconfig)::T
@code_warntype g1(x)
@code_warntype g2(x)
@code_warntype g3(x)
@code_warntype g4(x)
```
Using the in-place gradient is type-stable, as expected:
```julia
# these are good
g1!(G, x) = ForwardDiff.gradient!(G, f, x)
g2!(G, x) = ForwardDiff.gradient!(G, f, x, gconfig)
G = zero(x)
@code_warntype g1!(G, x)
@code_warntype g2!(G, x)
@time g1!(G, x)
@time g2!(G, x)
```
So I should probably just use that. Still, I’m curious what the cause of the instability is. On Julia 1.6.2, ForwardDiff v0.10.19. Thanks!
---
### Author:  [@longemen3000](https://discourse.julialang.org/u/longemen3000)
#### Post date: [September 30, 2021, 5:16am UTC](https://discourse.julialang.org/t/type-instability-in-forwarddiff-gradient/68973/2 "2021-09-30T05:16:25Z")
i think is because the chunk size is not specified, can you try with this?
```julia
gconfig = GradientConfig(f, x, Chunk{12}());
#or any other number depending on the aplication
#the default is 12
```
[https://juliadiff.org/ForwardDiff.jl/latest/user/advanced.html#Configuring-Chunk-Size-1](https://juliadiff.org/ForwardDiff.jl/latest/user/advanced.html#Configuring-Chunk-Size-1)
---
### Author:  [@ElOceanografo](https://discourse.julialang.org/u/ElOceanografo)
#### Post date: [September 30, 2021, 3:57pm UTC](https://discourse.julialang.org/t/type-instability-in-forwarddiff-gradient/68973/3 "2021-09-30T15:57:05Z")
No, unfortunately I tried that already and it doesn’t change anything…
---
### Author:  [@goerch](https://discourse.julialang.org/u/goerch)
#### Post date: [October 17, 2021, 3:19pm UTC](https://discourse.julialang.org/t/type-instability-in-forwarddiff-gradient/68973/4 "2021-10-17T15:19:36Z")
This does the trick for me:
```julia
using ForwardDiff, LinearAlgebra
f(x) = dot(x, x)
x = randn(100)
gconfig = ForwardDiff.GradientConfig(f, x)
# Signature of gradient
# function gradient(f, x::AbstractArray, cfg::GradientConfig{T} = GradientConfig(f, x), ::Val{CHK}=Val{true}()) where {T, CHK}
# all of these are now stable
g1(x) = ForwardDiff.gradient{Float64, Val{true}}(f, x)
g2(x) = ForwardDiff.gradient{Float64, Val{true}}(f, x, gconfig)
g3(x::T) where T = ForwardDiff.gradient{T, Val{true}}(f, x)::T
g4(x::T) where T = ForwardDiff.gradient{T, Val{true}}(f, x, gconfig)::T
@code_warntype g1(x)
@code_warntype g2(x)
@code_warntype g3(x)
@code_warntype g4(x)
```
---
### Author:  [@ElOceanografo](https://discourse.julialang.org/u/ElOceanografo)
#### Post date: [October 18, 2021, 7:15pm UTC](https://discourse.julialang.org/t/type-instability-in-forwarddiff-gradient/68973/5 "2021-10-18T19:15:09Z")
Thanks, that works for me too. I’m wondering why those types need to be specified manually, though?
---
### Author:  [@goerch](https://discourse.julialang.org/u/goerch)
#### Post date: [October 18, 2021, 8:54pm UTC](https://discourse.julialang.org/t/type-instability-in-forwarddiff-gradient/68973/6 "2021-10-18T20:54:31Z")
I don’t see how the compiler could infer `CHK` from the arguments `f`and `x`. And I don’t know if there is a syntactic way to specify `CHK` only and let the compiler infer T.
---
### Author:  [@ElOceanografo](https://discourse.julialang.org/u/ElOceanografo)
#### Post date: [October 18, 2021, 9:56pm UTC](https://discourse.julialang.org/t/type-instability-in-forwarddiff-gradient/68973/7 "2021-10-18T21:56:50Z")
Actually, those don’t work–the macro says they’re type-stable, but they don’t actually run. With `g1` and `x` defined as above, for example:
```julia
julia> g1(x)
ERROR: TypeError: in Type{...} expression, expected UnionAll, got a value of type typeof(ForwardDiff.gradient)
Stacktrace:
[1] g1(x::Vector{Float64})
@ Main .\REPL[8]:1
[2] top-level scope
@ REPL[17]:1
```
I get the same error for all four functions.
---
### Author:  [@goerch](https://discourse.julialang.org/u/goerch)
#### Post date: [October 19, 2021, 8:29am UTC](https://discourse.julialang.org/t/type-instability-in-forwarddiff-gradient/68973/8 "2021-10-19T08:29:19Z")
My bad. Didn’t bother to execute the function. Next try:
```julia
using ForwardDiff, LinearAlgebra
x = randn(2)
f(x) = dot(x, x)
cfg = ForwardDiff.GradientConfig(f, x, ForwardDiff.Chunk{2}())
@code_warntype ForwardDiff.gradient(f, x, cfg)
ForwardDiff.gradient(f, x, cfg)
```
with
```julia
Variables
#self#::Core.Const(ForwardDiff.gradient)
f::Core.Const(f)
x::Vector{Float64}
cfg::ForwardDiff.GradientConfig{ForwardDiff.Tag{typeof(f), Float64}, Float64, 2, Vector{ForwardDiff.Dual{ForwardDiff.Tag{typeof(f), Float64}, Float64, 2}}}
Body::Vector{Float64}
1 ─ %1 = Core.apply_type(ForwardDiff.Val, true)::Core.Const(Val{true})
│ %2 = (%1)()::Core.Const(Val{true}())
│ %3 = (#self#)(f, x, cfg, %2)::Vector{Float64}
└── return %3
2-element Vector{Float64}:
0.320787759225816
1.9478166209592147
```
---
### Author:  [@ElOceanografo](https://discourse.julialang.org/u/ElOceanografo)
#### Post date: [October 20, 2021, 12:04am UTC](https://discourse.julialang.org/t/type-instability-in-forwarddiff-gradient/68973/9 "2021-10-20T00:04:14Z")
No worries, I didn’t try to actually execute it at first either. This seems to work, and the reason I thought it didn’t [above](https://discourse.julialang.org/t/type-instability-in-forwarddiff-gradient/68973/3) seems to be that I’d defined the `GradientConfig` in the global scope. In the example below `g1` is not type-stable, but `g2` is.
```julia
using ForwardDiff, LinearAlgebra
x = randn(100)
f(x) = dot(x, x)
cfg = ForwardDiff.GradientConfig(f, x, ForwardDiff.Chunk{2}())
g1(x) = ForwardDiff.gradient(f, x, cfg)
g2 = let cfg = cfg
x -> ForwardDiff.gradient(f, x, cfg)
end
@code_warntype g1(x)
@code_warntype g2(x)
```
I’ll need to check my original, non-MWE code, but suspect the issue there may have also been scoping-related.