# Errors with autodiff in a for loop
**URL:** https://discourse.julialang.org/t/errors-with-autodiff-in-a-for-loop/75283
**Category:** New to Julia
**Tags:** question, flux, forwarddiff
**Created:** [January 27, 2022, 11:51am UTC](https://discourse.julialang.org/t/errors-with-autodiff-in-a-for-loop/75283 "2022-01-27T11:51:36Z")
**Posts on this page:** 3
**Page:** 1
### Author:  [@adeyemiadeoye](https://discourse.julialang.org/u/adeyemiadeoye)
#### Post date: [January 27, 2022, 11:51am UTC](https://discourse.julialang.org/t/errors-with-autodiff-in-a-for-loop/75283/1 "2022-01-27T11:51:36Z")
While implementing an on-line optimization method (with a for loop to access some new data), I am using `HesVec()` and `JacVec()` of [SparseDiffTools.jl](https://github.com/JuliaDiff/SparseDiffTools.jl) to compute sparse Hessian and Jacobian matrices with option `autodiff=false` for each. Inside the same loop, I have to compute gradients this time with `sparse(ForwardDiff.gradient(objectivefn, somevector))`. Each of these autodiff tools has to compute the objective value each time it is called (I guess). I do not get any error with `HesVec()` and `JacVec()` (I suppose everything is fine!); but after the first step (first data), `ForwardDiff.gradient()` throws a conversion error in the line where `Flux.Losses.logitcrossentropy(yhat, yi)` is called in the loss function (apparently, trying to do some `ForwardDiff.Dual` to `AbstractFloat` conversion). One guess is that there may be some NaN values somewhere from the computation of the gradients that Julia identifies as #unused# (I don’t know), as shown in the error text below. The problem is I find it difficult to properly trace this error to know the exact cause or to know maybe `ForwardDiff.gradient()` is unable to handle sparsity or largeness of values of the input vector after the first update?
```julia
MethodError: no method matching Float64(::ForwardDiff.Dual{ForwardDiff.Tag{Main.ModuleA.var"#4#6"{Matrix{Float32}, Main.ModuleA.var"#3#5"{Flux.var"#64#66"{Vector{AbstractArray{Float32}}}, Int64}, Flux.var"#64#66"{Vector{AbstractArray{Float32}}}, Int64}, Float32}, Float64, 12})
Closest candidates are:
(::Type{T})(::Real, ::RoundingMode) where T<:AbstractFloat at C:\Users\user\AppData\Local\Programs\Julia-1.7.1\share\julia\base\rounding.jl:200
(::Type{T})(::T) where T<:Number at C:\Users\user\AppData\Local\Programs\Julia-1.7.1\share\julia\base\boot.jl:770
(::Type{T})(::AbstractChar) where T<:Union{AbstractChar, Number} at C:\Users\user\AppData\Local\Programs\Julia-1.7.1\share\julia\base\char.jl:50
...
Stacktrace:
[1] convert(#unused#::Type{Float64}, x::ForwardDiff.Dual{ForwardDiff.Tag{Main.ModuleA.var"#4#6"{Matrix{Float32}, Main.ModuleA.var"#3#5"{Flux.var"#64#66"{Vector{AbstractArray{Float32}}}, Int64}, Flux.var"#64#66"{Vector{AbstractArray{Float32}}}, Int64}, Float32}, Float64, 12})
@ Base .\number.jl:7
```
On the other hand, I would like to know if there is a way to “record operations for autodiff” as in [tensorflow’s GradientTape](https://www.tensorflow.org/api_docs/python/tf/GradientTape), where I could call my objective function only once and use its value to compute the various derivatives with respect to a “watched” variable? With this, I think it would be easier to maybe trace this error. As I suspect Julia autodiff could be manipulating the input vector types somehow.
Thank you.
---
### Author:  [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)
#### Post date: [January 27, 2022, 12:58pm UTC](https://discourse.julialang.org/t/errors-with-autodiff-in-a-for-loop/75283/2 "2022-01-27T12:58:26Z")
You need to make sure your caches accept dual numbers. See [GitHub - SciML/PreallocationTools.jl: Speed at all costs](https://github.com/SciML/PreallocationTools.jl) as a tool to help with this.
---
### Author:  [@adeyemiadeoye](https://discourse.julialang.org/u/adeyemiadeoye)
#### Post date: [January 27, 2022, 10:51pm UTC](https://discourse.julialang.org/t/errors-with-autodiff-in-a-for-loop/75283/3 "2022-01-27T22:51:53Z")
That solves the problem! Thanks.