# ForwardDiff and Floats

**URL:** <https://discourse.julialang.org/t/forwarddiff-and-floats/47596>\
**Category:** Optimization (Mathematical)\
**Tags:** forwarddiff\
**Created:** [October 1, 2020, 1:02pm UTC](https://discourse.julialang.org/t/forwarddiff-and-floats/47596 "2020-10-01T13:02:45Z")\
**Posts on this page:** 14\
**Page:** 1

<div class="post-metadata">

**Author:** ![danicaratelli](https://avatars.discourse-cdn.com/v4/letter/d/90db22/32.png) [@danicaratelli](https://discourse.julialang.org/u/danicaratelli)\
**Post date:** [October 1, 2020, 1:02pm UTC](https://discourse.julialang.org/t/forwarddiff-and-floats/47596/1 "2020-10-01T13:02:45Z")

</div>

Hello,

I am trying to use Ipopt in JuMP to solve a minimization problem with automatic differentiation.

I am aware that I need to specify the types of the inputs as `Real` rather than `Float64` but it’s not clear to me to what extent I must do so because I still get this error message:

```julia

MethodError: no method matching InvF(::Dict{String,Any}, ::Array{ForwardDiff.Dual{ForwardDiff.Tag{JuMP.var"#110#112"{typeof(objfunc)},Float64},Float64,3},3}, ::Array{ForwardDiff.Dual{ForwardDiff.Tag{JuMP.var"#110#112"{typeof(objfunc)},Float64},Float64,3},2}, ::Array{ForwardDiff.Dual{ForwardDiff.Tag{JuMP.var"#110#112"{typeof(objfunc)},Float64},Float64,3},3}, ::Array{ForwardDiff.Dual{ForwardDiff.Tag{JuMP.var"#110#112"{typeof(objfunc)},Float64},Float64,3},2}, ::Array{ForwardDiff.Dual{ForwardDiff.Tag{JuMP.var"#110#112"{typeof(objfunc)},Float64},Float64,3},3}, ::Array{ForwardDiff.Dual{ForwardDiff.Tag{JuMP.var"#110#112"{typeof(objfunc)},Float64},Float64,3},1})

```

I hope that the lines below suffice for this, if not I am happy to provide more details:

The minimization problem

```julia
function objfunc(x1::Real,x2::Real,x3::Real)
    return sum(SSresid(mod_pars,x1,x2,x3).^2);
end
model = Model(with_optimizer(Ipopt.Optimizer, max_iter=100, print_level=0));
#adding variables
@variable(model, 0.8 <= x1 <= 1.2);
@variable(model, 95 <= x2 <= 100);
@variable(model, 1e-6 <= x3 <= 0.01);
#register function
register(model, :objfunc, 3, objfunc, autodiff=true);
@NLobjective(model, Min, objfunc(x1,x2,x3));
@time xsol_jump = JuMP.optimize!(model);

```

The functions called and the types specified:

```julia
function SSresid(mp::Dict,L::Real,K::Real,τE::Real)
    #Step 1: solve HH problem
    CUpol, CEpol, Us, Es, AUpol, AEpol = HHblock(mp,L,K,τE,false);
    println(typeof(CUpol))
    #Step 2: stationary distribution
    DistE, DistU = InvF(mp,CEpol,CUpol,AEpol,AUpol,Es,Us);
    #Step 3: clearing markets
    res = MCblock(mp,DistE,DistU,CEpol,CUpol,L,K,τE);
    return abs.(res);
end

function InvF(mp::Dict,CEpol::Array{Real}, CUpol::Array{Real},
              AEpol::Array{Real}, AUpol::Array{Real}, Es::Array{Real},
              Us::Array{Real}, tol=1e-6,maxiter=10000)
    #initializing distributions (to have everyone with no assets)
    DistE0 = zeros(Real,N,M,T); DistE0[1,1,1] = 0.92;
    DistU0 = zeros(Real,N); DistU0[1] = 1-0.92;
    DistE1 = copy(DistE0); DistU1 = copy(DistU0);
    #iterating until convergence
    itnum = 0; dist = Inf;
    while dist>tol && itnum<maxiter
        DistE1, DistU1 = updateDist(mp,DistE0, DistU0, CEpol, CUpol, AEpol, AUpol, Es, Us);
        dist = max(maximum(abs.(DistE1.-DistE0)),maximum(abs.(DistU1.-DistU0)));
        DistE0 = copy(DistE1);
        DistU0 = copy(DistU1);
        itnum +=1;
    end
    if itnum==1000
        @warn "Reached maximum number of iterations, iteration did not converge"
    end
    return DistE1, DistU1;
end

```

---

<div class="post-metadata">

**Author:** ![platawiec](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/platawiec/32/31914_2.png) [@platawiec](https://discourse.julialang.org/u/platawiec)\
**Post date:** [October 1, 2020, 3:17pm UTC](https://discourse.julialang.org/t/forwarddiff-and-floats/47596/2 "2020-10-01T15:17:19Z")

</div>

You can get rid of all of your type annotations. You don’t need them. Additionally, you will want to rewrite `DistE0` and `DistU0`, as you have written the entries are not concrete (check `isconcretetype(Real)` vs. `isconcretetype(Float64)`).

Also I’m not sure what `N`, `M`, and `T` are, possibly they are global variables? You should either pass them to `InvF` or calculate them from the matrices you already provided.

There are a few ways to do this. I’ve shown one way below:

```julia
function InvF(mp, CEpol, CUpol, AEpol, AUpol, Es, Us, tol=1e-6, maxiter=10000)
    #initializing distributions (to have everyone with no assets)
    DistE0 = zeros(eltype(CEpol),N,M,T); DistE0[1,1,1] = 0.92;
    DistU0 = zeros(eltype(CEpol),N); DistU0[1] = 1-0.92;

```

It will also run faster if you do `copyto!(DistE0, DistE1)` instead of `copy`. There are other performance gains you can eek out if you look through the [Performance Tips · The Julia Language](https://docs.julialang.org/en/v1/manual/performance-tips/)

---

<div class="post-metadata">

**Author:** ![odow](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/odow/32/28685_2.png) [@odow](https://discourse.julialang.org/u/odow)\
**Post date:** [October 2, 2020, 4:31am UTC](https://discourse.julialang.org/t/forwarddiff-and-floats/47596/3 "2020-10-02T04:31:21Z")

</div>

The underlying issue is this:

```nohighlight
julia> x = rand(3)
3-element Array{Float64,1}:
 0.6017620020347454
 0.09429667862972102
 0.39870937669167517

julia> typeof(x) <: Array{Real}
false

julia> typeof(x) <: Array{<:Real}
true

```

This prevents `InvF` being called with the arguments you expect. You want `CEpol::Array{<:Real}` instead of `CEpol::Array{Real}`.

Here is the relevant section of the documentation: [Types · The Julia Language](https://docs.julialang.org/en/v1/manual/types/#Parametric-Composite-Types)

---

<div class="post-metadata">

**Author:** ![DShiu](https://avatars.discourse-cdn.com/v4/letter/d/8e8cbc/32.png) [@DShiu](https://discourse.julialang.org/u/DShiu)\
**Post date:** [October 2, 2020, 4:55am UTC](https://discourse.julialang.org/t/forwarddiff-and-floats/47596/4 "2020-10-02T04:55:42Z")

</div>

Can this issue be fixed by removing all type annotations?

---

<div class="post-metadata">

**Author:** ![danicaratelli](https://avatars.discourse-cdn.com/v4/letter/d/90db22/32.png) [@danicaratelli](https://discourse.julialang.org/u/danicaratelli)\
**Post date:** [October 2, 2020, 1:06pm UTC](https://discourse.julialang.org/t/forwarddiff-and-floats/47596/5 "2020-10-02T13:06:03Z")

</div>

But what if a particular array really needs to be integers because it is an array of indices which I use to extract some other matrix’s elements?

---

<div class="post-metadata">

**Author:** ![DNF](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dnf/32/10191_2.png) [@DNF](https://discourse.julialang.org/u/DNF)\
**Post date:** [October 2, 2020, 2:00pm UTC](https://discourse.julialang.org/t/forwarddiff-and-floats/47596/6 "2020-10-02T14:00:25Z")

</div>

Nothing is stopping it from being integers. It’s just not necessary to specify it. If you feel the need to specify integers for whatever reason, I guess you can, but then that array should not be one of the optimization variables.

---

<div class="post-metadata">

**Author:** ![danicaratelli](https://avatars.discourse-cdn.com/v4/letter/d/90db22/32.png) [@danicaratelli](https://discourse.julialang.org/u/danicaratelli)\
**Post date:** [October 2, 2020, 6:35pm UTC](https://discourse.julialang.org/t/forwarddiff-and-floats/47596/7 "2020-10-02T18:35:31Z")

</div>

But when I try, I still get the mistake. This is the line where the error appears:

```julia
println("Type *** is ", typeof(convert(Array{Int64,1},xqi)))

```

this is the error:

```julia
MethodError: no method matching Int64(::ForwardDiff.Dual{ForwardDiff.Tag{JuMP.var"#110#112"{typeof(objfunc)},Float64},Float64,3})

```

so I can’t really convert. Also, `xqi` is **not** the optimizing variable.

---

<div class="post-metadata">

**Author:** ![DNF](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dnf/32/10191_2.png) [@DNF](https://discourse.julialang.org/u/DNF)\
**Post date:** [October 2, 2020, 8:17pm UTC](https://discourse.julialang.org/t/forwarddiff-and-floats/47596/8 "2020-10-02T20:17:26Z")

</div>

When you try what?

---

<div class="post-metadata">

**Author:** ![danicaratelli](https://avatars.discourse-cdn.com/v4/letter/d/90db22/32.png) [@danicaratelli](https://discourse.julialang.org/u/danicaratelli)\
**Post date:** [October 2, 2020, 9:44pm UTC](https://discourse.julialang.org/t/forwarddiff-and-floats/47596/9 "2020-10-02T21:44:38Z")

</div>

> [@odow](#):
>
> `typeof(x) <: Array{<:Real}`

Sorry, ignore the previous message. I screwed up.

---

<div class="post-metadata">

**Author:** ![danicaratelli](https://avatars.discourse-cdn.com/v4/letter/d/90db22/32.png) [@danicaratelli](https://discourse.julialang.org/u/danicaratelli)\
**Post date:** [October 2, 2020, 10:11pm UTC](https://discourse.julialang.org/t/forwarddiff-and-floats/47596/10 "2020-10-02T22:11:41Z")

</div>

Though I resolved some intermediate issues, I am still having some problems. The code is as follows:

```julia
using JuMP, Ipopt;
function objfunc(x1,x2,x3)
    return sum(SSresid(mod_pars,x1,x2,x3).^2);
end
model = Model(with_optimizer(Ipopt.Optimizer, max_iter=100, print_level=0));
#adding variables
@variable(model, 0.8 <= x1 <= 1.2);
@variable(model, 95 <= x2 <= 100);
@variable(model, 1e-6 <= x3 <= 0.01);
#register function
register(model, :objfunc, 3, objfunc, autodiff=true);
@NLobjective(model, Min, objfunc(x1,x2,x3));
@time xsol_jump = JuMP.optimize!(model);

function SSresid(mp,L,K,τE)
    println("DANIELE is HERE")
    #Step 1: solve HH problem
    println(" caca: ----> ", typeof(L))
    CUpol, CEpol, Us, Es, AUpol, AEpol = HHblock(mp,L,K,τE,false);
    #Step 2: stationary distribution
    DistE, DistU = InvF(mp,CEpol,CUpol,AEpol,AUpol,Es,Us);
    #Step 3: clearing markets
    res = MCblock(mp,DistE,DistU,CEpol,CUpol,L,K,τE);
    return abs.(res);
end

```

From what I can tell, the first time `SSresid` is run there are no problems and in particular, the type is `Float64`, the second time it’s `ForwardDiff.Dual{ForwardDiff.Tag{JuMP.var"#110#112"{typeof(objfunc)},Float64},Float64,3}` which then gives me an error when I run the `InvF` function (again, on the second call of `SSresid`).

Thoughts? Thank you!

---

<div class="post-metadata">

**Author:** ![odow](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/odow/32/28685_2.png) [@odow](https://discourse.julialang.org/u/odow)\
**Post date:** [October 2, 2020, 11:00pm UTC](https://discourse.julialang.org/t/forwarddiff-and-floats/47596/11 "2020-10-02T23:00:12Z")

</div>

Please provide a simplified reproducible example. I can’t run your code because I don’t know what things like `mod_pars` or `HHblock` are.

Your code needs to work regardless of what type `x1`, `x2`, or `x3` are. All you can assume is that the are a subtype of `Real`. The `no method matching Int64` error suggests you are trying to convert one of them to an integer. You can’t do this. A full (simplified) example will help people to offer more insight.

---

<div class="post-metadata">

**Author:** ![danicaratelli](https://avatars.discourse-cdn.com/v4/letter/d/90db22/32.png) [@danicaratelli](https://discourse.julialang.org/u/danicaratelli)\
**Post date:** [October 3, 2020, 7:25pm UTC](https://discourse.julialang.org/t/forwarddiff-and-floats/47596/12 "2020-10-03T19:25:11Z")

</div>

Thank you. Let me try this last question, if that fails I will try to put together a simple example (sorry the code I have is quite convoluted).

1st → all variables that I derive using my optimization variables in some way or another will derive its type, in my case `ForwardDiff.Dual{ForwardDiff.Tag{JuMP.var"#110#112"{typeof(objfunc)},Float64},Float64,3})`?

2nd → is there a way to convert such secondary variables to, say, Float64 within the code? Simple `convert` will not work.

Thank you!

---

<div class="post-metadata">

**Author:** ![Humphrey\_Lee](https://avatars.discourse-cdn.com/v4/letter/h/a88e4f/32.png) [@Humphrey\_Lee](https://discourse.julialang.org/u/Humphrey_Lee)\
**Post date:** [April 28, 2021, 8:01am UTC](https://discourse.julialang.org/t/forwarddiff-and-floats/47596/13 "2021-04-28T08:01:06Z")

</div>

@danicaratelli Have you found any solution to this problem? If yes, would you please share the solution for greater community learning? FYI, I’m having similar problems and have been duckduckgoing for days.

---

<div class="post-metadata">

**Author:** ![danicaratelli](https://avatars.discourse-cdn.com/v4/letter/d/90db22/32.png) [@danicaratelli](https://discourse.julialang.org/u/danicaratelli)\
**Post date:** [April 28, 2021, 2:18pm UTC](https://discourse.julialang.org/t/forwarddiff-and-floats/47596/14 "2021-04-28T14:18:58Z")

</div>

Hello, no, I ended up using a different solver with numerical differentiation. I now use NLsolve.