# \[ForwardDiff\] how to consider expression as constant

**URL:** <https://discourse.julialang.org/t/forwarddiff-how-to-consider-expression-as-constant/95651>\
**Category:** Machine Learning\
**Created:** [March 7, 2023, 8:21am UTC](https://discourse.julialang.org/t/forwarddiff-how-to-consider-expression-as-constant/95651 "2023-03-07T08:21:09Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![mariusd](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mariusd/32/23231_2.png) [@mariusd](https://discourse.julialang.org/u/mariusd)\
**Post date:** [March 7, 2023, 8:21am UTC](https://discourse.julialang.org/t/forwarddiff-how-to-consider-expression-as-constant/95651/1 "2023-03-07T08:21:09Z")

</div>

See the following code. Inside the function `f`, `g(x)` is called and I need the result to be used as both a variable and as constant.  
Now of course I can calculate `x20=g(x)` outside and provide this as an extra argument to the `f` function, but in this case `g(x)` is calculated twice. In my real problem, `g(x)` is an expensive function and I would avoid to call it twice. How can I achieve this?

```julia
using ForwardDiff

g = x->x^2

function f(x, y0)
    x2 = g(x)
    x20 = x2 # how to treet this as constant?

    return (x2 - x20*y0)^2
end

ForwardDiff.derivative(x->f(x, 2), 1)

```

---

<div class="post-metadata">

**Author:** ![albheim](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/albheim/32/34660_2.png) [@albheim](https://discourse.julialang.org/u/albheim)\
**Post date:** [March 7, 2023, 8:44am UTC](https://discourse.julialang.org/t/forwarddiff-how-to-consider-expression-as-constant/95651/2 "2023-03-07T08:44:01Z")

</div>

ForwardDiff will make `x2` be of type `Dual` containing the fields `value` and `partials`. So I think just accessing that should work, e.g. `x20 = x2.value`, though this relies on internals and might change (though I doubt it will change soon at least). It will also make you unable to call it with a normal value.

Looking at the source one can find a function `ForwardDiff.value(d)` which on a normal value just returns itself, and on a dual it returns the value field, though this is also undocumented so maybe equally “unreliable” as accessing the value field. But it will make your function `f` able to be called both normally and through ForwardDiff.

Can’t seem to find a documented way of doing this, maybe `ForwardDiff.value` should be documented as API?

---

<div class="post-metadata">

**Author:** ![mariusd](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mariusd/32/23231_2.png) [@mariusd](https://discourse.julialang.org/u/mariusd)\
**Post date:** [March 7, 2023, 10:11am UTC](https://discourse.julialang.org/t/forwarddiff-how-to-consider-expression-as-constant/95651/3 "2023-03-07T10:11:14Z")

</div>

I have encountered a small problem when x is an array. Using `value.(x2)` gives the correct result in this case. However, I’m not sure if this solution is the right one.  
I would also expect the `ForwardDiff.value(array)` to raise an error instead of silently giving an un-expected result.

```julia
using ForwardDiff

g = x->x^2

function f(x, y0)
    x2 = g.(x)
    # x20 = ForwardDiff.value(x2) # this does not work
    x20 = ForwardDiff.value.(x2) # this works

    return sum((x2 - x20*y0).^2)
end

ForwardDiff.derivative(x->f(x, 2), 1.f0) # returns -4.f0
ForwardDiff.gradient(x->f(x, 2), [1.f0]) # returns array [-4.f0]

```

---

<div class="post-metadata">

**Author:** ![albheim](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/albheim/32/34660_2.png) [@albheim](https://discourse.julialang.org/u/albheim)\
**Post date:** [March 7, 2023, 11:04am UTC](https://discourse.julialang.org/t/forwarddiff-how-to-consider-expression-as-constant/95651/4 "2023-03-07T11:04:08Z")

</div>

Yeah, seems like the implementation only handles `Dual` and not `Vector{Dual}`, and so the vector will fall back to just being returned.

```julia
@inline value(x) = x # Vector{Dual} falls back to this one
@inline value(d::Dual) = d.value

```

Maybe this is not the way to go then since it does not seem like the internal function `value` was intended for this. Though you could easily just create your own function, based on that implementation, and extend it to handle vectors.

Haven’t tested it, but something like this could probably work.

```julia
@inline value(x) = x                        
@inline value(d::Dual) = d.value
@inline value(d::Vector{<:Dual}) = value.(d)

```

---

<div class="post-metadata">

**Author:** ![mariusd](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mariusd/32/23231_2.png) [@mariusd](https://discourse.julialang.org/u/mariusd)\
**Post date:** [March 7, 2023, 11:26am UTC](https://discourse.julialang.org/t/forwarddiff-how-to-consider-expression-as-constant/95651/5 "2023-03-07T11:26:02Z")

</div>

Here is the updated MWE, based on your suggestion. Works as expected.

[Re-edited code]:

```julia
using ForwardDiff

@inline value(d::Vector{<:ForwardDiff.Dual}) = ForwardDiff.value.(d)

g = x->x^2

function f(x, y0)
    x2 = g.(x)
    x20 = value(x2) # treat this as a constant

    return sum((x2 - x20*y0).^2)
end

ForwardDiff.derivative(x->f(x, 2), 1.f0)
ForwardDiff.gradient(x->f(x, 2), [1.f0, 2.f0])

```

---

<div class="post-metadata">

**Author:** ![albheim](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/albheim/32/34660_2.png) [@albheim](https://discourse.julialang.org/u/albheim)\
**Post date:** [March 7, 2023, 12:41pm UTC](https://discourse.julialang.org/t/forwarddiff-how-to-consider-expression-as-constant/95651/6 "2023-03-07T12:41:49Z")

</div>

Yes, that should work.

There is an argument for creating your own function, instead of extending the one in forwarddiff with a new method. IIUC this should be considered [type piracy](https://docs.julialang.org/en/v1/manual/style-guide/#Avoid-type-piracy), since you extend functions you don’t own with types you don’t own.  
If you are only creating local scripts for you this doesn’t really matter, but if you want to make a package and share the code it is usually frowned upon to do this since it can change peoples code in unexpected ways.