# Compile FastDifferentiation derivatives one time only

**URL:** <https://discourse.julialang.org/t/compile-fastdifferentiation-derivatives-one-time-only/121816>\
**Category:** Machine Learning\
**Tags:** autodiff\
**Created:** [October 27, 2024, 6:04am UTC](https://discourse.julialang.org/t/compile-fastdifferentiation-derivatives-one-time-only/121816 "2024-10-27T06:04:36Z")\
**Posts on this page:** 8\
**Page:** 2

<div class="post-metadata">

**Author:** ![abraemer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/abraemer/32/51403_2.png) [@abraemer](https://discourse.julialang.org/u/abraemer)\
**Post date:** [October 28, 2024, 3:09pm UTC](https://discourse.julialang.org/t/compile-fastdifferentiation-derivatives-one-time-only/121816/21 "2024-10-28T15:09:43Z")

</div>

> [@gdalle](#):
>
> This is type piracy, I would strongly discourage it.

Is it though? The type `FunctionWithPreppedGradient` is owned by the user. So it is not type piracy in a strict sense I think. Depending on the dispatch rules inside `DifferentiationInterface` (which you are the expert of 🙂 ) it could cause method ambiguity problems and perhaps the signature needs to be more specific.

In this case it is convenient to define this additional method then the resulting code would work independently of whether the gradient has been prepared or not (if I understand correctly from the documentation that `value_and_gradient` does not require the prepared thing).

Thinking this over while typing, I agree that it probably would be better to define a custom function and just forward to DifferentiationInterface.jl.

---

<div class="post-metadata">

**Author:** ![gdalle](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gdalle/32/224680_2.png) [@gdalle](https://discourse.julialang.org/u/gdalle)\
**Post date:** [October 28, 2024, 3:11pm UTC](https://discourse.julialang.org/t/compile-fastdifferentiation-derivatives-one-time-only/121816/22 "2024-10-28T15:11:05Z")

</div>

> [@abraemer](#):
>
> Is it though? The type `FunctionWithPreppedGradient` is owned by the user. So it is not type piracy in a strict sense I think. Depending on the dispatch rules inside `DifferentiationInterface` (which you are the expert of 🙂 ) it could cause method ambiguity problems and perhaps the signature needs to be more specific.

Sorry yes, not piracy but it will definitely cause ambiguities because DI specializes on the backend and not the function, whereas this goes the other way.

---

<div class="post-metadata">

**Author:** ![abraemer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/abraemer/32/51403_2.png) [@abraemer](https://discourse.julialang.org/u/abraemer)\
**Post date:** [October 28, 2024, 5:53pm UTC](https://discourse.julialang.org/t/compile-fastdifferentiation-derivatives-one-time-only/121816/23 "2024-10-28T17:53:50Z")

</div>

I think you could make this type `FunctionWithPreppedGradient` in a way that it lazily saves the prepared hessian/gradient/etc on first call. Then it would not be very cumbersome I think? You’d just have to wrap each of the function into that struct and the rest of the code would look like the Tensorial.jl code, no? I don’t think you could get any closer - at least not easily.

---

<div class="post-metadata">

**Author:** ![gdalle](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gdalle/32/224680_2.png) [@gdalle](https://discourse.julialang.org/u/gdalle)\
**Post date:** [October 28, 2024, 8:45pm UTC](https://discourse.julialang.org/t/compile-fastdifferentiation-derivatives-one-time-only/121816/24 "2024-10-28T20:45:48Z")

</div>

But if your version with Tensorial’s custom differentiation operators works well, maybe there’s no need for more

---

<div class="post-metadata">

**Author:** ![KeitaNakamura](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/keitanakamura/32/211562_2.png) [@KeitaNakamura](https://discourse.julialang.org/u/KeitaNakamura)\
**Post date:** [October 30, 2024, 3:30am UTC](https://discourse.julialang.org/t/compile-fastdifferentiation-derivatives-one-time-only/121816/25 "2024-10-30T03:30:29Z")

</div>

Sorry for the late reply. If I understand correctly, I would still need to pass a `FunctionWithPreppedGradient` object instead of `prep` to each function. In that case, I don’t think it’s a fundamental solution, because if I want to call `func4` from within `func3` , for example, I would need to modify all the other functions as well. I think that structure is not good.

---

<div class="post-metadata">

**Author:** ![gdalle](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gdalle/32/224680_2.png) [@gdalle](https://discourse.julialang.org/u/gdalle)\
**Post date:** [October 30, 2024, 5:39am UTC](https://discourse.julialang.org/t/compile-fastdifferentiation-derivatives-one-time-only/121816/26 "2024-10-30T05:39:37Z")

</div>

> [@KeitaNakamura](#):
>
> In a macro, we cannot determine the size of `x`, so the approach does not work.

With a static array (or tensor) you can deduce the size from the type, so maybe a macro inside a generated function could be enough? I don’t know if that’s possible, just grasping at straws here.

---

<div class="post-metadata">

**Author:** ![KeitaNakamura](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/keitanakamura/32/211562_2.png) [@KeitaNakamura](https://discourse.julialang.org/u/KeitaNakamura)\
**Post date:** [October 31, 2024, 5:20am UTC](https://discourse.julialang.org/t/compile-fastdifferentiation-derivatives-one-time-only/121816/27 "2024-10-31T05:20:41Z")

</div>

Thank you always for your thoughtful suggestion. However, to construct the framework I want, it seems essential to evaluate the given function `f` at compile-time within the `derivative(f,x)` function, for example. This approach appears to be infeasible. Alternatively, the preparation process should return a bitstype similar to ForwardDiff.jl with `SArray` case.

---

<div class="post-metadata">

**Author:** ![moble](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/moble/32/23535_2.png) [@moble](https://discourse.julialang.org/u/moble)\
**Post date:** [November 12, 2024, 6:09am UTC](https://discourse.julialang.org/t/compile-fastdifferentiation-derivatives-one-time-only/121816/28 "2024-11-12T06:09:09Z")

</div>

Sorry to be late to the party, but I have experience with a very similar — though more complicated — problem. In fact, I think I might be the “someone” brianguenter mentioned in the context of `make_Expr`. Creating an `@generated` function that takes a `Val(func)` has worked out very well for me. The advice to avoid `@generated` functions should probably be more like “avoid them unless you understand exactly why you need them instead of regular functions.” But in this case, we do understand, and we still need them.

I should note that I’m actually not differentiating the `Val(func)`; I’m differentiating a function that depends on another type parameter (`PNOrder`). As mentioned above, you may need RuntimeGeneratedFunctions.jl to solve world-age issues. So I guess I’m saying, take this as a proof-of-principle example:

> <https://github.com/moble/PostNewtonian.jl/blob/e24356e1bff120d11a61faa14a734a3b0b599048/src/pn_expressions/binding_energy.jl#L153-L245>

[Previous page](https://discourse.julialang.org/t/compile-fastdifferentiation-derivatives-one-time-only/121816.md?page=1)