# What are \`autodiff\_deferred\` and \`autodiff\_thunk\` for in Enzyme?

**URL:** <https://discourse.julialang.org/t/what-are-autodiff-deferred-and-autodiff-thunk-for-in-enzyme/112065>\
**Category:** Machine Learning\
**Tags:** autodiff, enzyme, gradient\
**Created:** [March 25, 2024, 7:26am UTC](https://discourse.julialang.org/t/what-are-autodiff-deferred-and-autodiff-thunk-for-in-enzyme/112065 "2024-03-25T07:26:46Z")\
**Posts on this page:** 4\
**Page:** 2

<div class="post-metadata">

**Author:** ![gdalle](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gdalle/32/27854_2.png) [@gdalle](https://discourse.julialang.org/u/gdalle)\
**Post date:** [April 5, 2024, 7:09pm UTC](https://discourse.julialang.org/t/what-are-autodiff-deferred-and-autodiff-thunk-for-in-enzyme/112065/21 "2024-04-05T19:09:15Z")

</div>

> [@wsmoses](#):
>
> The thunks are stateless and can be called as many times as you like.
> 
> However, the extra “tape” (really value cache) for the reverse pass may be different, depending on the function (e.g. if a shadow pointer is captured/overwritten/etc).

Thanks! So, to to sum up, I can initialize

```julia
forw, rev = autodiff_thunk(ReverseSplitWithPrimal, Const{typeof(f)}, ...)

```

and use those two thunks as many times as I like _together_. However, as soon as I do

```julia
tape, y, shadow_y = forw(Const(f), ...)

```

the tape is stateful. This means that if I run

```julia
rev(Const(f), ..., tape)

```

once, I’m good, but if I call `rev` again, there’s no surefire way to trust the result. Correct?

> [@wsmoses](#):
>
> Of course use enzyme’s batchduplicated/etc to perform as many reverse passes in one reverse sweep as you want (thunk or no thunk).

Yeah I’m also thinking of adding that to DifferentiationInterface, but Enzyme is the only backend supporting it so we’ll see 🤷

---

<div class="post-metadata">

**Author:** ![vchuravy](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/vchuravy/32/8_2.png) [@vchuravy](https://discourse.julialang.org/u/vchuravy)\
**Post date:** [April 6, 2024, 4:38pm UTC](https://discourse.julialang.org/t/what-are-autodiff-deferred-and-autodiff-thunk-for-in-enzyme/112065/22 "2024-04-06T16:38:03Z")

</div>

Right essentially the thunks are necessary to implement Split-Mode, or to use Enzyme within ChainRules.

The reverse think is the pullback and the forward thunk is a yet not run forward function

---

<div class="post-metadata">

**Author:** ![wsmoses](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/wsmoses/32/26497_2.png) [@wsmoses](https://discourse.julialang.org/u/wsmoses)\
**Post date:** [April 6, 2024, 4:42pm UTC](https://discourse.julialang.org/t/what-are-autodiff-deferred-and-autodiff-thunk-for-in-enzyme/112065/23 "2024-04-06T16:42:45Z")

</div>

You may be able to query Julia’s effect analysis to see if something is captured.

Alternatively if desired enzyme can expose an API that says whether it is safe to redo with the same tape

---

<div class="post-metadata">

**Author:** ![gdalle](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gdalle/32/27854_2.png) [@gdalle](https://discourse.julialang.org/u/gdalle)\
**Post date:** [April 7, 2024, 5:37am UTC](https://discourse.julialang.org/t/what-are-autodiff-deferred-and-autodiff-thunk-for-in-enzyme/112065/24 "2024-04-07T05:37:00Z")

</div>

> [@vchuravy](#):
>
> or to use Enzyme within ChainRules.

Yeah but that’s related to my point: if I understand correctly, the pullback closure we generate that way is one-use only. So I think this is at odds with the ChainRules convention, where it can be reused

[Previous page](https://discourse.julialang.org/t/what-are-autodiff-deferred-and-autodiff-thunk-for-in-enzyme/112065.md?page=1)