# Is it possible to do Nested AD ~elegantly~ in Julia? (PINNs)

**URL:** <https://discourse.julialang.org/t/is-it-possible-to-do-nested-ad-elegantly-in-julia-pinns/98888>\
**Category:** General Usage\
**Tags:** machine-learning\
**Created:** [May 15, 2023, 5:12pm UTC](https://discourse.julialang.org/t/is-it-possible-to-do-nested-ad-elegantly-in-julia-pinns/98888 "2023-05-15T17:12:01Z")\
**Posts on this page:** 1\
**Showing post:** 39

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [June 2, 2023, 8:48pm UTC](https://discourse.julialang.org/t/is-it-possible-to-do-nested-ad-elegantly-in-julia-pinns/98888/39 "2023-06-02T20:48:18Z")

</div>

If you’re using 32 bit floats, that’s way too small of an epsilon. You may want to see this introduction to automatic differentiation which describes why in detail:

[![](https://global.discourse-cdn.com/julialang/original/3X/c/5/c558ba3c55176136f6a071e2309f37df18dedf77.jpeg "Forward-Mode Automatic Differentiation (AD) via High Dimensional Algebras") ](https://www.youtube.com/watch?v=zHPXGBiTM5A)

So notice that:

```julia
julia> eps(Float32)
1.1920929f-7

```

which means that your derivative term, when `1e-6` is effectively “holding” the value in its 7th decimal place onwards, but you only have 7 decimal places, and thus the accuracy of your derivative will not be more than one digit. So from quick pen and paper, it’s not surprising it’s different in the second decimal place. You’ll do a bit better then to use half of the digits for the differentiation, or in other words, use epsilon as:

```julia
julia> sqrt(eps(Float32))
0.00034526698f0

```

If you really need to use Float32, you may want to use central differencing to improve this a bit (and change epsilon to `cbrt(eps(Float32))`)

---

_[View the full topic](https://discourse.julialang.org/t/is-it-possible-to-do-nested-ad-elegantly-in-julia-pinns/98888)._