# NaN errors in Flux

**URL:** <https://discourse.julialang.org/t/nan-errors-in-flux/80092>\
**Category:** General Usage\
**Tags:** flux\
**Created:** [April 26, 2022, 9:49pm UTC](https://discourse.julialang.org/t/nan-errors-in-flux/80092 "2022-04-26T21:49:34Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Christopher\_Fisher](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/christopher_fisher/32/26132_2.png) [@Christopher\_Fisher](https://discourse.julialang.org/u/Christopher_Fisher)\
**Post date:** [April 26, 2022, 9:49pm UTC](https://discourse.julialang.org/t/nan-errors-in-flux/80092/1 "2022-04-26T21:49:34Z")

</div>

Hi all,

I am trying to replicate a [Likelihood Approximation Network](https://elifesciences.org/articles/65074#s4) (LAN) with Flux.jl. LANs are used to learn the likelihood function of intractable computational models. As a proof of concept, I am trying to apply the method to two simple models for which the likelihood function is known: a Gaussian model and a decision model called the Linear Ballistic Accumulator (LBA). I was successful in developing a LAN for the [Gaussian model](https://github.com/itsdfish/LANSandBox.jl/blob/main/examples/gaussian/run_gaussian_lan.jl), but the LAN for the [LBA](https://github.com/itsdfish/LANSandBox.jl/blob/main/examples/LBA/run_lba_lan.jl) produces NaNs as predictions. I tried various solutions from other threads, such as decreasing the learning rate and using `BatchNorm`, but those recommendations did not solve the problem. Changing the activation function to `relu`, solved the `NaN` problem, but interfered with the ability of the NN to learn the likelihood function.

Can I do anything to fix this problem? Please let me know if there are more details I can provide.

---

<div class="post-metadata">

**Author:** ![goerch](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/goerch/32/29122_2.png) [@goerch](https://discourse.julialang.org/u/goerch)\
**Post date:** [April 26, 2022, 9:53pm UTC](https://discourse.julialang.org/t/nan-errors-in-flux/80092/2 "2022-04-26T21:53:25Z")

</div>

Just to reiterate an old point of discussion: [signaling NaNs](https://discourse.julialang.org/t/support-for-signaling-nans/75571) would help you to discover the root of the problem…

---

<div class="post-metadata">

**Author:** ![Christopher\_Fisher](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/christopher_fisher/32/26132_2.png) [@Christopher\_Fisher](https://discourse.julialang.org/u/Christopher_Fisher)\
**Post date:** [April 26, 2022, 10:59pm UTC](https://discourse.julialang.org/t/nan-errors-in-flux/80092/3 "2022-04-26T22:59:07Z")

</div>

Indeed, it would be helpful to know where the NaN originated. As far as I can tell, I replicated the procedure described in the paper. This makes me wonder whether there is a problem with the AD.

---

<div class="post-metadata">

**Author:** ![Christopher\_Fisher](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/christopher_fisher/32/26132_2.png) [@Christopher\_Fisher](https://discourse.julialang.org/u/Christopher_Fisher)\
**Post date:** [April 27, 2022, 4:45pm UTC](https://discourse.julialang.org/t/nan-errors-in-flux/80092/4 "2022-04-27T16:45:10Z")

</div>

I was wondering whether someone might be able to tell me if I misspecified the NN model or if NaNs are likely due to a bug?

---

<div class="post-metadata">

**Author:** ![Christopher\_Fisher](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/christopher_fisher/32/26132_2.png) [@Christopher\_Fisher](https://discourse.julialang.org/u/Christopher_Fisher)\
**Post date:** [April 27, 2022, 10:33pm UTC](https://discourse.julialang.org/t/nan-errors-in-flux/80092/5 "2022-04-27T22:33:04Z")

</div>

I tracked down the problem to a few large outliers in the training data, which caused a NaN when passed to `tanh`. So far, removing the outliers seems to have solved the problem.