# Mixed NN and functions fail with NeuralPDE
**URL:** https://discourse.julialang.org/t/mixed-nn-and-functions-fail-with-neuralpde/103447
**Category:** Modelling & Simulations
**Tags:** question
**Created:** [September 1, 2023, 2:53pm UTC](https://discourse.julialang.org/t/mixed-nn-and-functions-fail-with-neuralpde/103447 "2023-09-01T14:53:56Z")
**Posts on this page:** 8
**Page:** 1
### Author:  [@Knud\_Sorensen](https://discourse.julialang.org/u/Knud_Sorensen)
#### Post date: [September 1, 2023, 2:53pm UTC](https://discourse.julialang.org/t/mixed-nn-and-functions-fail-with-neuralpde/103447/1 "2023-09-01T14:53:56Z")
Hi.
I’m trying to use NeuralPDE to approximate a solution to a PDE containing both Neural nets and functions. Something goes wrong, can you help ? I am new to NeuralPDE, so it might just be a newbie error.
Here is the code:
```plaintext
import ModelingToolkit: Interval
@parameters x,y,z
L=5
domains = [x ∈ Interval(-L, +L),
y ∈ Interval(-L, +L),
z ∈ Interval(-L, +L)]
@variables G(..)
Fx0(x,y,z)= -sin(atan(y,x))*exp(-(sqrt(x*x+y*y)-2.0)^2-z*z)
Fy0(x,y,z)= cos(atan(y,x))*exp(-(sqrt(x*x+y*y)-2.0)^2-z*z)
Fz0(x,y,z)= 0.0
Dx = Differential(x)
Dy = Differential(y)
Dz = Differential(z)
eqs0=[Dy(Fz0(x,y,z))-Dz(Fy0(x,y,z))-Dx(G(x,y,z))~0,
Dz(Fx0(x,y,z))-Dy(Fz0(x,y,z))-Dy(G(x,y,z))~0,
Dx(Fy0(x,y,z))-Dy(Fx0(x,y,z))-Dz(G(x,y,z))~0]
bcs0 = [G(-L,y,z)~G(L,y,z), G(x,-L,z)~G(x,L,z),G(x,y,-L)~G(x,y,L),
Dx(G(-L,y,z))~Dx(G(L,y,z)), Dy(G(x,-L,z))~Dy(G(x,L,z)),Dz(G(x,y,-L))~Dz(G(x,y,L))]
input_ = length(domains)
n = 15
chain = Lux.Chain(Dense(input_, n, Lux.asinh), Dense(n, n, Lux.asinh), Dense(n, 1))
strategy = QuadratureTraining()
discretization = PhysicsInformedNN(chain, strategy)
@named pdesystem = PDESystem(eqs0, bcs0, domains, [x,y,z], G(x,y,z))
prob = discretize(pdesystem, discretization)
sym_prob = symbolic_discretize(pdesystem, discretization)
pde_inner_loss_functions = sym_prob.loss_functions.pde_loss_functions
bcs_inner_loss_functions = sym_prob.loss_functions.bc_loss_functions
callback = function (p, l)
print("loss: ", l)
print(" pde: ", map(l_ -> l_(p), pde_inner_loss_functions))
println(" bcs: ", map(l_ -> l_(p), bcs_inner_loss_functions))
return false
end
res = Optimization.solve(prob, Adam(0.5); callback = callback, maxiters = 5)
```
I get the following output:
````julia
loss: NaN pde: [4.9889631620980905, 4.507107273136388, NaN] bcs: [0.0, 0.0, 0.0, 0.0, 0.0, 0.0]
loss: NaN pde: [NaN, NaN, NaN] bcs: [NaN, NaN, NaN, NaN, NaN, NaN]
loss: NaN pde: [NaN, NaN, NaN] bcs: [NaN, NaN, NaN, NaN, NaN, NaN]
loss: NaN pde: [NaN, NaN, NaN] bcs: [NaN, NaN, NaN, NaN, NaN, NaN]
loss: NaN pde: [NaN, NaN, NaN] bcs: [NaN, NaN, NaN, NaN, NaN, NaN]
loss: Inf pde: [4.9889631620980905, 4.507107273136388, NaN] bcs: [0.0, 0.0, 0.0, 0.0, 0.0, 0.0]```
````
---
### Author:  [@baggepinnen](https://discourse.julialang.org/u/baggepinnen)
#### Post date: [September 1, 2023, 4:02pm UTC](https://discourse.julialang.org/t/mixed-nn-and-functions-fail-with-neuralpde/103447/2 "2023-09-01T16:02:43Z")
Are you perhaps sending values to `atan` that are out of domain?
---
### Author:  [@Knud\_Sorensen](https://discourse.julialang.org/u/Knud_Sorensen)
#### Post date: [September 1, 2023, 4:46pm UTC](https://discourse.julialang.org/t/mixed-nn-and-functions-fail-with-neuralpde/103447/3 "2023-09-01T16:46:41Z")
The domain for the atan I use is all real numbers. So, it doesn’t seem to be the problem.
---
### Author:  [@avikpal](https://discourse.julialang.org/u/avikpal)
#### Post date: [September 2, 2023, 3:40am UTC](https://discourse.julialang.org/t/mixed-nn-and-functions-fail-with-neuralpde/103447/4 "2023-09-02T03:40:06Z")
You could use the new `DebugLayer` in Lux to try and see if the NaNs star appearing inside the Lux layer [Debugging Lux Models | LuxDL Docs](https://lux.csail.mit.edu/dev/manual/debugging#tracking-down-nans)
---
### Author:  [@Knud\_Sorensen](https://discourse.julialang.org/u/Knud_Sorensen)
#### Post date: [September 4, 2023, 9:39am UTC](https://discourse.julialang.org/t/mixed-nn-and-functions-fail-with-neuralpde/103447/5 "2023-09-04T09:39:56Z")
When I try to use it I get the error:
````julia
julia> debug_chain=Lux.Experimental.@debug_mode chain
ERROR: LoadError: UndefVarError: `@debug_mode` not defined```
````
---
### Author:  [@Knud\_Sorensen](https://discourse.julialang.org/u/Knud_Sorensen)
#### Post date: [September 4, 2023, 10:09am UTC](https://discourse.julialang.org/t/mixed-nn-and-functions-fail-with-neuralpde/103447/6 "2023-09-04T10:09:10Z")
After updating Lux it seems to work.
---
### Author:  [@Knud\_Sorensen](https://discourse.julialang.org/u/Knud_Sorensen)
#### Post date: [September 4, 2023, 11:50am UTC](https://discourse.julialang.org/t/mixed-nn-and-functions-fail-with-neuralpde/103447/7 "2023-09-04T11:50:00Z")
After runing it I get ´´´
“NaNs detected in pullback output for Dense(15 =\> 1) at location chain.layers.layer\_3!”)```
So, it seems to appear in the start of the backwards pass.
Is there a way to follow the Nans in the solve function ?
Or test the loss function or the differentiated loss function ?
---
### Author:  [@Knud\_Sorensen](https://discourse.julialang.org/u/Knud_Sorensen)
#### Post date: [September 5, 2023, 3:13pm UTC](https://discourse.julialang.org/t/mixed-nn-and-functions-fail-with-neuralpde/103447/8 "2023-09-05T15:13:04Z")
The differential of the functions Fx0 and Fy0 have a singularity at (x,y) = (0,0).
Is that a way to remove that line form the domain ?
Something like ```
```julia
domains = [x ∈ Interval(-L, +L),
y ∈ Interval(-L, +L),
z ∈ Interval(-L, +L),
(x,y) ∉ (0,0)]
```
But just works.