# Using Flux for a neural net solution to differential equations

**URL:** <https://discourse.julialang.org/t/using-flux-for-a-neural-net-solution-to-differential-equations/41996>\
**Category:** General Usage\
**Created:** [June 24, 2020, 4:02pm UTC](https://discourse.julialang.org/t/using-flux-for-a-neural-net-solution-to-differential-equations/41996 "2020-06-24T16:02:00Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![RuckerC](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ruckerc/32/16095_2.png) [@RuckerC](https://discourse.julialang.org/u/RuckerC)\
**Post date:** [June 24, 2020, 4:02pm UTC](https://discourse.julialang.org/t/using-flux-for-a-neural-net-solution-to-differential-equations/41996/1 "2020-06-24T16:02:00Z")

</div>

I have recently started using Flux to construct a physics informed neural network. To do this, I start with a model

`u = Chain(Dense(1, 10), Dense(10,15), Dense(15,1))`

Then I construct a function to impose the differential condition y’(t) - y(t) = 0

> function f(t)  
> y = sum(u(t))  
> dy = gradient(() → u(t)[1], params(t))  
> f = dy[t][1] - y  
> end

with cost function

`cost(t) = abs(f(t))^2`

I would like to minimize this cost with respect to the parameters of u but when I call

`grad = gradient(() -> cost(t), params(u))`

I get a long stacktrace starting with “ERROR: Can’t differentiate foreigncall expression”. The full stacktrace is in this [gist](https://gist.github.com/cody-rucker/358d3ae7c63fa310f92debfef1893613).  
I am having troubles figuring out what is causing the error. I want to be able to train a network u(x) to respect a condition u’(x) - u(x) = 0 but don’t know how to specify this with Flux. If anyone has any suggestions I would greatly appreciate it.

---

<div class="post-metadata">

**Author:** ![contradict](https://avatars.discourse-cdn.com/v4/letter/c/ac91a4/32.png) [@contradict](https://discourse.julialang.org/u/contradict)\
**Post date:** [June 24, 2020, 4:22pm UTC](https://discourse.julialang.org/t/using-flux-for-a-neural-net-solution-to-differential-equations/41996/2 "2020-06-24T16:22:29Z")

</div>

I believe the error is from the Dict access on the last line of `f()`. `dy[t]`, finding the `t` derivative of `dy` looks like it does a dictionary lookup. I am surprised this doesn’t work, but I don’t know how to fix it either.

Rather than using a derivative constraint, you might try using an integral constraint with a differential equation solver. The [DiffEqFlux.jl](https://julialang.org/blog/2019/01/fluxdiffeq/) package was written for this kind of exploration and might give you some ideas.

---

<div class="post-metadata">

**Author:** ![RuckerC](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ruckerc/32/16095_2.png) [@RuckerC](https://discourse.julialang.org/u/RuckerC)\
**Post date:** [June 24, 2020, 4:25pm UTC](https://discourse.julialang.org/t/using-flux-for-a-neural-net-solution-to-differential-equations/41996/3 "2020-06-24T16:25:14Z")

</div>

Alright I will check out DiffEqFlux.jl, thank you!

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [June 25, 2020, 2:55am UTC](https://discourse.julialang.org/t/using-flux-for-a-neural-net-solution-to-differential-equations/41996/4 "2020-06-25T02:55:58Z")

</div>

> [@RuckerC](#):
>
> I have recently started using Flux to construct a physics informed neural network. To do this, I start with a model

Look at NeuralNetDiffEq.jl which is a repo of physics-informed neural network implementations and deep BSDE methods.

> **[GitHub - SciML/NeuralPDE.jl: Physics-Informed Neural Networks (PINN) and Deep...](https://github.com/SciML/NeuralPDE.jl)**
>
> Physics-Informed Neural Networks (PINN) and Deep BSDE Solvers of Differential Equations for Scientific Machine Learning (SciML) accelerated simulation - GitHub - SciML/NeuralPDE.jl: Physics-Informe...

---

<div class="post-metadata">

**Author:** ![Elmo](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/elmo/32/17979_2.png) [@Elmo](https://discourse.julialang.org/u/Elmo)\
**Post date:** [January 13, 2021, 4:16pm UTC](https://discourse.julialang.org/t/using-flux-for-a-neural-net-solution-to-differential-equations/41996/5 "2021-01-13T16:16:12Z")

</div>

Hi all, I also ran into this issue. I opened an issue on Flux and they solved it, see [here](https://discourse.julialang.org/t/gradient-error-in-flux-model-inputs/53259) for my discourse question and [here](https://github.com/FluxML/Flux.jl/issues/1464) for the answered flux issue.