# Gradient calculation in PINN

**URL:** <https://discourse.julialang.org/t/gradient-calculation-in-pinn/61525>\
**Category:** General Usage\
**Tags:** flux, zygote\
**Created:** [May 20, 2021, 3:48pm UTC](https://discourse.julialang.org/t/gradient-calculation-in-pinn/61525 "2021-05-20T15:48:12Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![JunichiFukui](https://avatars.discourse-cdn.com/v4/letter/j/ad7895/32.png) [@JunichiFukui](https://discourse.julialang.org/u/JunichiFukui)\
**Post date:** [May 20, 2021, 3:48pm UTC](https://discourse.julialang.org/t/gradient-calculation-in-pinn/61525/1 "2021-05-20T15:48:12Z")

</div>

Hi, everyone,

I am trying to build a Physics Informed Neural Network (PINN) for solving the Navier-Stokes equations like NVidia SimNet using Flux. The input of the neural network is the position `X_i`, `Y_i`, `Z_i` of each points in 3D space and the output is the velocity `Ux_i`, `Uy_i`, `Uz_i` and pressure `P_i` at each points. I need to check if the output from the neural network satisfies the Navier-Stokes equations. How can I calculate the first derivative of velocity `∂U_i/∂X_i` and pressure `∂P/∂X_i`, and second derivative of velocity `∂^2U_i/∂X_i^2` included in the Navier-Stokes equations by automatic differentiation using Zygote?

In other PINN programs using TensorFlow, the gradients are calculated in `tf.gradients` as follows

```
u_x = tf.gradients(u, x)[0]
u_y = tf.gradients(u, y)[0]
v_x = tf.gradients(v, x)[0]
v_y = tf.gradients(v, y)[0]

```

However, since `tf.gradients` calculates the gradient of the sum of the functions `grad(sum(f(x)))`, I assume that it does not correctly calculate the gradient of the physical quantity at each point.

---

<div class="post-metadata">

**Author:** ![JunichiFukui](https://avatars.discourse-cdn.com/v4/letter/j/ad7895/32.png) [@JunichiFukui](https://discourse.julialang.org/u/JunichiFukui)\
**Post date:** [May 21, 2021, 4:30am UTC](https://discourse.julialang.org/t/gradient-calculation-in-pinn/61525/2 "2021-05-21T04:30:59Z")

</div>

I may have found the information I was looking for on how to find the first derivative in the discussion shown in the link below.

[Gradient of Flux model wrt to weights](https://discourse.julialang.org/t/gradient-of-flux-model-wrt-to-weights/61405)

Based on this information, the part of the PINN programme I would like to create would look like this.

```julia
using Flux
model = Flux.Dense(24,32)
baseline = Flux.params(model)
input = [-1 1 -1 1 -1 1 -1 1 -1 -1 1 1 -1 -1 1 1 -1 -1 -1 -1 1 1 1 1]
output = model(input)
grad = Flux.jacobian(x -> model(x), input)

```

In the program, `input` is location of 8 points in 3-dimensional domain (X[-1, 1], Y[-1, 1], Z[-1, 1]), and `output` is velocity in X, Y, Z direction and pressure at the each points.

I am looking for a way to calculate the second order derivative of the output to the input of this neural network, and when I use the Hessian function I get the following error message

```julia
grad2 = Flux.hessian(x -> model(x), some_input)
ERROR: output an array, so the gradient is not defined. Perhaps you wanted jacobian.

```

Could you tell me how to solve this error?

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [June 2, 2021, 5:08pm UTC](https://discourse.julialang.org/t/gradient-calculation-in-pinn/61525/3 "2021-06-02T17:08:05Z")

</div>

The right answer is Diffractor, or to use the modified getindex gradients of Flux. NeuralPDE.jl uses some tricks in the meantime.

---

<div class="post-metadata">

**Author:** ![Khemraj\_Shukla](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/khemraj_shukla/32/19917_2.png) [@Khemraj\_Shukla](https://discourse.julialang.org/u/Khemraj_Shukla)\
**Post date:** [June 3, 2021, 6:00am UTC](https://discourse.julialang.org/t/gradient-calculation-in-pinn/61525/4 "2021-06-03T06:00:00Z")

</div>

@ChrisRackauckas Could you please provide slightly more detail or reference about Diffractor?

---

<div class="post-metadata">

**Author:** ![JunichiFukui](https://avatars.discourse-cdn.com/v4/letter/j/ad7895/32.png) [@JunichiFukui](https://discourse.julialang.org/u/JunichiFukui)\
**Post date:** [June 3, 2021, 1:18pm UTC](https://discourse.julialang.org/t/gradient-calculation-in-pinn/61525/5 "2021-06-03T13:18:04Z")

</div>

Hi Chris,

Thank you for your reply.

Diffractor.jl doesn’t seem to be available to the public yet, when will it be?

I still don’t understand the details of your second solution, “to use the modified getindex gradients of Flux”, but I will try to read the source code of NeuralPDE.jl.

---

<div class="post-metadata">

**Author:** ![Anupama](https://avatars.discourse-cdn.com/v4/letter/a/2bfe46/32.png) [@Anupama](https://discourse.julialang.org/u/Anupama)\
**Post date:** [January 16, 2023, 6:02pm UTC](https://discourse.julialang.org/t/gradient-calculation-in-pinn/61525/6 "2023-01-16T18:02:50Z")

</div>

@JunichiFukui Did you figure out how can we calculate the gradient of the physics at each point?