Approximating sine function using Flux for Solving PINN problem

Specific Domains Machine Learning
1

In this code I am trying to predict u(x,t)= NN(\theta) so that after I predict this I can use the neural network to solve PINN problems. However, I’m not able to train the model and reduce the loss function.

import Flux, DifferentialEquations, Plots, DiffEqFlux
using Flux, DifferentialEquations, Plots, DiffEqFlux
x=0:0.01:1
y=sin.(2*pi*x); # actual Data

# initializing random parameters
W1=rand(10,1)
b1=rand(10)

W2=rand(5,10)
b2=rand(5)

W3=rand(1,5)
b3=rand(1)

# Defining Layers 
L1(x)=σ.(W1*x+b1)
L2(x)=σ.(W2*x+b2)
L3(x)=σ.(W3*x+b3)

# predicting the outcome
predict(x)=L3(L2(L1(x)));
yp=[predict.(i) for i in x] # stores all the values in yp of individual vector matrices
yp1=mapreduce(vec,vcat,yp); # this converts vector matrices of 1*1 to n*1 size


# loss function 

function loss_p()
   loss= sum(abs2,yp1.-y)
    return loss
end
loss_p()

# now we train our loss function
data= Iterators.repeated((),200)
opt=ADAM(0.1)
Flux.train!(loss_p,params(predict()),data,opt)

Furthermore, Instead If ,I use the commands in julia i.e
example: m=chain(dense(…)), will I be able to differentiate wrt x and t to solve 1D convection equation using PINN? Please help me out to solve the problem using the approach that I have used, and suggestions would help the most.
Thank you.

2

predict() isn’t a functor, so I’m pretty sure params(predict()) is empty. In this form, you’d want to directly do params([W1,b1,W2,b2,W3,b3])

1 Like
3

yes sir, I’ll try that. Thank you

4

Sir, this worked out well, however, even after training the model the error hasn’t reduced. The parameters haven’t updated.

5

What did you run?

6

Sir the params([w1,b1,w2,b2,w3,b3]) gave the parameters of the neural network for 1 iteration. So, from the loss function defined above , I tried to iterate by updating the parameters for using

p=params([w1,b1,w2,b2,w3,b3])

Flux.train!(loss_p,p=p,data,opt). However, I have received the same loss function value as before.

Furthermore, Is there any alternate way to solve PINN problems apart from using NeuralPDE.jl? I mean in a step by step way.
Eg: 1D convection equation

  1. creating u(t,x) using Flux.jl where u is a neural network
  2. Differentiating ‘u’ w.r.t time and space coordinates. in (DifferentialEquations.jl)
  3. Then defining the loss function so that we get same solution similar to the way we solve using NeuralPDE.jl

I thought solving in this way will make me better at using julia and understanding in depth.
Thank you.