Same response of NODE for different parameter vectors despite nn results are different

General Usage
,
1

Hi,

I’m getting sometimes the same predictions result of my node model for different parameter vectors despite that the neural network return different outputs for the different parameter vectors. Can somebody tell me what I’ve done wrong?

test nn output

nn_output1= nn_model(vcat(u0[1], u0[2], vec(ex[Int(round(10.0 * 0.1)), :])) , p_ini)
nn_output2 = nn_model(vcat(u0[1], u0[2], vec(ex[Int(round(10.0 * 0.1)), :])) , p_test)

# node output 
y_model1 = predict_neuralode_rk4(p_ini)
y_model2 = predict_neuralode_rk4(p_test)

p_ini == p_test => false
nn_output1==nn_output2 => false
y_model1 == y_model2 => true

Complete code:

using DifferentialEquations,  DiffEqFlux

# excitation and response data
ex = rand(Float32, 38, 4)
y = rand(Float32, 2, 38)

# Initial condition
u0 = [y[1,1], y[2,1]]

idxs = [size(ex, 1)]

sim_start = 1
sim_length = idxs[1]

tspan = (Float32(sim_start*0.1), Float32(sim_length*0.1))
tsteps = range(tspan[1], tspan[2], length = sim_length-sim_start+1)

nn_model = FastChain(FastDense(6,8, sigmoid), FastDense(8, 2))
p_ini = initial_params(nn_model)
nn_model = FastChain(FastDense(6,8, sigmoid), FastDense(8, 2))
p_test = initial_params(nn_model)

# test parameters
p_test==p_ini

# test nn output
nn_output1= nn_model(vcat(u0[1], u0[2], vec(ex[Int(round(10.0 * 0.1)), :])) , p_ini)  
nn_output2 = nn_model(vcat(u0[1], u0[2], vec(ex[Int(round(10.0 * 0.1)), :])) , p_test)  

nn_output1==nn_output2

## uode
dudt(u, p, t) = nn_model(vcat(u[1], u[2], vec(ex[Int(round(10.0 * t)), :])) , p)    

# define ode prob
global prob = ODEProblem(dudt, u0, tspan, nothing)

# prediction function
predict_neuralode_rk4(p) =  Array(solve(remake(prob,p=p), RK4(), dt = 0.01, saveat=tsteps, adaptive= false))

y_model1 = predict_neuralode_rk4(p_ini)
y_model2 = predict_neuralode_rk4(p_test)

p_ini == p_test
y_model1 == y_model2
2

I can’t reproduce. When I run your script from start to finish I get:

julia> y_model1 = predict_neuralode_rk4(p_ini)
2×38 Matrix{Float32}:
 0.747323  0.674339  0.600233  0.531114  0.456608  0.379255  0.310249  0.249577  0.180026  …  -1.3501    -1.41294   -1.47288  -1.52799   -1.57535   -1.62621  -1.68573     
 0.205668  0.199354  0.193894  0.188424  0.183375  0.178696  0.1712    0.164441  0.157471     -0.221071  -0.246768  -0.27372  -0.297929  -0.318126  -0.34017  -0.364195    

julia> y_model2 = predict_neuralode_rk4(p_test)
2×38 Matrix{Float32}:
 0.747323  0.859295  0.974333  1.08831   1.20072   1.32124   1.44831   1.5701    1.69544   …  4.97717  5.14622   5.31484   5.48256  5.65342  5.82341  5.99101  6.15886
 0.205668  0.222905  0.238186  0.253739  0.275671  0.296184  0.313948  0.333725  0.356905     0.92373  0.954812  0.988091  1.01951  1.05461  1.09325  1.12895  1.1609

so the NODE’s response is different between the two as expected. Try running it from a clean new REPL