Unpredictable learning of neural network based controller for dc motor


I am new to the Julia language. I am trying to make neural network based position controller using Julia libraries. My code is given below.

J = 0.1999
Ke = 2.1918
Kt = 38.185
L = 0.020453
R = 3.9694
Tq = 0.20142
b = 0.37219

function dc_motor(du, u, h, p, t)
    i, w, pos, pd, pdd, vin = u

    #vin = p(t)

    hd1 = h(p, t - 0.01)[2]
    hd2 = h(p, t - 0.02)[2]

    du[1] = (vin - R*i - Ke*w)/L

    du[2] = (Kt*i - b*w - Tq)/J

    du[3] = w

    du[4] = hd1

    du[5] = hd2


function dc_motor_controller(du, u, h, p, t)
    voltage = 0
    dc_motor(du, u[1:6], h, t->voltage, t)

controller = FastChain((x, p) -> x, FastDense(4, 32, tanh), FastDense(32, 16, tanh), FastDense(16, 1))
weights = initial_params(controller)

h(p, t) = zeros(6)
lags = [0.02]

u0 = [0.0, 0.0, 0.0, 0.0, 0.0, 0.0]
params  = weights
tspan = (0.0, 0.5)
N = 50
tsteps = range(tspan[1], length = N, tspan[2])
dt = (tspan[2] - tspan[1]) / N

prob = DDEProblem(dc_motor_controller, u0, h, tspan, params; constant_lags=lags)

function control_loop_new!(integrator)

    voltage = controller([integrator.u[3], integrator.u[4], integrator.u[5], integrator.p[end]], integrator.p[1:end-1])[1]

    if abs(voltage) >= 12
        voltage = 12* (abs(voltage)/voltage)

    integrator.u[6] = voltage


cb_controller = DiscreteCallback(condition, control_loop_new!)

function predict_neuralode(p, angle)
    u0 = [0.0, 0.0, 0.0, 0.0, 0.0, 0.0]
    tmp_prob = remake(prob, u0 = u0, p = vcat(p, angle))
    solve(tmp_prob, MethodOfSteps(Tsit5()), callback = cb_controller, tstops=timesteps, sensealg = ReverseDiffAdjoint(), )

function loss_neuralode(p)

    angle = deg2rad(45.0)
    pred = predict_neuralode(p, angle)
    pos = pred[3, :]
    voltage = pred[6, :]
    loss = sum((pos.-angle).^2)

    return loss, pred, pos, voltage, angle

index = 0 # training epoch counter

# callback function after each training epoch
callback = function (p, l, pred, pos, voltage, angle; doplot = true)
  global index += 1

  # ouput every few epochs
  if index % 50 == 0

    goal_position = ones(size(pred.t)[1])*rad2deg(angle)

    println("loss:", l," angle:", angle)
    plot(pred.t, rad2deg.(pos), label = ["position"])
    display(plot!(pred.t, goal_position, label = ["goal position"]))

    display(plot(pred.t, voltage, label = ["voltage"]))

  return false


result = DiffEqFlux.sciml_train(
  cb = callback,
  maxiters = 1500,

The problem is the controller learning it some times and not learning another time. I didn’t understand why it failing to learn sometimes. Is it because it need more iterations for learning a better control input.

The console log while controller learn to move towards the target position is given below.

`**loss:32.92571892751448 angle:0.7853981633974483
loss:32.627985828601574 angle:0.7853981633974483
loss:32.66226410447133 angle:0.7853981633974483
loss:32.567279853922514 angle:0.7853981633974483
loss:32.39154175827185 angle:0.7853981633974483
loss:32.81732092780774 angle:0.7853981633974483
loss:32.68729051161607 angle:0.7853981633974483
loss:32.74095819476493 angle:0.7853981633974483
loss:32.68435789697162 angle:0.7853981633974483
loss:33.09537254216951 angle:0.7853981633974483
loss:32.21350057852734 angle:0.7853981633974483
loss:33.71009342429282 angle:0.7853981633974483
loss:20.667760968747295 angle:0.7853981633974483
loss:20.65695448742362 angle:0.7853981633974483
loss:20.656843169457485 angle:0.7853981633974483
loss:20.65684311701577 angle:0.7853981633974483
loss:20.65684312504848 angle:0.7853981633974483
loss:20.6568431080726 angle:0.7853981633974483
loss:20.656843096951825 angle:0.7853981633974483
loss:20.65684310399092 angle:0.7853981633974483
loss:20.6568431122952 angle:0.7853981633974483
loss:20.656843098994212 angle:0.7853981633974483
loss:20.656843072282655 angle:0.7853981633974483
loss:20.65684304326965 angle:0.7853981633974483
loss:20.656843033931708 angle:0.7853981633974483
loss:20.656843053280774 angle:0.7853981633974483
loss:20.656843033519785 angle:0.7853981633974483**`

The console log while controller fail to move towards the target position is given below.

loss:32.641555388495185 angle:0.7853981633974483
loss:32.39069355004289 angle:0.7853981633974483
loss:32.43803035943642 angle:0.7853981633974483
loss:32.44203212699004 angle:0.7853981633974483
loss:32.325860035952495 angle:0.7853981633974483
loss:32.51288608170606 angle:0.7853981633974483
loss:32.4815533535547 angle:0.7853981633974483
loss:32.52985186829405 angle:0.7853981633974483
loss:32.46712305262051 angle:0.7853981633974483
loss:32.458913219436454 angle:0.7853981633974483
loss:32.445410725124525 angle:0.7853981633974483
loss:32.32018348205613 angle:0.7853981633974483
loss:32.38906862376797 angle:0.7853981633974483
loss:32.41120510538121 angle:0.7853981633974483
loss:32.48788975351786 angle:0.7853981633974483
loss:32.343650732335576 angle:0.7853981633974483
loss:32.30791129886353 angle:0.7853981633974483
loss:32.34361485838685 angle:0.7853981633974483
loss:32.328218206659635 angle:0.7853981633974483
loss:32.30852924469172 angle:0.7853981633974483
loss:32.345039342464716 angle:0.7853981633974483
loss:32.43607365411419 angle:0.7853981633974483
loss:32.36205242715388 angle:0.7853981633974483
loss:32.44062731662369 angle:0.7853981633974483
loss:32.50512118025965 angle:0.7853981633974483
loss:32.26274936510309 angle:0.7853981633974483
loss:32.30490991398758 angle:0.7853981633974483
loss:32.44811765382228 angle:0.7853981633974483
loss:32.48192486772814 angle:0.7853981633974483
loss:32.50304027000976 angle:0.7853981633974483

I really interested to investigate further deep into this issue. Any suggestions or help will be highly appreciated.

That would just mean that the learning rate might be too high.

Thanks for the suggestion. I will reduce the learning rate and examine the result.