RNN is not trained

Specific Domains Machine Learning
7

Here’s an example that has been a little bit reworked that looks to train properly:

using CSV
using DataFrames
using Flux
using Plots

df = DataFrame(CSV.File("airline-passengers.txt"))

# Get the two datasets of one point time-series
# The train data and target 
X_raw = Float32.(df[!,:Passengers][1:nrow(df)-1]) ./ 100
Y_raw = Float32.(df[!,:Passengers][2:nrow(df)]) ./ 100

Plots.plot(X_raw)
plot!(Y_raw)

X = [X_raw[i:i] for i in 1:length(X_raw)]
Y = Y_raw

# Model and Training 
model = Chain(RNN(1,4), Dense(4, 1), x -> reshape(x, :))
# model.(X)
loss(x, y) = sum(abs2.(Flux.stack(model.(x), 1) .- y))

ps = Flux.params(model);
opt = Flux.ADAM(0.01)
epochs = 100

for epoch in 1:epochs
        @show epoch
        gs = Flux.gradient(ps) do
             loss(X, Y)
        end
        Flux.Optimise.update!(opt, ps, gs)
end

out = vec(Flux.stack(model.(X), 1))
Plots.plot(out)
plot!(Y)

Something that may have contributed to the issue is the scaling the of the features/target. Here I roughly divided each by 100.

1 Like