I stumbled upon this very nice example from @SebastianCallh in which Neural ODE are used to model and forecast the weather in the city of Delhi. I have a keen interest in learning how to apply differential equations using Julia, so I thought I could give it a try and adapt this code to work on my data.
Took some time to understand the basics of Julia (first timer coming from Python!), but in the end I managed to use @SebastianCallh example to put everything together as follows:
using Dates using Plots using Plotly using DiffEqFlux using Statistics using DataFrames, CSV using OrdinaryDiffEq, Flux, Random function neural_ode(t, data_dim; saveat = t) f = FastChain(FastDense(data_dim, 64, swish), FastDense(64, 32, swish), FastDense(32, data_dim)) node = NeuralODE(f, (minimum(t), maximum(t)), Tsit5(), saveat = saveat, abstol = 1e-9, reltol = 1e-9) end function train(node, θ, y, opt, maxiters, y0 = y[:, 1]; kwargs...) predict(θ) = Array(node(y0, θ)) loss(θ) = begin ŷ = predict(θ) Flux.mse(ŷ, y) end θ = θ == nothing ? node.p : θ res = DiffEqFlux.sciml_train( loss, θ, opt, maxiters = maxiters; kwargs... ) return res.minimizer end log_results(θs, losses) = (θ, loss) -> begin push!(θs, copy(θ)) push!(losses, loss) false end ################ # Main program # ################ # Loading data and setting up metadata meta_cols = ["rowid", "SiteId", "datetime", "longitude", "latitude"] data_cols = ["datetime", "DryBulbTemperature_Celsius"] # Loading 168 records of hourly temperature for a given week in July 2018 # Note: Contains 'missing' values that are casted to NaN and then ignored with filter(!isnan, ) path_in = "PATH/week_of_hourly_temperature_records.csv" df = DataFrame!(CSV.read(path_in))[data_cols] df["DryBulbTemperature_Celsius"] = replace(df["DryBulbTemperature_Celsius"], missing=>NaN) # Normalizing and splitting on train/test T = 72 # first 3 days for training, remaining 4 days for testing mean_temp = mean(filter(!isnan, df["DryBulbTemperature_Celsius"])) # taking care of NaNs stdev_temp = std(filter(!isnan, df["DryBulbTemperature_Celsius"])) y = Matrix(df[:, ["DryBulbTemperature_Celsius"]] |> y -> Matrix(y)' |> y -> (y .- mean_temp) ./ stdev_temp) train_dates, test_dates = df["datetime"][1:T], df["datetime"][T:end] vals_train_norm, vals_test_norm = y[1:T], y[T:end] # Starting with the analysis Random.seed!(1); maxiters = 150 θs, losses = ,  θ = nothing num_obs = 4:4:length(vals_train_norm) for k in num_obs chunk = filter(!isnan, vals_train_norm[1:k]) node = neural_ode(chunk, size(y, 1)) θ = train( node, θ, chunk, ADAMW(8e-3), maxiters; cb = log_results(θs, losses) ) end
However, I found this persistent error and I don’t know how to fix it:
ERROR: LoadError: **UndefVarError: θ not defined** Stacktrace:  top-level scope at /PATH/JuliaProjects/basics_of/attempt.jl:87 in expression starting at /PATH/JuliaProjects/basics_of/attempt.jl:78
Because this θ is first set to “nothing” and it seems that will become iteratively calculated/updated. So it seems it makes sense to set up the variable with and empty value and let it change. I printed “node” and “f” and they contain values, so something starts happening, but then it crashes. I even changed the Greek symbol to “theta” just in case there was some problem there , but nothing.
Please, could you let me know what is missing in this code? How can I “define” theta properly? Perhaps an extra library import is required? Maybe the author can share the version creating these nice plots as well?
Thanks for your help!