# Forecasting time series data with neural ordinary differential equations in Julia
**URL:** https://discourse.julialang.org/t/forecasting-time-series-data-with-neural-ordinary-differential-equations-in-julia/92977
**Category:** General Usage
**Tags:** ode, machine-learning, sciml, ml, neural-network
**Created:** [January 15, 2023, 1:41am UTC](https://discourse.julialang.org/t/forecasting-time-series-data-with-neural-ordinary-differential-equations-in-julia/92977 "2023-01-15T01:41:17Z")
**Posts on this page:** 20
**Page:** 1
### Author:  [@dmjalal90](https://discourse.julialang.org/u/dmjalal90)
#### Post date: [January 15, 2023, 1:41am UTC](https://discourse.julialang.org/t/forecasting-time-series-data-with-neural-ordinary-differential-equations-in-julia/92977/1 "2023-01-15T01:41:17Z")
Dear all,
I am a new user of Julia and following a tutorial on NeuralODE (please see the tutorial: [Experiments with Neural ODEs in Julia](https://computationalmindset.com/en/neural-networks/experiments-with-neural-odes-in-julia.html#exp2)).
How can I forecast ENSO data (time series data in a CSV file) using the above tutorial? In fact, I would like to know how to use real data for NODEs in Julia.
Note: I could not reproduce a tutorial, so called Forecasting the weather with neural ODEs ([Forecasting the weather with neural ODEs | Sebastian Callh personal blog](https://sebastiancallh.github.io/post/neural-ode-weather-forecast/)).
Regards,
Jalal
---
### Author:  [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)
#### Post date: [January 16, 2023, 3:25am UTC](https://discourse.julialang.org/t/forecasting-time-series-data-with-neural-ordinary-differential-equations-in-julia/92977/2 "2023-01-16T03:25:07Z")
What did you try?
---
### Author:  [@dmjalal90](https://discourse.julialang.org/u/dmjalal90)
#### Post date: [January 18, 2023, 3:06am UTC](https://discourse.julialang.org/t/forecasting-time-series-data-with-neural-ordinary-differential-equations-in-julia/92977/3 "2023-01-18T03:06:10Z")
I tried to follow this tutorial: [Experiments with Neural ODEs in Julia](https://computationalmindset.com/en/neural-networks/experiments-with-neural-odes-in-julia.html#exp2).
Data download link (monthly enso): [Dropbox - monthly\_enso.csv - Simplify your life](https://www.dropbox.com/s/8ejvc62b1c2l9uz/monthly_enso.csv?dl=0)
using Flux, DiffEqFlux, DifferentialEquations, Plots
using DataFrames, CSV
data = CSV.read(“monthly\_enso.csv”,DataFrame)
first(data,5)
println(size(data))
tbegin = 0.0
tend = 456
t = range(tbegin,tend,length=tend)
u0 = [2.5; 0.5]
tspan = (tbegin,tend)
trange = range(tbegin,tend,length=tend)
dataset\_ts = data
math\_law(u) = sin.(2. \* u) + cos.(2. \* u)
dudt = Chain(u → math\_law(u),Dense(2, 50, tanh),Dense(50, 2))
reltol = 1e-7 # tol = tolerances
abstol = 1e-9
n\_ode = NeuralODE(dudt, tspan, Tsit5(), saveat=trange, reltol=reltol,abstol=abstol)
ps = Flux.params(n\_ode.p)
function loss\_n\_ode()
pred = n\_ode(u0)
loss = sum(abs2, dataset\_ts .- pred)
end
n\_epochs = 400
learning\_rate = 0.01
data = Iterators.repeated((), n\_epochs)
opt = ADAM(learning\_rate)
cb = function () # callback function to observe training
loss = loss\_n\_ode()
println("Loss: ", loss)
end
println();
cb() # Display the ODE with the initial parameter values.
Flux.train!(loss\_n\_ode, ps, data, opt, cb=cb)
pl = plot(
trange,
dataset\_ts[1,:],
linewidth=2, ls=:dash,
title=“Neural ODE for forecasting”,
xaxis=“t”,
label=“original timeseries x(t)”,
legend=:right)
display(pl)
pl = plot!(
trange,
dataset\_ts[2,:],
linewidth=2, ls=:dash,
label=“original timeseries y(t)”)
display(pl)
pred = n\_ode(u0)
pl = plot!(
trange,
pred[1,:],
linewidth=1,
label=“predicted timeseries x(t)”)
display(pl)
pl = plot!(
trange,
pred[2,:],
linewidth=1,
label=“predicted timeseries y(t)”)
display(pl)
---
### Author:  [@SebastianCallh](https://discourse.julialang.org/u/SebastianCallh)
#### Post date: [January 18, 2023, 1:25pm UTC](https://discourse.julialang.org/t/forecasting-time-series-data-with-neural-ordinary-differential-equations-in-julia/92977/4 "2023-01-18T13:25:29Z")
Hi Jalal!
I’m the author of the blog you linked to and wonder if you could tell me more about how you could not reproduce it. I’m happy to assist with it and update the blog if it is misleading somehow.
---
### Author:  [@dmjalal90](https://discourse.julialang.org/u/dmjalal90)
#### Post date: [January 19, 2023, 2:10am UTC](https://discourse.julialang.org/t/forecasting-time-series-data-with-neural-ordinary-differential-equations-in-julia/92977/5 "2023-01-19T02:10:56Z")
Hi Sebastian Callh,
I used the same data you used for the tutorial. However, the following codes show the ‘year and month’ are undefined. Could you check, please?
delhi[:,:year] = Float64.(year.(delhi[:,:date]))
delhi[:,:month] = Float64.(month.(delhi[:,:date]))
df\_mean = by(delhi, [:year, :month],
:meantemp =\> mean,
:humidity =\> mean,
:wind\_speed =\> mean,
:meanpressure =\> mean)
rename!(df\_mean, [:year, :month, :meantemp,
:humidity, :wind\_speed, :meanpressure])
df\_mean[!,:date] .= df\_mean[:,:year] .+ df\_mean[:,:month] ./ 12;
I would be very happy if I can use your tutorial for my research. I look forward to your reply.
Regards,
Jalal
---
### Author:  [@nilshg](https://discourse.julialang.org/u/nilshg)
#### Post date: [January 19, 2023, 5:47am UTC](https://discourse.julialang.org/t/forecasting-time-series-data-with-neural-ordinary-differential-equations-in-julia/92977/6 "2023-01-19T05:47:15Z")
That just looks like a case of outdated DataFrames syntax - the post is two and a half years old so probably uses a pre 1.0 version of DataFrames.
`by` has been deprecated, so you want to replace that line with
```julia
df_mean = combine(groupby(delhi, [:year, :month]),
[:meantemp, :humidity, :wind_speed, :meanpressure] .=> mean)
```
---
### Author:  [@SebastianCallh](https://discourse.julialang.org/u/SebastianCallh)
#### Post date: [January 19, 2023, 8:12pm UTC](https://discourse.julialang.org/t/forecasting-time-series-data-with-neural-ordinary-differential-equations-in-julia/92977/7 "2023-01-19T20:12:56Z")
Ohh yeah there’s a lot of old Julia in there. I’ll try to make time to update it over the weekend.
In the meantime, the [code on Github](https://github.com/SebastianCallh/neural-ode-weather-forecast) (which is linked from the blogpost) was updated a few months ago to use Lux.jl and Optimization.jl and should play nicely with modern Julia code.
---
### Author:  [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)
#### Post date: [January 20, 2023, 7:03pm UTC](https://discourse.julialang.org/t/forecasting-time-series-data-with-neural-ordinary-differential-equations-in-julia/92977/8 "2023-01-20T19:03:52Z")
Can we make this into an example in the DiffEqFlux docs? The docs are tested with every code change so that will be more robust. We can link back to your blog and papers from there, but this would help make sure it’s kept up to date.
---
### Author:  [@SebastianCallh](https://discourse.julialang.org/u/SebastianCallh)
#### Post date: [January 20, 2023, 7:40pm UTC](https://discourse.julialang.org/t/forecasting-time-series-data-with-neural-ordinary-differential-equations-in-julia/92977/9 "2023-01-20T19:40:13Z")
I’m happy to if it is helpful to people. Do you have any instructions on how? I see there are .md files in DiffEqFlux.jl/src/examples. Is it enough to PR an .md document? Where should images be hosted?
---
### Author:  [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)
#### Post date: [January 20, 2023, 9:31pm UTC](https://discourse.julialang.org/t/forecasting-time-series-data-with-neural-ordinary-differential-equations-in-julia/92977/10 "2023-01-20T21:31:03Z")
> [@SebastianCallh](#):
>
> Is it enough to PR an .md document?
Yup
> [@SebastianCallh](#):
>
> Where should images be hosted?
They get generated during the doc build (it runs the code)
---
### Author:  [@dmjalal90](https://discourse.julialang.org/u/dmjalal90)
#### Post date: [January 22, 2023, 3:07am UTC](https://discourse.julialang.org/t/forecasting-time-series-data-with-neural-ordinary-differential-equations-in-julia/92977/11 "2023-01-22T03:07:18Z")
Thank you so much for your kind response. I am getting the same problem.
---
### Author:  [@nilshg](https://discourse.julialang.org/u/nilshg)
#### Post date: [January 22, 2023, 7:23am UTC](https://discourse.julialang.org/t/forecasting-time-series-data-with-neural-ordinary-differential-equations-in-julia/92977/12 "2023-01-22T07:23:25Z")
And that’s why it’s helpful to provide stack traces! This is not about a variable called year but the function year, which is from the Dates standard library so you are missing `using Dates`
---
### Author:  [@dmjalal90](https://discourse.julialang.org/u/dmjalal90)
#### Post date: [January 22, 2023, 12:59pm UTC](https://discourse.julialang.org/t/forecasting-time-series-data-with-neural-ordinary-differential-equations-in-julia/92977/13 "2023-01-22T12:59:57Z")
Thank you so much. It works now. But I stucked on the final step.
[Warning: FastChain is being deprecated in favor of Lux.jl. Lux.jl uses functions with explicit parameters f(u,p) like FastChain, but is fully featured and documented machine learning library. See the Lux.jl documentation for more details].
And [Warning: sciml\_train is being deprecated in favor of direct usage of Optimization.jl. Please consult the Optimization.jl documentation for more details. Optimization.jl’s PolyOpt solver is the polyalgorithm of sciml\_train].
---
### Author:  [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)
#### Post date: [January 22, 2023, 1:20pm UTC](https://discourse.julialang.org/t/forecasting-time-series-data-with-neural-ordinary-differential-equations-in-julia/92977/14 "2023-01-22T13:20:43Z")
Those are just warnings about upcoming deprecations (well, now the deprecations are soon because the update process started about 2 years ago 😅). We will work with @SebastianCallh to make an updated example be in the docs.
---
### Author:  [@DoktorMike](https://discourse.julialang.org/u/DoktorMike)
#### Post date: [January 22, 2023, 2:09pm UTC](https://discourse.julialang.org/t/forecasting-time-series-data-with-neural-ordinary-differential-equations-in-julia/92977/15 "2023-01-22T14:09:01Z")
Quick question @ChrisRackauckas . If you’re replacing Flux with Lux where does that leave SimpleChains? Wouldn’t SC be more appropriate for the sciml community in general? My reasoning is that i rarely see really big neural networks in sciml contexts so the speedups in SC on CPU starts looking really attractive. This assumes that I’m correct in thinking that the sciml primary use case is small neural networks on a CPU architecture.
---
### Author:  [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)
#### Post date: [January 22, 2023, 4:19pm UTC](https://discourse.julialang.org/t/forecasting-time-series-data-with-neural-ordinary-differential-equations-in-julia/92977/16 "2023-01-22T16:19:26Z")
> [@DoktorMike](#):
>
> If you’re replacing Flux with Lux where does that leave SimpleChains? Wouldn’t SC be more appropriate for the sciml community in general?
Yes, for most SciML use cases people should probably be using SimpleChains. But it’s tricky. SimpleChains is a performance optimization so it’s naturally a less user-friendly library than Flux/Lux. We support all 3 in most places now (note that SciMLSensitivity supports it, but DiffEqFlux’s pre-built layers only Flux or Lux right now).
We should probably highlight SimpleChains in more tutorials than we do right now (and get the DE layers to all allow SimpleChains, that would be worth an issue), but I’d be weary to make it the thing that people grab as a first choice because you do need to be careful when using it. It has support for less types of layers and you need to be careful with multithreading and memory caching in a way you don’t need to with Flux/Lux.
---
### Author:  [@SebastianCallh](https://discourse.julialang.org/u/SebastianCallh)
#### Post date: [January 28, 2023, 8:39am UTC](https://discourse.julialang.org/t/forecasting-time-series-data-with-neural-ordinary-differential-equations-in-julia/92977/17 "2023-01-28T08:39:53Z")
Have been pretty busy but just sat down to get cracking on this and realized we don’t want to put data from Kaggle in the example docs, and I honestly do not know how to write down an ODE that produces similar data that we can use in the example to train on. I _could_ produce noisy sample from the model fit in the blog post and hard code those into the docs example. Would that work or does that open the “who owns the output of a model trained on licensed data?” can of worms? If a ground truth ODE is preferred I’d appreciate some help creating it.
---
### Author:  [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)
#### Post date: [January 28, 2023, 8:51am UTC](https://discourse.julialang.org/t/forecasting-time-series-data-with-neural-ordinary-differential-equations-in-julia/92977/18 "2023-01-28T08:51:55Z")
> [@SebastianCallh](#):
>
> Have been pretty busy but just sat down to get cracking on this and realized we don’t want to put data from Kaggle in the example docs
I think that would be fine? We use DataDeps.jl in another spot for the MNIST data already.
---
### Author:  [@SebastianCallh](https://discourse.julialang.org/u/SebastianCallh)
#### Post date: [January 28, 2023, 9:13am UTC](https://discourse.julialang.org/t/forecasting-time-series-data-with-neural-ordinary-differential-equations-in-julia/92977/19 "2023-01-28T09:13:02Z")
Seems like DataDeps.jl would works great if only the data was publicly available, but Kaggle requires a user login to be able to access datasets.
---
### Author:  [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)
#### Post date: [January 28, 2023, 7:57pm UTC](https://discourse.julialang.org/t/forecasting-time-series-data-with-neural-ordinary-differential-equations-in-julia/92977/20 "2023-01-28T19:57:56Z")
What’s the licensing on Kaggle datasets? I’ve never head to deal with that before.
[Next page](https://discourse.julialang.org/t/forecasting-time-series-data-with-neural-ordinary-differential-equations-in-julia/92977.md?page=2)