Saving an Echo State Network (ReservoirComputing.jl)

Specific Domains Machine Learning
1

Hi,

I am trying to save an Echo State Network (ESN) struct + output Matrix in Julia to load it again in another script to predict a time series. Out of several thousand ESNs, I am choosing the hundred best ones and try to save them. I tried it with JLD, however I cannot load them again. I guess because the struct does not purely consist of arrays. The error I get:

error parsing type string ReservoirComputing.ESN{Core.Int64,Core.Array{Core.Float64,2},ReservoirComputing.Default,ReservoirComputing.NLADefault,Core.Array{Core.Float64,2},ReservoirComputing.RNN{NNlib.#tanh_fast,Core.Float64},Core.Array{Core.Float64,2},Core.Array{Core.Float64,2},ReservoirComputing.StandardStates,Core.Int64,Core.Array{Core.Float64,2}}
Error encountered while load FileIO.File{FileIO.DataFormat{:JLD}, String}("esn_states/esn.jld").

Fatal error:
ERROR: LoadError: syntax: incomplete: premature end of input

How can I save the ESN (or the determining matrices) in the smartest way to be able to construct the ESN again?

2

You have to load the packages before re-loading the model from JLD

3

Hi Chris,

I load the same packages that I used in the script to save before. I try it with JLD2 now, saving and loading seems to work:

using JLD2
jldsave("W_Matrix/W_1.jld2"; Wₒᵤₜ)
jldsave("esn_states/esn_1.jld2"; esn)
esn = jldopen("esn_states/esn_1.jld2")
Wₒᵤₜ =jldopen("W_Matrix/W_1.jld2")
prediction = esn(Predictive(val_x), Wₒᵤₜ)

however I get

ERROR: LoadError: MethodError: objects of type JLD2.JLDFile{JLD2.MmapIO} are not callable

when I try to predict with the loaded ESN.

4

Interesting. Open an issue.

5

In JLD2.jl or ReservoirComputing.jl?

6

ReservoirComputing.jl

7

Since v0.12 ReservoirComputing.jl builds on LuxCore.jl, so saving an ESN only requires saving the parameters ps and states st. Here is the documentation page with a quick example.

Assuming you have defined and trained an ESN, now you only need to specify the construction parameters

spec = (
  in_size = 3,
  res_size = 300,
  out_size = 3,
  radius = 1.2,
  sparsity = 6/300,
  state_modifiers = :NLAT2,
  # include any non-default knobs you used:
  # leak_coefficient = 1.0, input_scaling = 0.1, use_bias=false, etc.
)

and save them

using JLD2
@save "esn_trained.jld2" ps st spec

You can then load them in the same way, and rebuild the ESN:

@load "esn_trained.jld2" ps st spec

esn = ESN(spec.in_size, spec.res_size, spec.out_size;
          init_reservoir=rand_sparse(; radius=spec.radius, sparsity=spec.sparsity),
          state_modifiers = getfield(ReservoirComputing, spec.state_modifiers))

which is now fully defined by the ps and st you have loaded