Save Model Outputs

Specific Domains Optimization (Mathematical)
1

Hi,
I am running a small model several times (1000 to be more precise) and for each run I need to store the results. Usually I turn it to a DataFrame and save as a csv. But in this case I do not want to save 1000 csvs. I understand that this question is not exactly optimization but I wanted to know if there is any alternative in JuMP for this case. In R you can use RDS so I was trying to use jld, but I am not able to save and load the save data.

# Dataframe with model results 
df = DataFrame(variable = [1, 2, 3, 4, 5], 
               value = [800, 250, 700, 1400, 120]
               )

vec_dfs = []

# Simulated sequential results
push!(vec_dfs,df)
push!(vec_dfs,df)


save("/home/myfile.jld", "vec_dfs", vec_dfs)

d = load("/home/myfile.jld")

During the save I get:

┌ Warning: `object_info(obj::Union{File, Object})` is deprecated, use `API.h5o_get_info1(checkvalid(obj))` instead.
│   caller = typeindex(parent::JLD.JldFile, addr::UInt64) at jld_types.jl:812
└ @ JLD ~/.julia/packages/JLD/6OyJe/src/jld_types.jl:812
error parsing type string @where(Core.AbstractArray{#s77,1},#s77)
Error encountered while load File{FileIO.DataFormat{:JLD}, String}("/home/yuri/myfile.jld").

For simple variables I am able to sabe the jld.

I also tried to save as txt but had a similar problem trying to read.
Am I doing something wrong? Or can you help me with alternatives?
Thanks!

2

I don’t know what the problem is with load, but it’s likely that vec_dfs = [] is a Vector{Any}. I guess it needs to be a vector of concrete types? Try vec_dfs = DataFrame[].

Otherwise though, why not create one large DataFrame with a run_id::Int column and save that to a CSV (or a database)?

1 Like
3

Hi,
Thanks for the answer, I was trying to avoid one long file so I opted not to use the stacked csv. Also I am more used to work with a list o tables.
I did manage no save by the following way:

f = open("/home/resultados.jl", "w")
print(f, vec_results)
close(f)

,and reading by:

vec_res=include("/home/resultados.jl")

The only thing I had to do was change from DataFrame to Array and it worked fine.
Thanks!

1 Like