I’m curious what the right way to append elements to a vector stored in a JLD2 file, assuming that I want to save the data to disk every time I add an element (because each element is expensive to compute).
The first thing I tried was:
jldopen("test.jld2", "a+") do file file["x"] =  end for i in 1:10 jldopen("test.jld2", "a+") do file push!(file["x"], 1) end end
but that actually results in the file being empty, because doing
file["x"] gives me a copy of the data, and appending to that copy does nothing to the data on disk.
So, instead, I could do:
results =  for i in 1:10 push!(results, 1) jldopen("test.jld2", "w") do file file["x"] = results end end
but this will write the entire
results vector to disk at every iteration, which seems wasteful.
Am I missing something obvious? Is there a better way?