Way to migrate jld2 data to have new fields without corruption?

General Usage
1

Let’s say you have a struct, Foo:

struct Foo
  fizz::Int
end

Then you make a slew of Foo’s and save them to a file using JLD2.jl

Now, your boss tells you to add a buzz flag:

struct Foo
  fizz::Int 
  buzz::Bool
end

On booting up a jupyter notebook, you now find your N hours of Foo’s are corrupted:

WARNING: saved type Bar.Foo is missing field buzz in workspace type; reconstructing
Error encountered while loading "/Users/picard/Desktop/notebook_folder/foo.jld2".
Fatal error:

MethodError: Cannot `convert` an object of type JLD2.ReconstructedTypes.##Bar.Foo#673 to an object of type Bar.AbstractFoo
This may have arisen from a call to the constructor Bar.AbstractFoo(...),
since type constructors fall back to convert methods.

Is there a way you can work around this issue?

// it seems like migrations are a fundamental building block of storing data

2

Is that really a full example? What is Bar.AbstractFoo?

julia> struct Foo
         fizz::Int
       end

julia> a = [Foo(1), Foo(2)]
2-element Array{Foo,1}:
 Foo(1)
 Foo(2)

julia> using JLD2
[ Info: Recompiling stale cache file /Users/kristoffer/.julia/compiled/v1.1/JLD2/O1EyT.ji for JLD2 [033835bb-8acc-5ee8-8aae-3f567f8a3819]

julia> @save "foo.jld" a

➜  ~/julia/julia -q # restart Julia

julia> struct Foo
         fizz::Int
         bar::Bool
       end

julia> using JLD2

julia> @load "foo.jld" a
┌ Warning: saved type Foo is missing field bar in workspace type; reconstructing
└ @ JLD2 ~/.julia/packages/JLD2/KjBIK/src/data.jl:402
1-element Array{Symbol,1}:
 :a

julia> a
2-element Array{getfield(JLD2.ReconstructedTypes, Symbol("##Foo#360")),1}:
 getfield(JLD2.ReconstructedTypes, Symbol("##Foo#360"))(1)
 getfield(JLD2.ReconstructedTypes, Symbol("##Foo#360"))(2)

julia> a[1]
getfield(JLD2.ReconstructedTypes, Symbol("##Foo#360"))(1)

julia> a[1].fizz
1
3

First, this is the actual struct:

// with Foo mapping to Reactor and buzz mapping to closure

And AbstractFoo is something I use so functions can be built before I define the Foo type (i.e. for load order).

See:

4

I just decided to delete the new field and never change the struct ever again.

Bada bing. Bada boom. Problem solved.