How to deserialize inside a module

General Usage
1

Consider the following scenario:
I define a custom type and a function in Main:

mutable struct MyType
    x
    f
end
my_f(x) = 2x
v = MyType(2,my_f)

using Serialization
serialize("v.a",v)

I create an instance of that type, containing some number and my function and then save it on disk using serialize.

Now I restart the Julia session and try to load that file into a variable, but inside a module (let’s say it lives in the file "test.jl"):

module Test
using Serialization
mutable struct MyType
    x
    f
end
my_f(x) = 2x

w = deserialize("v.a")

end

And I get the following error
**UndefVarError: MyType not defined**

How can I deserialize the file? The Type and function are available, since I specifically defined them inside the module, but I still get an error.
As context to my problem, I try to load a conv net model generated and saved on disk in Main module into a new module(in order to be used for transfer learning).

1 Like
2

Hi,

I’m pretty sure that julia considers the first MyType a completely different type than the one inside the Test module.

It should work if you define MyType inside a module which Test uses. I think this holds for pretty much all serialization packages given what I wrote above.

1 Like
3

The problem is I don’t have control over the first scenario. The file was generated by someone else( trained the network in Main, which is time consuming) and I would not want to repeat training with MyType inside my module.
As a workaround I ended up loading the file in Main, exporting it to my module and then manually converting it to MyType which is inside my module. This is obviously not the best way to do it.
Does this mean that the usage of serialized files between modules developed by different people is very restrictive?
I also encountered this with packages like JLD or BSON.

1 Like
4

I don’t think it is restrictive in general, it is just that the Main module might be bit special since you can’t easily “import” it in a certain state.

As far as I know, if MyType was initially declared in Main it has to be declared in Main again to be deserialized.

I guess this goes into the reasons why one should basically always do development inside modules.


julia> struct MyType
       a
       b
       end

julia> xx = MyType(1,2)
MyType(1, 2)

julia> using Serialization

julia> serialize("xx.jl", xx)

Julia has exited.
Press Enter to start a new session.

Starting Julia...

julia> module MyModule
       using Serialization
       # No need to do "using Main" as it is always included 
       deser(s) = deserialize(s)
       end
Main.MyModule

julia> MyModule.deser("xx.jl")
ERROR: UndefVarError: MyType not defined

julia> struct MyType
       a
       b
       end

julia> MyModule.deser("xx.jl")
MyType(1, 2)

Btw, I made this some time ago out of frustration with brittle serialization. Since it serializes into an external format it does not care what module the model was defined it. If you want to deserialize as an own type it might be a little bit of work to achieve that though.

2 Likes