Serializing a Module

General Usage
1

Hi,

is it possible to deserialize a Module in a new process to get that module (without defining it in the second process first)? I get the following error message when I use the Serialization library as follows

Process 1

using Serialization
module A; x = 1; end
serialize("ModuleA", A)

Process 2

julia> using Serialization

julia> deserialize("ModuleA")
ERROR: UndefVarError: A not defined
Stacktrace:
  [1] deserialize_module(s::Serializer{IOStream})
    @ Serialization ~/packages/julias/julia-1.7/share/julia/stdlib/v1.7/Serialization/src/Serialization.jl:984
  [2] handle_deserialize(s::Serializer{IOStream}, b::Int32)
    @ Serialization ~/packages/julias/julia-1.7/share/julia/stdlib/v1.7/Serialization/src/Serialization.jl:883
  [3] deserialize(s::Serializer{IOStream})
    @ Serialization ~/packages/julias/julia-1.7/share/julia/stdlib/v1.7/Serialization/src/Serialization.jl:801
  [4] handle_deserialize(s::Serializer{IOStream}, b::Int32)
    @ Serialization ~/packages/julias/julia-1.7/share/julia/stdlib/v1.7/Serialization/src/Serialization.jl:907
  [5] deserialize
    @ ~/packages/julias/julia-1.7/share/julia/stdlib/v1.7/Serialization/src/Serialization.jl:801 [inlined]
  [6] deserialize(s::IOStream)
    @ Serialization ~/packages/julias/julia-1.7/share/julia/stdlib/v1.7/Serialization/src/Serialization.jl:788
  [7] open(f::typeof(deserialize), args::String; kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
    @ Base ./io.jl:330
  [8] open
    @ ./io.jl:328 [inlined]
  [9] deserialize(filename::String)
    @ Serialization ~/packages/julias/julia-1.7/share/julia/stdlib/v1.7/Serialization/src/Serialization.jl:798
 [10] top-level scope
    @ REPL[2]:1
2 Likes
2

Why is there not even a single comment on this? Is it considered trivial? Is it considered uninteresting? I am finding the answer to the last two to be ‘no’.

1 Like
3

To my best knowledge, this is not possible, in Julia or most other langauges. If it were possible, you would be able to distribute modules in a binary format. This would unnecessarily duplicate the functionality of precompiling a module or experimental efforts on static compilation.

4

I’m trying to think of a reason to serialize/deserialize a module.

Why not write the module to a .jl (plain text) file and include that file in both processes?

I guess the underlying question could be: what’s the difference between includeing a text file and deserializing a binary file, while both can contain a module? That’s actually interesting.

The help in ?include and ?deserialize suggest that include will evaluate the contents of the file, while deserialize will only read the value.

Note that the same happens when you serialize/deserialize a function in one process:

julia> f(x) = x + 1
f (generic function with 1 method)

julia> serialize("function", f)

In another process:

julia> deserialize("function")
ERROR: UndefVarError: `#f` not defined in `Main`

Note be careful, if f or A already exists in the other process, deserialize seems to continue without error, but doesn’t actually overwrite the existing function/module:

julia> f(x) = x + 5
f (generic function with 1 method)

julia> deserialize("function")
f (generic function with 1 method)

julia> f(5) # remember we serialized `f(x) = x + 1`, so expect 6
10

I bet you have similar behavior for custom type definitions.

So my summary: since deserialize only reads values, and does not evaluate any new code like include, you cannot actually deserialize functions/modules/type definitions (or anything else that requires code evaluation after importing the file contents). And you get an error when a symbol doesn’t exist yet.

5

What you could do instead is serialize a string or an expression of the string (see Metaprogramming), and then deserialize and evaluate that expression:

julia> ex = Meta.parse("module A; x = 1; end")
:(module A
  #= none:1 =#
  #= none:1 =#
  x = 1
  end)

julia> serialize("expression", ex)

and then in another process:

julia> eval(deserialize("expression"))
Main.A

julia> A.x
1

Whether this is a good idea or not, I will leave up to the reader.