How to serialize an object on a x64 system so it'll work on a 32-bit system?

General Usage
1

I need to run some processes on a x64 system (like most modern computers), save the results (e.g. by serializing them to a file), and then load and use the results on a Raspberry Pi (which is a x86 system). The object in question is a Interpolations.jl object.

Is there any way to accomplish this?

Right now, I understandably get this error when trying to deserialize the object on the Pi:

julia> c = deserialize("tmp")
ERROR: TypeError: in Vararg, in count, expected Int32, got a value of type Int64
Stacktrace:
 [1] deserialize_datatype(::Serializer{IOStream}, ::Bool) at /home/pi/work/build/usr/share/julia/stdlib/v1.5/Serialization/src/Serialization.jl:1276
 [2] handle_deserialize(::Serializer{IOStream}, ::Int32) at /home/pi/work/build/usr/share/julia/stdlib/v1.5/Serialization/src/Serialization.jl:826
 [3] deserialize(::Serializer{IOStream}) at /home/pi/work/build/usr/share/julia/stdlib/v1.5/Serialization/src/Serialization.jl:773
 [4] deserialize_datatype(::Serializer{IOStream}, ::Bool) at /home/pi/work/build/usr/share/julia/stdlib/v1.5/Serialization/src/Serialization.jl:1276
 [5] handle_deserialize(::Serializer{IOStream}, ::Int32) at /home/pi/work/build/usr/share/julia/stdlib/v1.5/Serialization/src/Serialization.jl:826
 [6] deserialize(::Serializer{IOStream}) at /home/pi/work/build/usr/share/julia/stdlib/v1.5/Serialization/src/Serialization.jl:773
 [7] deserialize_datatype(::Serializer{IOStream}, ::Bool) at /home/pi/work/build/usr/share/julia/stdlib/v1.5/Serialization/src/Serialization.jl:1276
 [8] handle_deserialize(::Serializer{IOStream}, ::Int32) at /home/pi/work/build/usr/share/julia/stdlib/v1.5/Serialization/src/Serialization.jl:826
 [9] deserialize(::Serializer{IOStream}) at /home/pi/work/build/usr/share/julia/stdlib/v1.5/Serialization/src/Serialization.jl:773
 [10] deserialize_datatype(::Serializer{IOStream}, ::Bool) at /home/pi/work/build/usr/share/julia/stdlib/v1.5/Serialization/src/Serialization.jl:1276
 [11] handle_deserialize(::Serializer{IOStream}, ::Int32) at /home/pi/work/build/usr/share/julia/stdlib/v1.5/Serialization/src/Serialization.jl:826
 [12] deserialize(::Serializer{IOStream}) at /home/pi/work/build/usr/share/julia/stdlib/v1.5/Serialization/src/Serialization.jl:773
 [13] deserialize_datatype(::Serializer{IOStream}, ::Bool) at /home/pi/work/build/usr/share/julia/stdlib/v1.5/Serialization/src/Serialization.jl:1267
 [14] handle_deserialize(::Serializer{IOStream}, ::Int32) at /home/pi/work/build/usr/share/julia/stdlib/v1.5/Serialization/src/Serialization.jl:826
 [15] deserialize(::Serializer{IOStream}) at /home/pi/work/build/usr/share/julia/stdlib/v1.5/Serialization/src/Serialization.jl:773
 [16] deserialize_datatype(::Serializer{IOStream}, ::Bool) at /home/pi/work/build/usr/share/julia/stdlib/v1.5/Serialization/src/Serialization.jl:1276
 [17] handle_deserialize(::Serializer{IOStream}, ::Int32) at /home/pi/work/build/usr/share/julia/stdlib/v1.5/Serialization/src/Serialization.jl:826
 [18] deserialize(::Serializer{IOStream}) at /home/pi/work/build/usr/share/julia/stdlib/v1.5/Serialization/src/Serialization.jl:773
 [19] handle_deserialize(::Serializer{IOStream}, ::Int32) at /home/pi/work/build/usr/share/julia/stdlib/v1.5/Serialization/src/Serialization.jl:833
 [20] deserialize(::Serializer{IOStream}) at /home/pi/work/build/usr/share/julia/stdlib/v1.5/Serialization/src/Serialization.jl:773
 [21] handle_deserialize(::Serializer{IOStream}, ::Int32) at /home/pi/work/build/usr/share/julia/stdlib/v1.5/Serialization/src/Serialization.jl:879
 [22] deserialize(::Serializer{IOStream}) at /home/pi/work/build/usr/share/julia/stdlib/v1.5/Serialization/src/Serialization.jl:773
 [23] deserialize at /home/pi/work/build/usr/share/julia/stdlib/v1.5/Serialization/src/Serialization.jl:760 [inlined]
 [24] open(::typeof(deserialize), ::String; kwargs::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}) at ./io.jl:325
 [25] open at ./io.jl:323 [inlined]
 [26] deserialize(::String) at /home/pi/work/build/usr/share/julia/stdlib/v1.5/Serialization/src/Serialization.jl:770
 [27] top-level scope at REPL[4]:1
2

From the docstring of serialize:

serialize(stream::IO, value)

Write an arbitrary value to a stream in an opaque format, such that it can be read back by deserialize. The read-back value
will be as identical as possible to the original. In general, this process will not work if the reading and writing are done
by different versions of Julia, or an instance of Julia with a different system image

Emphasis is mine. You need to use a different way to store the object.

4 Likes
3

You might consider using JLD2.jl, BSON.jl, or a similar package.

3 Likes
4

Oh would that work? I’ll try tomorrow!

5

didn’t work.

Working on the JLD2 attempt in this issue. Thanks for the suggestions!

6

Plain HDF5?

7

Well, the goal is to save an object, not just matrices/floats/etc.

8

My point is that if it is just one kind of object, it might be easier to write a serializer by hand with HDF5.

9

No idea how to do that, it’s an interpolation from Interpolations.jl… But see the comments here, sounds like it’d be hard?

10

I see. It does look complicated.

1 Like
11

What do you want to do with the deserialized object? If you want to serialize arbitrary object and expect them to behave exactly the same after deserialization then no that’s simply impossible. The object can in general encode architectural dependent info and there’s no way to automatically figure out how those should be converted. You’ve already seen this in your error message. The deserializer cannot make the decision of Int32 vs Int64 for you.

What you need first is a set of architecture independent objects before you can even talk about how to pass them between different machines. This will necessarily limit what you can serialize and after figuring that part out it’s pretty much guaranteed that your allowed object can be very easily encoded in any of the standard formats.

1 Like
12

Yap, what you suggested is what finally worked for me (detailed in that issue). It’s a workaround that works :slight_smile: