Unable to serialize SharedArrays

General Usage
1

I use Serialization.serialize() function to write julia objects on disk since I found it more efficient than other formats (like JLD2) and it’s for short term shortage so I can use the the same Julia runtime to read it back.

It works well except that SharedArray cannot be serialized properly. It seems that it may be just saving the pointers so the output file is very small even though I have a large array. Is there any to do this properly? I prefer not to translate it to a regular Array because it’s already large and I don’t have enough memory…

julia> using SharedArrays, Serialization

julia> A = SharedArray{Float64,2}((2,1000000));

julia> open("A.jo", "w") do file serialize(file, A) end

julia> stat("A.jo")
StatStruct(mode=0o100666, size=181)
2

Can you reinterpret a SharedArray and then serialize that?

3

I guess it makes sense to only pass pointers rather than real data when sending a SharedArray from one process to another (which is the real reason for serialization in the first place) but it doesn’t work in my case.

Reinterpreting seems to allocate:

julia> typeof(A)
SharedArray{Float64,2}

julia> size(A)
(2, 1000000)

julia> Base.summarysize(A)
16000365

julia> f(A) = collect(reinterpret(Float64, A))
f (generic function with 1 method)

julia> @allocated f(A)
16185831
4

See Big overhead with the new lazy reshape/reinterpret - #32 by pablosanjose