This is the code I used for testing julia> struct Tree left::Tree right::Tree Tree() = new() Tree(l, r) = new(l, r) end julia> a = Tree() Tree(#undef, #undef) julia> for i=1:20 a = Tree(a, a) end julia> using Serialization jul…

Semantically they copy, though in this case the compiler can optimise things by using references instead (probably because you have possibly undefined fields), which is why your example doesn’t blow up.

Good point, they are probably the same thing. My temporary solution is keeping a dictionary of objects, and then dump this dictionary. Wondering if it can be made more elegant.

[image] 1115: The resulting file has size 10MB. This seems pretty reasonable for a tree with 2^20 nodes?

The actual storage is 20 x (the object size of Tree type). Because we only create this many objects. They are conceptually big, but in the computer memory, they are just 20 linked addresses with a lot memory sharing.

How does it scale if you double the tree nodes? Maybe other things take up the space. Another thing might be a hidden copy of data. Did you try to use Ref instead?

julia> a = Tree() Tree(#undef, #undef) julia> for i=1:21 a = Tree(a, a) end julia> serialize("s.dat", a) shell> ls s.dat -l -rw-rw-r-- 1 leo leo 20971537 Mar 15 19:55 s.dat julia> a = Tree() Tree(#undef, #undef) julia> for i=1:20 a = Tree(a, a) end julia> se…

Did you check the object IDs after deserialization ?

Your struct is immutable, so has copy semantics, so indeed you have about 2^21 objects to serialise. If you define it to be mutable, which has reference semantics, it should serialise to a small file.

Yes, the objectid of left and right branches after deserialization are the same.

I think immutable types do not always copy, otherwise the following code would blow up my memory: julia> struct Tree left::Tree right::Tree Tree() = new() Tree(l, r) = new(l, r) end julia> a = Tree() Tree(#undef, #undef) julia> for i=1:200 …

Serializing a tree requires a huge amount of storage

General Usage

goerch March 15, 2022, 10:32pm 2

We had a similar discussion recently. But interestingly there we talked about time, not space…

1 Like

Topic		Replies	Views
Serialize() is prohibitively slow for recursive structs. What am I missing? New to Julia serialization , recursion	15	1940	February 14, 2022
Efficiently copying a tree with shared children Performance question , package , performance	20	739	September 27, 2022
Serialization compresses very well? General Usage	0	405	August 17, 2018
Best way to store a sparse, unbalanced, and not complete binary tree General Usage mutable-structure , tree	18	2752	January 23, 2022
Efficiently traverse binary tree represented as tuple of tuples Performance memory-allocation , benchmark	9	1983	August 7, 2019

Serializing a tree requires a huge amount of storage

Related topics