# Custom serializer for arrays of Union types
**URL:** https://discourse.julialang.org/t/custom-serializer-for-arrays-of-union-types/129016
**Category:** Performance
**Created:** [May 15, 2025, 3:11am UTC](https://discourse.julialang.org/t/custom-serializer-for-arrays-of-union-types/129016 "2025-05-15T03:11:33Z")
**Posts on this page:** 3
**Page:** 1
### Author:  [@Kirby\_Zhang](https://discourse.julialang.org/u/Kirby_Zhang)
#### Post date: [May 15, 2025, 3:11am UTC](https://discourse.julialang.org/t/custom-serializer-for-arrays-of-union-types/129016/1 "2025-05-15T03:11:33Z")
I’m looking to speed up serialization of arrays that allow for missing values. Performance falls by an order of magnitude according to the following test:
```julia
N = Int(1e8);
buffer = Vector{UInt8}(undef, 8 * N);
io = IOBuffer(buffer, write=true)
test_cases = [
("BitArray of trues", BitArray(true for i in 1:N)),
("Array of zeros (Int)", zeros(Int, N)),
("Uninitialized Vector{Int}", Vector{Int}(undef, N)),
("Array of zeros (Union{Int, Float64})", zeros(Union{Int, Float64}, N)),
("Array of zeros (Union{Int, Missing})", zeros(Union{Int, Missing}, N)),
("Array of missings (Missing)", Vector{Missing}(missing, N)),
("Array of missings (Union{Int, Missing})", Vector{Union{Int, Missing}}(missing, N)),
("Uninitialized Vector{Union{Int, Float64}}", Vector{Union{Int, Float64}}(undef, N))
]
for (desc, arr) in test_cases
println("\n$desc:")
empty!(buffer)
@time Serialization.serialize(io, arr)
end
```
My results:
```julia
BitArray of trues:
0.003427 seconds (24 allocations: 1.516 KiB)
Array of zeros (Int):
0.354058 seconds (20 allocations: 351.369 MiB, 18.31% gc time)
Uninitialized Vector{Int}:
0.459178 seconds (20 allocations: 651.290 MiB, 13.53% gc time)
Array of zeros (Union{Int, Float64}):
0.535546 seconds (20 allocations: 512.001 MiB, 5.32% gc time)
Array of zeros (Union{Int, Missing}):
0.516552 seconds (20 allocations: 512.001 MiB, 2.61% gc time)
Array of missings (Missing):
0.000015 seconds (18 allocations: 1.422 KiB)
Array of missings (Union{Int, Missing}):
6.071769 seconds (100.00 M allocations: 1.990 GiB, 6.16% gc time)
Uninitialized Vector{Union{Int, Float64}}:
2.304365 seconds (100.00 M allocations: 1.990 GiB, 15.80% gc time)
```
I found code [here](https://stackoverflow.com/questions/49007433/how-to-implement-custom-serialization-deserialization-for-a-struct-in-julia) that show how I can override serialization for a type.
```julia
using Serialization
# The target struct
struct Foo
x::Int
y::Union{Int, Nothing} #we do not want to serialize this field
end
# Custom Serialization of a Foo instance
function Serialization.serialize(s::AbstractSerializer, instance::Foo)
Serialization.writetag(s.io, Serialization.OBJECT_TAG)
Serialization.serialize(s, Foo)
Serialization.serialize(s, instance.x)
end
# Custom Deserialization of a Foo instance
function Serialization.deserialize(s::AbstractSerializer, ::Type{Foo})
x = Serialization.deserialize(s)
Foo(x,nothing)
end
foo1 = Foo(1,2)
# Serialization
write_iob = IOBuffer()
serialize(write_iob, foo1)
seekstart(write_iob)
content = read(write_iob)
# Deserialization
read_iob = IOBuffer(content)
foo2 = deserialize(read_iob)
@show foo1
@show foo2
```
But what I would really like to do is inspect if an array is of Union type with a Missing type parameter (I’d use a bitarray to flag missing entries), and pass it to the default implementation if I can’t handle the structure. I’m not sure how I can intercept the serialization for `AbstractVector`, and still have the default implementation that I can use as fallback. I’m also not sure if I can easily subclass `AbstractSerializer`, or how I can organize a custom serializer.
Thanks for any help you can offer!
---
### Author:  [@jling](https://discourse.julialang.org/u/jling)
#### Post date: [May 15, 2025, 3:54am UTC](https://discourse.julialang.org/t/custom-serializer-for-arrays-of-union-types/129016/2 "2025-05-15T03:54:24Z")
> [@Kirby\_Zhang](#):
>
> I’m looking to speed up serialization of arrays that allow for missing values.
why can’t you use Arrow.jl?
---
### Author:  [@Kirby\_Zhang](https://discourse.julialang.org/u/Kirby_Zhang)
#### Post date: [May 15, 2025, 4:08am UTC](https://discourse.julialang.org/t/custom-serializer-for-arrays-of-union-types/129016/3 "2025-05-15T04:08:59Z")
I had no idea about Arrow.jl. I’m not sure how missing values would affect its performance. I don’t really need the language interop aspect of Arrow.
I think I’m just going to add temporary bit vectors in the DataFrames to allow me to hide/unhide missing values when I’m transmitting.