The professional developers on my team using c#, convert the floats (or Ints or bools), to an unsigned integers, then use base64 encoding. I’m told this will guarantee what is written out will be read back in, assuming using a windows or Mac computer. We don’t use Linux. We store everything as vector, and whatever format (json, xml, TOML) we are using also store dimensions and the datatype of the original data. Below is code I use to convert a vector to that hashed format and reading it back into a vector.
We’ve had a lot of success using TOML to store thousands of regression and unit tests across our systems using this. It made it very easy for me to write Julia versions of our c# and guarantee that I get the same results as our c#.
In the TOML, we store the binary representation of the data, but also the values written out. We do that so that from inspecting the file you can see the values, but if you want the exact information the program would use the binary rep. TOML makes it very easy to convert to Dict and back. In fact, we round trip TOML to Dict to Julia Structs.
If a vector, the size is a single number. If the data is a scalar, size is not included in the input.
We prefer TOML since it understands nan, infinity, easily maps to Dicts, human readable, and a lot of libraries are available.
The TOML file for one of the inputs looks like this:
size = [2,3]
dtype = "Float64"
data = [0.09618765866371004, 0.44433488489147255, 0.9005311838970897, 0.6818201970996056, 0.8526313332776663, 0.41132839089506557]
bdata = "UEw9IMGfuD/QK8WV+2/cP5q3+8Um0ew/THEJl3jR5T9WHn+BwUjrPwzhs1A0U9o/"
Here is the Julia code with an example for working with a vector to create what we call the binary representation:
x = rand(10)
str = writebinarydata(x)
x2 = readbinarydata(str, Float64)
x .== x2
if eltype(x) == Float64
type = UInt64
elseif eltype(x) == Bool
type = UInt8
elseif eltype(x) == Int64
type = UInt64
@warn "Unsupported type"
reinterpret.(type,x) |> base64encode
readbinarydata(x::String, dtype::Type) = dtype.(reinterpret(dtype, x |> base64decode))