JSON type serialization

#1

Hi,

I wonder why serialisation of JSONs does not contain information about type? Has anyone encountered similar need?

For example

struct TestType{A}
  a::A
end

println(JSON.json(TestType("Hello")))
{"a":"Hello"}

Does not tell me anything about serialised type being TestType?

Thanks for help

#2

You can change the way a particular type is serialized by implementing the JSON.lower() method:

julia> JSON.lower(t::TestType) = Dict("type"=>"TestType", "a"=>t.a)

julia> println(JSON.json(TestType("Hello")))
{"type":"TestType","a":"Hello"}

if you want more full-featured serialization of complex data types in Julia, you might want to check out https://github.com/simonster/JLD2.jl

#3

Thanks a lot.

#4

You can also check out https://github.com/quinnj/JSON2.jl

#5

Cool! I didn’t know that existed.

#6

JSON is typically considered an “anonymous type” kind of specification, so there isn’t really a notion of what “type” an object is. More specifically, there are only a limited set of “types” in JSON:

  • Object: specified by { ... } brackets and key-value pairs, w/ keys being strings
  • Array: specified by [ ... ] brackets, each element being separated by a comma
  • String: specified by " ... " double quotation marks
  • Number: either integer or float values
  • Null: the literal null value

In this schema, as I mentioned, the closest kind of thing to a custom Julia type is an Object, so the natural representation of your TestType example is indeed {"a":"Hello"}. As you’ve noticed, this can pose challenges w/ a type like TestType{A} in Julia, where the inner field is a parameter and can essentially be one of many types.

In JSON2, this is made a bit simpler by allowing the user to specify the type that they wish to deserialize, like the following example:

julia> struct TTT{A}
       x::A
       end

julia> a = TTT(1.0)
TTT{Float64}(1.0)

julia> b = TTT("hey")
TTT{String}("hey")

julia> aa = JSON2.write(a)
"{\"x\":1.0}"

julia> bb = JSON2.write(b)
"{\"x\":\"hey\"}"

julia> JSON2.read(aa, TTT{Float64})
TTT{Float64}(1.0)

julia> JSON2.read(bb, TTT{String})
TTT{String}("hey")

In this case, I’m able to read and write our custom struct TTT with either a Float64 or String inner field because when reading, we’re able to specify the exact type we want to construct, and JSON2 can, using julia’s powerful reflection tools, figure out how to construct one of those types.

#7

Hi Jacob,

this is nice. What would happen if the specification of TTT contains one more element indicating the type?

Can I ask what are the areas where JSON2 is better than JSON?

Last but not least, I cannot install it. I am getting following error:

Pkg.add("JSON2")
ERROR: JSON2 can't be installed because it has no versions that support 0.6.2 of julia. You may need to update METADATA by running `Pkg.update()`

Thanks for replys.

Tomas

#8

Yeah, sorry, JSON2 is 0.7 only for now, since it relies on new features coming w/ the new release. I haven’t publicized the package much.

#9

Bool also, true and false

#10

JSON2’s functionality is very intriguing - I’m also very much looking forward for it to be “officially” released by Jacob! (If you are looking for other good data oriented packages for Julia, Jacob’s ones are a good place to start,
such as CSV.jl also)

#11

Can I ask how JSON2 will deserialize array? JSON deserialize every array to ANY, which is quite inconvenient.

julia> JSON.parse("""{"a": [1,2,3,4]}""")
Dict{String,Any} with 1 entry:
  "a" => Any[1, 2, 3, 4]

Thanks for an answer.

Tomas

#12

With LazyJSON, it will seem that you get an AbstractVector{Any}. However, because of the lazy implementation, no vector has actually been created yet. So, if you then convert to a desired vector type, the vector will be constructed by directly parsing the JSON text (there is no intermediate Vector{Any} representation).

julia> using LazyJSON
LazyJSON.
julia> x = LazyJSON.value("""{"a": [1,2,3,4]}""")
LazyJSON.Object{String} with 1 entry:
  "a" => Any[1, 2, 3, 4]

julia> v = convert(Vector{Int}, x.a)
4-element Array{Int64,1}:
 1
 2
 3
 4
#13

Thanks Sam.

I have on the end found that I really need to keep Any, because JSONs are wild and if typing is too strict, it might happen that I would create too tight dictionaries.

Tomas