Announce: A different way to read JSON data, LazyJSON.jl

Specific Domains Data
2

I’ve been playing around with @quinnj’s JSON2.jl package a little more in the last few days. For me the two standout features are:

  • '.' notation (getproperty) for field access (in JSON2 this is achived by using NamedTuples)
  • automagically filling in a struct from a JSON text using JSON2.read(json_string, StructType)

I’ve been inspired to add similar capabilities to the latest version of LazyJSON by:

  • putting a getproperty wrapper around LazyJSON.Object so that 'o.field' is syntaxtic sugar for o["field"] (the field lookup is still done by the usual lazy getindex method).
  • adding a Base.convert(::Type{T}, ::LazyJSON.Object} method that constructs new structs from JSON text.

e.g. '.' notation for fields:

julia> arrow_json =
       """{
           "label": "Hello",
           "segments": [
                {"a": {"x": 1, "y": 1}, "b": {"x": 2, "y": 2}},
                {"a": {"x": 2, "y": 2}, "b": {"x": 3, "y": 3}}
            ],
            "dashed": false
       }"""

julia> lazy_arrow = LazyJSON.value(arrow_json)

julia> lazy_arrow.segments[1].a.x
1

e.g. convert to struct:

julia> struct Point
           x::Int
           y::Int
       end

julia> struct Line
           a::Point
           b::Point
       end

julia> struct Arrow
           label::String
           segments::Vector{Line}
           dashed::Bool
       end

julia> convert(Arrow, lazy_arrow)
Arrow("Hello", Line[Line(Point(1, 1), Point(2, 2)), Line(Point(2, 2), Point(3, 3))], false)

Implementation notes:

'.' notation is implemented by:
Base.getproperty(d::PropertyDict, n::Symbol) = getindex(d, String(n))

convert is implemented by a @generated function that generates somthing like:

function convert{::Type{Arrow},  o::JSON.Object)
    i = o.i
    i, label = get_field(o.s, "label", i)
    i, segments = get_field(o.s, "segments", i)
    i, dashed = get_field(o.s, "dashed", i)
    Arrow(label, segments, dashed)
end

The local variable i keeps track of the current string index to make finding the fields faster (if they are in the expected order). The call to get_field(o.s, "segments", i) returns a LazyJSON.Array object. When this is passed to the Arrow constructer, Julia automatically calls convert(Vector{Line}, ...) to convert it to the type of the struct field. In this way the conversion process works recursively for arbitrarily nested structs.

But, if you care about minimising copying and memory allocation, and you don’t need to access all the fields in a JSON object, it might be better to do away with structs and use the '.' notation feature to access the JSON values in-place. In my experience with web services APIs it is quite common to have a whole mess of JSON returned by an API request, when I’m only interested in a few particular fields.

7 Likes