[ANN] JSON3.jl - Yet another JSON package for Julia

Haha, it’s funny you mention these defaults, because that’s exactly how things are implemented in the package right now; that is, we just treat Base types like custom types and define their JSON3.StructType to be one of the JSON types:

1 Like

Nice, in that case the only thing I’d have done differently would be to have it fail more for other types.

Well, like I said, the problem is that we have a generic fallback defined here, which says, “hey, if a type doesn’t define a more specific StructType, then treat it as a basic JSON3.Struct()” which means we get the behavior like you mentioned w/ skipmissing where we just write out the fields by default. I mentioned I’d be willing to get rid of that definition and have it throw instead, which seems like a nice way to avoid those hard-to-debug scenarios where you forgot to define the StructType for your type.

3 Likes

That would definitely make me much better off, it would be interesting to hear if there are any differing opinions on the matter.

2 Likes

I agree with your suggestion, I very much prefer throwing errors instead of providing defaults that can backfire in some scenarios.

When externalizing objects from a language as complex as Julia, I think one should just let go of the idea that it should “just work” even for complicated cases, and be prepared to make various choices explicit. Instead of offering comprehensive fallback/default choices, this should just ideally be made easy. I think the trait-based solution is very friendly.

7 Likes

On the other hand, when I try to log an application error, my logs are structured json, and some unexpected type snuck in there (perhaps causing the error, perhaps not), I’d prefer logging succeed with some output awkward to read rather than no useful logs getting collected.

2 Likes

This is really cool. The only aspect I find hard to accept is the subtypekey. Adding an additional type::String field to all my concrete types – a field that will always contain the name of said concrete type – is irritating, for lack of a better word. I think I understand its purpose, but it would be amazing to avoid the need of adding this type field.

For me this becomes relevant when I need to save a vector that contains elements that can be of any of the sub (concrete) types of a single abstract type. It just occurred to me that I might just try to read the json string as a Vector{Any} instead of Vector{MyAbstractType}. If this works, it would be preferable to adding a type::String field to all the concrete types that this vector might contain…

Thanks again for this awesome package. I’m sure it’ll become the standard JSON package.

1 Like

Having support for multidimensional arrays would be very nice. I see that the relevant test is broken.

Maybe I missed something here, but how does this work with parametric types? Do we need to define a JSON3.StructType for every parameterized combination:

using JSON3
struct MyParametricType{T}
    t::T
    MyParametricType{T}(t) where {T} = new(t)
end
MyParametricType(t::T) where {T} = MyParametricType{T}(t)

x = MyParametricType(1)

JSON3.StructType(::Type{MyParametricType}) = JSON3.Struct()
str = JSON3.write(x) # ERROR: ArgumentError: MyParametricType{String,Int64} doesn't have a defined `JSON3.StructType`

JSON3.StructType(::Type{MyParametricType{Int}}) = JSON3.Struct()
str = JSON3.write(x) # fine

I posted an issue on this, but I’ll gladly close it if someone can show me what I’ve missed.

I haven’t tried this at all but isn’t it enough to just write

JSON3.StructType(::Type{<:MyParametricType}) = JSON3.Struct()

?

2 Likes

Yes it is! Thanks! I’ll add this to the docs.

2 Likes

Very interesting! I recently did a lot of work to optimise BSON.jl and it sounds like there is quite a lot of overlap. However you seem to have gone further in some areas. It occurs to me that a lot of this could probably be generalised to both BSON, JSON and probably a lot of other formats as well. It is mostly the same patterns being repeated with quite a high cost in terms of development time.

7 Likes

Really great work. You’ve taken the best of what’s come before and made the best Julia JSON package I could have dreamed of.

Particularly missing from JSON.jl was the custom struct parsing. Quite pleased to have that. Then the performance is fantastic. Could not have designed it better or asked for more.

Should be the default Julia JSON package.

Thanks for your work!

There are no “default” packages in the registry in any formal sense. People use what they like, and sometimes a package emerges as a widely used solution (eg DataFrames.jl).

2 Likes

Very nice package. Yet, I could not find a direct way to deal with multi-dimensional arrays, like:

using StructTypes
StructTypes.StructType(::Type{A}) = StructTypes.Struct()
struct A
    a :: Array{Int64}
end
x = A(zeros(2,2))
julia> x
A([0 0; 0 0])
s = JSON3.write(x)
julia> JSON3read(s,A)
A([0, 0, 0, 0])

This gets written as a single columns. Which I parsed afterwards, and that works fine, but it would be nice if a direct approach was available. Is there any option to deal with that? Thank you.

There currently isn’t a builtin/clean way to transform multidimensional arrays into json and back out. There’s an open issue for it, but it needs some thinking to the right API.

2 Likes

Something new in this regard, please? How to write a matrix correctly? Thx

If you structure is mutable, you can use reshape:

julia> mutable struct A
         x  
       end

julia> using StructTypes

julia> StructTypes.StructType(::Type{A}) = StructTypes.Struct()

julia> using JSON3

julia> a = A(rand(3,3))

julia> s = JSON3.write(a)

julia> sread = JSON3.read(s,A)

julia> sread.x = reshape(sread.x,3,3)
3×3 Array{Any,2}:
 0.89818   0.503697  0.0568256
 0.424703  0.505502  0.570105
 0.385958  0.721714  0.46072

If the structure is not mutable, then what I do is to reshape the vectors read saving them in new arrays, and then I initialize a new structure with the data reshaped.

Thanks @leandromartinez98 for the tips, but I’d rather adjust the structure attribute of Matrix to Vector (structure is immutable).
Now I have a question how to write/read Dates.Period?

mutable struct Foo
  samplingPeriod::Period
end
JSON3.StructType(::Type{<:Foo}) = JSON3.Struct()
JSON3.StructType(::Type{<:Period}) = JSON3.StringType()

str = JSON3.write(Foo(Second(3600)))
"{\"samplingPeriod\":\"3600 seconds\"}"

o = JSON3.read(str, Foo)
ERROR: MethodError: no method matching Period(::String)

Do I have to overload the Period method or is there a more elegant way? Any experience? Thanks.

I am having an issue when serializing If someone can help on this regards: