Haha, it’s funny you mention these defaults, because that’s exactly how things are implemented in the package right now; that is, we just treat Base types like custom types and define their JSON3.StructType to be one of the JSON types:
Well, like I said, the problem is that we have a generic fallback defined here, which says, “hey, if a type doesn’t define a more specific StructType, then treat it as a basic JSON3.Struct()” which means we get the behavior like you mentioned w/ skipmissing where we just write out the fields by default. I mentioned I’d be willing to get rid of that definition and have it throw instead, which seems like a nice way to avoid those hard-to-debug scenarios where you forgot to define the StructType for your type.
I agree with your suggestion, I very much prefer throwing errors instead of providing defaults that can backfire in some scenarios.
When externalizing objects from a language as complex as Julia, I think one should just let go of the idea that it should “just work” even for complicated cases, and be prepared to make various choices explicit. Instead of offering comprehensive fallback/default choices, this should just ideally be made easy. I think the trait-based solution is very friendly.
On the other hand, when I try to log an application error, my logs are structured json, and some unexpected type snuck in there (perhaps causing the error, perhaps not), I’d prefer logging succeed with some output awkward to read rather than no useful logs getting collected.
This is really cool. The only aspect I find hard to accept is the subtypekey. Adding an additional type::String field to all my concrete types – a field that will always contain the name of said concrete type – is irritating, for lack of a better word. I think I understand its purpose, but it would be amazing to avoid the need of adding this type field.
For me this becomes relevant when I need to save a vector that contains elements that can be of any of the sub (concrete) types of a single abstract type. It just occurred to me that I might just try to read the json string as a Vector{Any} instead of Vector{MyAbstractType}. If this works, it would be preferable to adding a type::String field to all the concrete types that this vector might contain…
Thanks again for this awesome package. I’m sure it’ll become the standard JSON package.
Maybe I missed something here, but how does this work with parametric types? Do we need to define a JSON3.StructType for every parameterized combination:
using JSON3
struct MyParametricType{T}
t::T
MyParametricType{T}(t) where {T} = new(t)
end
MyParametricType(t::T) where {T} = MyParametricType{T}(t)
x = MyParametricType(1)
JSON3.StructType(::Type{MyParametricType}) = JSON3.Struct()
str = JSON3.write(x) # ERROR: ArgumentError: MyParametricType{String,Int64} doesn't have a defined `JSON3.StructType`
JSON3.StructType(::Type{MyParametricType{Int}}) = JSON3.Struct()
str = JSON3.write(x) # fine
I posted an issue on this, but I’ll gladly close it if someone can show me what I’ve missed.
Very interesting! I recently did a lot of work to optimise BSON.jl and it sounds like there is quite a lot of overlap. However you seem to have gone further in some areas. It occurs to me that a lot of this could probably be generalised to both BSON, JSON and probably a lot of other formats as well. It is mostly the same patterns being repeated with quite a high cost in terms of development time.
Really great work. You’ve taken the best of what’s come before and made the best Julia JSON package I could have dreamed of.
Particularly missing from JSON.jl was the custom struct parsing. Quite pleased to have that. Then the performance is fantastic. Could not have designed it better or asked for more.
There are no “default” packages in the registry in any formal sense. People use what they like, and sometimes a package emerges as a widely used solution (eg DataFrames.jl).
Very nice package. Yet, I could not find a direct way to deal with multi-dimensional arrays, like:
using StructTypes
StructTypes.StructType(::Type{A}) = StructTypes.Struct()
struct A
a :: Array{Int64}
end
x = A(zeros(2,2))
julia> x
A([0 0; 0 0])
s = JSON3.write(x)
julia> JSON3read(s,A)
A([0, 0, 0, 0])
This gets written as a single columns. Which I parsed afterwards, and that works fine, but it would be nice if a direct approach was available. Is there any option to deal with that? Thank you.
There currently isn’t a builtin/clean way to transform multidimensional arrays into json and back out. There’s an open issue for it, but it needs some thinking to the right API.
julia> mutable struct A
x
end
julia> using StructTypes
julia> StructTypes.StructType(::Type{A}) = StructTypes.Struct()
julia> using JSON3
julia> a = A(rand(3,3))
julia> s = JSON3.write(a)
julia> sread = JSON3.read(s,A)
julia> sread.x = reshape(sread.x,3,3)
3×3 Array{Any,2}:
0.89818 0.503697 0.0568256
0.424703 0.505502 0.570105
0.385958 0.721714 0.46072
If the structure is not mutable, then what I do is to reshape the vectors read saving them in new arrays, and then I initialize a new structure with the data reshaped.
Thanks @leandromartinez98 for the tips, but I’d rather adjust the structure attribute of Matrix to Vector (structure is immutable).
Now I have a question how to write/read Dates.Period?
mutable struct Foo
samplingPeriod::Period
end
JSON3.StructType(::Type{<:Foo}) = JSON3.Struct()
JSON3.StructType(::Type{<:Period}) = JSON3.StringType()
str = JSON3.write(Foo(Second(3600)))
"{\"samplingPeriod\":\"3600 seconds\"}"
o = JSON3.read(str, Foo)
ERROR: MethodError: no method matching Period(::String)
Do I have to overload the Period method or is there a more elegant way? Any experience? Thanks.