JSON3: ERROR: MethodError: no method matching Array{Int64,2}(::Array{Int64,1})

General Usage
1

Hi,
i need some help with JSON3. If I add Arrays into my custom structure, I get error on JSON3.read.
MWE:

Base.@kwdef mutable struct MyStruct3
    name::String = ""
    par1::Int = 10
    par2::Int32 = 32
    par3::Array{Int,2} = [0 0 0]
end

JSON3.StructTypes.StructType(::Type{MyStruct3}) = JSON3.StructTypes.Mutable()
ms=MyStruct3()
JSON3.write(ms)
write(tempdir() * "/p1.json",JSON3.write(ms))
js=read(tempdir() * "/p1.json")
JSON3.read(js,MyStruct3)

and error:

ERROR: MethodError: no method matching Array{Int64,2}(::Array{Int64,1})
Closest candidates are:
  Array{Int64,2}(::AbstractArray{S,N}) where {T, N, S} at array.jl:562
  Array{Int64,2}(::UndefInitializer, ::Int64, ::Int64) where T at boot.jl:408
  Array{Int64,2}(::UndefInitializer, ::Int64...) where {T, N} at boot.jl:412

 
Stacktrace:
 [1] construct(::Type{T} where T, ::Array{Int64,1}; kw::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}) at C:\Users\znidar\.julia\packages\StructTypes\CpLmq\src\StructTypes.jl:301
 [2] construct(::Type{T} where T, ::Array{Int64,1}) at C:\Users\znidar\.julia\packages\StructTypes\CpLmq\src\StructTypes.jl:301
 [3] #readarray#29 at C:\Users\znidar\.julia\packages\JSON3\hC8pW\src\structs.jl:207 [inlined]
 [4] readarray at C:\Users\znidar\.julia\packages\JSON3\hC8pW\src\structs.jl:187 [inlined]
 [5] #read#27 at C:\Users\znidar\.julia\packages\JSON3\hC8pW\src\structs.jl:183 [inlined]
 [6] read at C:\Users\znidar\.julia\packages\JSON3\hC8pW\src\structs.jl:183 [inlined]
 [7] #read#26 at C:\Users\znidar\.julia\packages\JSON3\hC8pW\src\structs.jl:182 [inlined]
 [8] read at C:\Users\znidar\.julia\packages\JSON3\hC8pW\src\structs.jl:182 [inlined]
 [9] #_#36 at C:\Users\znidar\.julia\packages\JSON3\hC8pW\src\structs.jl:373 [inlined]
 [10] MutableClosure at C:\Users\znidar\.julia\packages\JSON3\hC8pW\src\structs.jl:372 [inlined]
 [11] applyfield! at C:\Users\znidar\.julia\packages\StructTypes\CpLmq\src\StructTypes.jl:666 [inlined]
 [12] #read!#39 at C:\Users\znidar\.julia\packages\JSON3\hC8pW\src\structs.jl:438 [inlined]
 [13] read! at C:\Users\znidar\.julia\packages\JSON3\hC8pW\src\structs.jl:389 [inlined]
 [14] #read#37 at C:\Users\znidar\.julia\packages\JSON3\hC8pW\src\structs.jl:380 [inlined]

Any suggestion?

2

I think JSON3 does not support multi-dimensional arrays. You have to read them as unidimensional and then reshape.

1 Like
3

Defining the struct:

Base.@kwdef mutable struct MyStruct4
    name::String = ""
    par1::Int = 10
    par2::Int32 = 32
    par3::Array{Int,2} = [0 0 0;1 2 3]
end

returns Dict as:

JSON3.Object{Array{UInt8,1},Array{UInt64,1}} with 4 entries:
  :name => ""
  :par1 => 10
  :par2 => 32
  :par3 => [0, 1, 0, 2, 0, 3]

par3 is already flattened and read into structure returns error.

Are there any plans to add support for multi-dims arrays in written JSON (like JSONTables) and/or automatic reshaping in JSON3.read as structure is known - parameter to JSON3.read?

4

I asked that some time ago, and I don’t think so.

In your specific case, since your structure is mutable, you can directly reshape that field with obj.par3 = reshape(obj.par3,2,3). My issue was worse, because my structure was immutable, and I had to define another structure just to read the json file with the same fields, but with a linear array, and then initialize the actual data structure from this temporary structure with linearized arrays.

By the way, my solution ended up not being very elegant, and someone can point to a better solution which might be useful to you as well. What I did is to define four macros, one with the fields before the matrix, two with either the matrix or the linearized form (a vector) and a third with the fields after that matrix. Something like:

macro Results_Start()
  ex = quote
    data1
    data2 
  end; esc(ex)
end
macro Results_Matrix()
  ex = quote
    atoms :: Array{Float64,2}
  end; esc(ex)
end
macro Results_Vector()
  ex = quote
    atoms :: Vector{Float64}
  end; esc(ex)
end
macro Results_End()
  ex = quote
    data4
    data5
  end; esc(ex)
end

Then I define two structs, with these macros, which share all the fields except the matrix or vector one:

struct Result
  @Result_Start()
  @Result_Matrix()
  @Result_End()
end

mutable struct MutableResult
  @Result_Start()
  @Result_ector()
  @Result_End()
end

The MutableResult struct is only used for reading the JSON file, reshape that matrix and initialize the immutable struct with

  return Result([getfield(R,field) for field in fieldnames(Result)]...)

All this seems pretty ugly, so I guess there are better solutions.

1 Like
5

@lmiq thanks for sharing you workaround.

I did a small comparation to JSON:

Base.@kwdef mutable struct MyStruct4
    name::String = ""
    par1::Int = 10
    par2::Int32 = 32
    par3::Array{Int,2} = [0 0 0;1 2 3]
end

julia> JSON.json(ms)
"{\"name\":\"\",\"par1\":10,\"par2\":32,\"par3\":[[0,1],[0,2],[0,3]]}"

julia> JSON3.write(ms)
"{\"name\":\"\",\"par1\":10,\"par2\":32,\"par3\":[0,1,0,2,0,3]}"

One can see that JSON deals with Array in a good way. However JSON3 flatten it. Parsing back should be done with JSON using Unmarshall.jl. The reason I checked JSON3 is that JSON3 works well with missing and redundant keys, what is not case with JSON.

6

Does JSON read the array correctly afterwards?

I had the impression that JSON was considered deprecated in favor of JSON3 at this point.

7

My current usage to write/read my struct (it just configuration for an algorithm therefore performance is not relevant) in JSON format is as follows:

using JSON, Unmarshal
ms=MyStruct4()
json_data=JSON.json(ms) 
##write to file, read back
ms_data=JSON.parse(json_data)
ms=Unmarshal.unmarshal(MyStruct4, ms_data)

julia> using JSON, Unmarshal

julia> ms=MyStruct4()
MyStruct4("", 10, 32, [0 0 0; 1 2 3])

julia> json_data=JSON.json(ms)
"{\"name\":\"\",\"par1\":10,\"par2\":32,\"par3\":[[0,1],[0,2],[0,3]]}"

julia> ms_data=JSON.parse(json_data)
Dict{String,Any} with 4 entries:
  "name" => ""
  "par1" => 10
  "par3" => Any[Any[0, 1], Any[0, 2], Any[0, 3]]
  "par2" => 32

julia> ms=Unmarshal.unmarshal(MyStruct4, ms_data)
MyStruct4("", 10, 32, [0 0 0; 1 2 3])

julia>

The reason to try JSON3 was two fold: depreciation and handling missing or redundant keys in JSON file (more reliable). However no support for arrays is a drawback.

1 Like
8

Thank you. But we still need a temporary struct to read the data, it seems. In that case it is not that different from reading and reshaping afterwards. Ideally the parse or read functions of JSON would directly read the data into the array of the appropriate shape (although I understand that this would only be possible if the exact shape of the data is known in advance and defined in the structure - I see that this is not a very common scenario).

9

JSON + Unmarshal:

  • deals with Array
  • deals with redundand keys in JSON data
  • supports missing data in a limited way: field type has to be Union with Nothing/nullable/Missing
  • does not support if type has default value assigned

JSON3:

  • do not deals with Array
  • deals with redundand keys in JSON data
  • supports missing data: default value is assigned if it is defined in field

A CR in JSON3 to cover Arrays would be best approach.

1 Like