JSON3 Package: ArgumentError: invalid JSON at byte .... : InvalidChar?

New to Julia
1

Suppose I have two Structs one for Tweet and another for Author of the tweet as follows:

struct User
    id::Union{Int64, Nothing}
    id_str::Union{String, Nothing}
    created_at::Union{String, Nothing}
    name::Union{String, Nothing}
    screen_name::Union{String, Nothing}
    location::Union{String, Nothing}
    statuses_count::Union{Int64, Nothing}
    followers_count::Union{Int64, Nothing}
    description::Union{String, Nothing}
    profile_image_url::Union{String, Nothing}
end

struct Tweet
    id::Union{Int64, Nothing}
    id_str::Union{String, Nothing}
    created_at::Union{String, Nothing}
    favorite_count::Union{Int, Nothing}
    retweet_count::Union{Int, Nothing}
    full_text::Union{String, Nothing}
    lang::Union{String, Nothing}
    place::Union{String, Nothing}
    truncated::Union{Bool, Nothing}
    user::User
end

Now I have a Json String as :

json_str = "{\"id_str\":\"1305501948074835974\",\"created_at\":\"Mon Sep 14 13:41:34 +0000 2020\",\"place\":null,\"id\":1305501948074835974,\"user\":{\"name\":\"Donald J. Trump\",\"id_str\":\"25073877\",\"created_at\":\"Wed Mar 18 13:46:38 +0000 2009\",\"id\":25073877}}"

I am trying to define a New Struct function and read the string in to value of type:

JSON3.StructType(::Type{Tweet}) = JSON3.Struct()
JSON3.read(json_str, Tweet)

I am getting the following error:

ArgumentError: invalid JSON at byte position 11 while parsing type Int64: InvalidChar
{"id_str":"1305501948074835974","cre

If someone can guide me what is the best approach/package for such Serialization in Julia which is stable. Does the string need to match the exact order of the Struct? or Not sure where is the Issue. Though the JSON3.read(json_str) works fine.

I have tried the same stuff with JSON.jl and Unmarshal.jl first read the json string in to json with JSON.parse(json_str) and the using unmarshal(Tweet, parsed_json) to do the same but it also has some bugs and was not able to solve it so far. Both of the issues has been created in their corresponding project.

If someone can tell me:

  • How to read such data in to some value of Given struct type (Tweet in that example)
  • Is it possible to convert the value to Dictionary using Json3 or has the following conversion feature:
    • json string to parsed Dict , json string to Defined struct, Dict to Defined Struct and vice versa.
2

There are two issues in the code you are using

  • JSON3.Struct() passes the arguments by order meaning it would tries to parse id_str as a id::Union{Int64, Nothing} which gives the error as it tries to
parse(Int64, "\"1305501948074835974\"")
  • You are missing the StructType(::Type{User}) declaration

The way to fix it, especially using Twitter data, is to use Mutable like this:

using JSON3: JSON3, StructType, Mutable
using Dates: DateTime, DateFormat
import Base: show, summary
mutable struct User
    id :: UInt64
    name :: String
    created_at :: DateTime
    User() = new()
end
mutable struct Status
    id :: UInt64
    user :: User
    created_at :: DateTime
    place :: Union{Missing, String}
    Status() = new()
end
summary(io::IO, obj::Status) = print(io, "Twitter Status: $(obj.id)")
function show(io::IO, obj::Status)
    println(io, summary(obj))
    println(io, "  User: $(obj.user.name)")
    println(io, "  On: $(obj.created_at)")
end
StructType(::Type{User}) = Mutable()
StructType(::Type{Status}) = Mutable()
json_str = "{\"id_str\":\"1305501948074835974\",\"created_at\":\"Mon Sep 14 13:41:34 +0000 2020\",\"place\":null,\"id\":1305501948074835974,\"user\":{\"name\":\"Donald J. Trump\",\"id_str\":\"25073877\",\"created_at\":\"Wed Mar 18 13:46:38 +0000 2009\",\"id\":25073877}}"
JSON3.read(json_str, Status, dateformat = DateFormat("e u dd HH:MM:SS +0000 yyyy"))
3 Likes