A problem with parsing JSON files located at a specific address

New to Julia
1

I am having a great deal of (beginner’s) problems parsing JSON files.

Here is the situation:
I have a bundle of JSON-files in a specific directory (actually, the content of each file is in JSON-format, but the files do not have a .json extention - I don’t know if that information is important).

Let’s assume the data is located at

"C:/Users/me/data/"

The content of each file is causing me some problem. If I for the moment ignore reading files from the particular location, and instead copy-paste the content of each file to parse, I get an error. For example, the following causes error

JSON.parse(""" {"op":"mcm","clk":"1801003563","pt":1451592321710,"mc":[{"id":"1.122431815","rc"[{"ltp":1.67,"id":8597476}]}]}{"op":"mcm","clk":"1801026105","pt":1451593397438,"mc":[{"id":"1.122431815","rc":[{"ltp":1.66,"id":8597476}]}]} """)

However, the following does not:

JSON.parse(""" {"op":"mcm","clk":"1801003563","pt":1451592321710,"mc":[{"id":"1.122431815","rc":[{"ltp":1.67,"id":8597476}]}]} """)

The problem appears to be that my file (as suggested by my first attempt) contains a series of rows, (one for each line of the document) which can be parsed independently. But when I try to parse across multiple lines (rows), I get an “Expected end of input” error.

So, my first question is, how do I parse my file, when it has these multiple independent rows?

My second question relates to parsing files located at a particular place. When I read the files as .txt there is no problem. For example, I get no error when I do the following:

f = open("C:/Users/me/data/1.122431815")

However, when I try the simillar-looking syntax for parsing JSON, I hit an error:

f = JSON.parse("C:/Users/me/data/1.122431815")

So, my second question is, how do I parse a JSON file located at a specific directory?

2

If you look at the help for JSON.parse it has two forms:

Parses the given JSON string into corresponding Julia types

and:

Parses JSON from the given IO stream into corresponding Julia types.

Neither of those open a file and read the contents. Which would be an issue anyway since the file isn’t a single JSON object, but mutliple.

An easy solution for you (assuming NONE or your JSON objects have a \n in them) is to do:

loaded = read("C:/Users/me/data/1.122431815", String)
for object in split(loaded, "\n")
    json = JSON.parse(object)
end

The first line will load whole file as a string. split() will break the file into multiple lines, so a JSON object must be on a single line in the file. Then JSON.parse() can decode a single JSON object.

1 Like
3

Or, if your independent lines are not new line spearated like in your first example you can do this, to parse each independent line into an element of an array:
This is the same line as in a your example above (without the little typo which is inside):

julia> jsonstring=""" {"op":"mcm","clk":"1801003563","pt":1451592321710,"mc":[{"id":"1.122431815","rc":[{"ltp":1.67,"id":8597476}]}]},{"op":"mcm","clk":"1801026105","pt":1451593397438,"mc":[{"id":"1.122431815","rc":[{"ltp":1.66,"id":8597476}]}]} """

julia> jsonstring=jsonstring=replace(jsonstring, r"}\s*{" => "},{")

julia> jsonstring=""" {"a":[ """*jsonstring*"]}"

julia> p=JSON.parse(jsonstring)

julia> p["a"]
2-element Array{Any,1}:
 Dict{String,Any}("clk" => "1801003563","op" => "mcm","pt" => 1451592321710,"mc" => Any[Dict{String,Any}("rc" => Any[Dict{String,Any}("ltp" => 1.67,"id" => 8597476)],"id" => "1.122431815")])
 Dict{String,Any}("clk" => "1801026105","op" => "mcm","pt" => 1451593397438,"mc" => Any[Dict{String,Any}("rc" => Any[Dict{String,Any}("ltp" => 1.66,"id" => 8597476)],"id" => "1.122431815")])
1 Like