How to read JSON from HTML?

Specific Domains Web Stack
, ,
1

I know we can use Requests.jl to get the data, but I am not sure how we can parse the JSON with JSON.jl:

using Requests, JSON

r = get("https://api.github.com/users/juliohm")
parse(r.data) # should JSON.Parser.parse work here?
1 Like
2

I think you should probably try HTTP.jl

I don’t think Requests is recommended or actively maintained
quoting malmaud the biggest contributor

I think HTTP.jl is probably the future of the Julia we stack. Would be
awesome to get it merged there.

1 Like
3

@musm my choice for Requests.jl over HTTP.jl was based on the number of stars on GitHub (100 vs. 19), could you please confirm HTTP.jl is the future?

4

I can’t predict the future but here is the original quote HTTP/2 Support by sorpaas · Pull Request #133 · JuliaWeb/Requests.jl · GitHub

1 Like
5

The answer with HTTP.jl:

using HTTP, JSON

resp = HTTP.get("https://api.github.com/users/juliohm")
str = String(resp.body)
jobj = JSON.Parser.parse(s)
3 Likes
6

@juliohm, should it be JSON.parse(str) * ?

7

@miguelraz I don’t remember what happened, but I had to use JSON.Parser.parse(str) instead as I wrote on my answer. Maybe it has changed since then.

1 Like
8

JSON.parse should work, but will not autocomplete in the REPL due to a Julia REPL limitation.

1 Like
9

I am new to julia and came here searching for HTTP scraping…

This is our minimal sample (working as of 20201008):

import Pkg; 
Pkg.add("HTTP")
Pkg.add("JSON")
Pkg.add("JSON")
Pkg.add("JSON3")
Pkg.add("LazyJSON")


import HTTP
import JSON
import JSON;
import JSON3;
import LazyJSON;

using JSON
using HTTP
using JSON;
using JSON3;
using LazyJSON;

# tags
# https://api.github.com/repos/xamarin/AndroidX/tags
# https://api.github.com/repos/xamarin/Essentials/tags
# releases
# https://api.github.com/repos/xamarin/AndroidX/releases
# https://api.github.com/repos/xamarin/Essentials/releases

url = "https://raw.githubusercontent.com/xamarin/AndroidX/20200915-mono.cecil-fix/config.json"
resp = HTTP.get(url)

str = String(resp.body)
println("str = ", str)

jobj_JSON = JSON.parse(str)
println("jobj_JSON = ", jobj_JSON)

jobj_JSON3 = JSON3.read(str)
println("jobj_JSON3 = ", jobj_JSON)


jobj_LazyJSON = LazyJSON.value(str)
println("jobj_LazyJSON = ", jobj_LazyJSON)