Easiest way to load a DataFrame from a compressed, newline delimited json file on the cloud?

Specific Domains Data
3

We’re actively thinking/working on a better overall Tables.jl solution here, but for now, DataFrames.jl has this functionality in push!, so in your case, something like:

df = open(path) do file
    df = DataFrame()
    for line in eachline(GzipDecompressorStream(file))
        push!(df, JSON3.read(line); cols=:union)
    end
end
3 Likes