Dear All, I’m handling a large JSON file (in my case, a 100GB of ndjson). I would need to query it in various and still TBD ways. In my lucky case we can rely on a fixed schema, which is nested (although not deep, we got only three levels down). I could transform this into a database or a csv, e.g…

wrangling large json files

swt30 December 10, 2018, 10:20am 3

I don’t think any of Julia’s JSON packages currently support ndJSON—it would probably be a simple change, but worth keeping in mind that they probably won’t work for you out of the box.

1 Like

Topic		Replies	Views
Reading a large JSON file make Julia crashing Data	10	1195	December 22, 2021
Announce: A different way to read JSON data, LazyJSON.jl Data	19	10126	October 2, 2018
JSON Performance Tests Data	7	2306	November 6, 2018
Recommended way to save and read DataFrames in JSON format Data	8	3741	February 12, 2018
Processing JSON from a .txt file and converting to a DataFrame New to Julia dataframes , json3	7	2646	May 15, 2021

wrangling large json files

Related topics