# Memory usage with large JSON

**URL:** <https://discourse.julialang.org/t/memory-usage-with-large-json/51277>\
**Category:** Performance\
**Created:** [December 4, 2020, 9:38pm UTC](https://discourse.julialang.org/t/memory-usage-with-large-json/51277 "2020-12-04T21:38:12Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![evanfields](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/evanfields/32/1744_2.png) [@evanfields](https://discourse.julialang.org/u/evanfields)\
**Post date:** [December 4, 2020, 9:38pm UTC](https://discourse.julialang.org/t/memory-usage-with-large-json/51277/1 "2020-12-04T21:38:12Z")

</div>

I was using `JSON` to parse a large (1.6 GB) JSON file, and the memory usage was surprising. More surprisingly, perhaps, was that even after removing the reference to the parsed object and garbage collecting, the memory usage remained high:

```julia
using JSON
# write and parse a 25 byte JSON to compile
# Julia memory usage ~200 MB
d = JSON.parsefile("really_big.json")
# takes a long time
# Julia memory usage 12 GB
d = nothing
Base.GC.gc()
# Julia memory usage 8 GB

```

Is this pattern expected? Can anything be done to mitigate? Thanks!

---

<div class="post-metadata">

**Author:** ![Oscar\_Smith](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oscar_smith/32/25343_2.png) [@Oscar\_Smith](https://discourse.julialang.org/u/Oscar_Smith)\
**Post date:** [December 4, 2020, 9:39pm UTC](https://discourse.julialang.org/t/memory-usage-with-large-json/51277/2 "2020-12-04T21:39:13Z")

</div>

Have you tried `JSON3`? From what I know it generally is just better than `JSON`

---

<div class="post-metadata">

**Author:** ![evanfields](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/evanfields/32/1744_2.png) [@evanfields](https://discourse.julialang.org/u/evanfields)\
**Post date:** [December 5, 2020, 7:20pm UTC](https://discourse.julialang.org/t/memory-usage-with-large-json/51277/3 "2020-12-05T19:20:58Z")

</div>

The situation is indeed better with JSON3. Is this a known problem of standard lib JSON? Is JSON3 uniformly recommended instead?

---

<div class="post-metadata">

**Author:** ![Oscar\_Smith](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oscar_smith/32/25343_2.png) [@Oscar\_Smith](https://discourse.julialang.org/u/Oscar_Smith)\
**Post date:** [December 5, 2020, 7:42pm UTC](https://discourse.julialang.org/t/memory-usage-with-large-json/51277/4 "2020-12-05T19:42:13Z")

</div>

As far as I know, it is. The main reason we still have `JSON` around is that we need something in the STDLIB that can parse JSON for package management stuff.

---

<div class="post-metadata">

**Author:** ![ericphanson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ericphanson/32/215186_2.png) [@ericphanson](https://discourse.julialang.org/u/ericphanson)\
**Post date:** [December 5, 2020, 9:07pm UTC](https://discourse.julialang.org/t/memory-usage-with-large-json/51277/5 "2020-12-05T21:07:00Z")

</div>

> [@evanfields](#):
>
> standard lib JSON

> [@Oscar\_Smith](#):
>
> The main reason we still have `JSON` around is that we need something in the STDLIB that can parse JSON for package management stuff.

JSON isn’t a standard library, it’s just a normal package; maybe you’re thinking of TOML?

---

<div class="post-metadata">

**Author:** ![evanfields](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/evanfields/32/1744_2.png) [@evanfields](https://discourse.julialang.org/u/evanfields)\
**Post date:** [December 6, 2020, 12:45am UTC](https://discourse.julialang.org/t/memory-usage-with-large-json/51277/6 "2020-12-06T00:45:19Z")

</div>

Shows what I know. Learned something new tonight–thanks!