# Loading entire jld2 file

**URL:** https://discourse.julialang.org/t/loading-entire-jld2-file/42610
**Category:** General Usage
**Created:** [July 6, 2020, 1:55pm UTC](https://discourse.julialang.org/t/loading-entire-jld2-file/42610 "2020-07-06T13:55:41Z")
**Posts on this page:** 13
**Page:** 1

<div class="post-metadata">

### Author: ![gideonsimpson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gideonsimpson/32/1928_2.png) [@gideonsimpson](https://discourse.julialang.org/u/gideonsimpson)
#### Post date: [July 6, 2020, 1:55pm UTC](https://discourse.julialang.org/t/loading-entire-jld2-file/42610/1 "2020-07-06T13:55:41Z")

</div>

If I’ve saved my data to a `jld2` format file using `FileIO`, is there a quick way to just load in the entire file at once? I seem to have to name all the entries in the file one by one.

---

<div class="post-metadata">

### Author: ![kristoffer.carlsson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kristoffer.carlsson/32/22_2.png) [@kristoffer.carlsson](https://discourse.julialang.org/u/kristoffer.carlsson)
#### Post date: [July 6, 2020, 1:59pm UTC](https://discourse.julialang.org/t/loading-entire-jld2-file/42610/2 "2020-07-06T13:59:08Z")

</div>

I think it is just `@load file`.

---

<div class="post-metadata">

### Author: ![hendri54](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/hendri54/32/9621_2.png) [@hendri54](https://discourse.julialang.org/u/hendri54)
#### Post date: [July 6, 2020, 4:13pm UTC](https://discourse.julialang.org/t/loading-entire-jld2-file/42610/3 "2020-07-06T16:13:04Z")

</div>

You may have seen this, but the package page now says:

**WARNING** : This package is not actively maintained and has several known bugs that are unlikely to be resolved in the near future. Please consider using [Julia’s built-in serializer](https://docs.julialang.org/en/v1/stdlib/Serialization/) if you need performance, or [the previous JLD package](https://github.com/JuliaIO/JLD.jl) if you need HDF5 compatibility.

---

<div class="post-metadata">

### Author: ![gideonsimpson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gideonsimpson/32/1928_2.png) [@gideonsimpson](https://discourse.julialang.org/u/gideonsimpson)
#### Post date: [July 6, 2020, 4:47pm UTC](https://discourse.julialang.org/t/loading-entire-jld2-file/42610/4 "2020-07-06T16:47:52Z")

</div>

I hadn’t noticed that. Is there a tutorial anywhere on using the serializer as a replacement for `JLD2`? Alternatively, is there a way to just use `FileIO` to handle this?

---

<div class="post-metadata">

### Author: ![gideonsimpson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gideonsimpson/32/1928_2.png) [@gideonsimpson](https://discourse.julialang.org/u/gideonsimpson)
#### Post date: [July 6, 2020, 4:55pm UTC](https://discourse.julialang.org/t/loading-entire-jld2-file/42610/5 "2020-07-06T16:55:51Z")

</div>

Alternatively, what’s the recommended way to save a bunch of variables to disk now? I am not specifically wedded to HDF5 based tools, though they certainly have advantages.

---

<div class="post-metadata">

### Author: ![biona001](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/biona001/32/16497_2.png) [@biona001](https://discourse.julialang.org/u/biona001)
#### Post date: [July 6, 2020, 6:30pm UTC](https://discourse.julialang.org/t/loading-entire-jld2-file/42610/6 "2020-07-06T18:30:41Z")

</div>

Maybe you can define your own `struct` that contains all of your variables, and then just save/load that 1 variable. Not sure if this is sensible…

---

<div class="post-metadata">

### Author: ![hendri54](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/hendri54/32/9621_2.png) [@hendri54](https://discourse.julialang.org/u/hendri54)
#### Post date: [July 6, 2020, 8:13pm UTC](https://discourse.julialang.org/t/loading-entire-jld2-file/42610/7 "2020-07-06T20:13:31Z")

</div>

There isn’t one recommended way. Lots of people ask that question, and you can find lots of answers if you search discourse for threads about saving / loading user defined types. The package to use depends on the type of data that you plan to store.

One recommendation that I have seen a few times and that I am beginning to come around to: write your own converter from your user defined types to something easy to serialize (e.g. a Dict) and then store that in a format that is robust (e.g. JSON).

I have not used the serialize / deserialize approach because those files can become unreadable at any time. They are meant for short term storage only.

---

<div class="post-metadata">

### Author: ![gideonsimpson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gideonsimpson/32/1928_2.png) [@gideonsimpson](https://discourse.julialang.org/u/gideonsimpson)
#### Post date: [July 6, 2020, 8:40pm UTC](https://discourse.julialang.org/t/loading-entire-jld2-file/42610/8 "2020-07-06T20:40:11Z")

</div>

Does it appeare the `JLD` module have continued support? I can’t remember what the reason was, but for some reason they came out with `JLD2`. Was it because `JLD` depended on HDF5 library or something?

---

<div class="post-metadata">

### Author: ![xiaodai](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/xiaodai/32/15937_2.png) [@xiaodai](https://discourse.julialang.org/u/xiaodai)
#### Post date: [July 7, 2020, 12:38am UTC](https://discourse.julialang.org/t/loading-entire-jld2-file/42610/9 "2020-07-07T00:38:50Z")

</div>

> [@gideonsimpson](#):
>
> my data

Is ur data tabular? If it is these r your options [Unable to save Unitful unit into a JLD file - #10 by xiaodai](https://discourse.julialang.org/t/unable-to-save-unitful-unit-into-a-jld-file/40706/10)

---

<div class="post-metadata">

### Author: ![hendri54](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/hendri54/32/9621_2.png) [@hendri54](https://discourse.julialang.org/u/hendri54)
#### Post date: [July 7, 2020, 11:24am UTC](https://discourse.julialang.org/t/loading-entire-jld2-file/42610/10 "2020-07-07T11:24:46Z")

</div>

I have been thinking: perhaps the problem is not nearly as hard as it looks.  
The hard part is constructors. We are asking our code to load a `Foo{T}` struct and then figure out how to construct that object.  
At least in my applications, I know how to construct the object that I’m trying to load. Moreover, pretty much everything that I am trying to save and load is a `DataTypes` object (in the sense of [StructTypes.jl](https://juliadata.github.io/StructTypes.jl/dev/)).  
I think I could cover 95% of my use cases (especially the most tricky ones where I try to save nested user defined types) with the simple approach of:

1. Recursively convert the object to a `Dict{:Symbol, simple known types}`
2. Save that to a robust format, say JSON.
3. When loading, provide a randomly initialized, fully constructed `Foo{T}` object. Recursively walk the loaded `Dict` and copy values into `Foo`.

This only works when everything is mutable, but that would be a price I would be willing to pay. Am I missing a reason why this would be a bad idea?

---

<div class="post-metadata">

### Author: ![kristoffer.carlsson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kristoffer.carlsson/32/22_2.png) [@kristoffer.carlsson](https://discourse.julialang.org/u/kristoffer.carlsson)
#### Post date: [July 7, 2020, 12:47pm UTC](https://discourse.julialang.org/t/loading-entire-jld2-file/42610/11 "2020-07-07T12:47:12Z")

</div>

> [@hendri54](#):
>
> The hard part is constructors.

What makes you say that? When you deserialize something you don’t run the constructor, you just directly create the type from the data.

---

<div class="post-metadata">

### Author: ![gideonsimpson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gideonsimpson/32/1928_2.png) [@gideonsimpson](https://discourse.julialang.org/u/gideonsimpson)
#### Post date: [July 7, 2020, 1:12pm UTC](https://discourse.julialang.org/t/loading-entire-jld2-file/42610/12 "2020-07-07T13:12:44Z")

</div>

My data is a couple of arrays (which could be large) along with some scalar parameters. The JLD/JLD2 file formats worked pretty well for this.

---

<div class="post-metadata">

### Author: ![hendri54](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/hendri54/32/9621_2.png) [@hendri54](https://discourse.julialang.org/u/hendri54)
#### Post date: [July 7, 2020, 5:40pm UTC](https://discourse.julialang.org/t/loading-entire-jld2-file/42610/13 "2020-07-07T17:40:20Z")

</div>

In part, I am basing this on my experience with JLD2. If I define a `Foo` object in some module `A` and later load it, I need to [issue `using A` in `Main`](https://github.com/JuliaIO/JLD2.jl/issues/134). Otherwise, my understanding is that JLD2 returns its own version of a `Foo` object (not an `A.Foo`).

More generally, when I thought about how to actually write code that saves / loads the structs that I define, I could not figure out how to call the constructor correctly. As an example, if I try to load an `A.Foo` that contains a `B.Bar`, I would have to bring `A` and `B` into the namespace and then determine (from propertynames?) how to call the constructors (recursively).

Things get more complicated when the types are parameterized.
