# Dict loaded from jld file very long time... Why 22 seconds?

**URL:** <https://discourse.julialang.org/t/dict-loaded-from-jld-file-very-long-time-why-22-seconds/1629>\
**Category:** General Usage\
**Tags:** jld\
**Created:** [January 21, 2017, 6:58pm UTC](https://discourse.julialang.org/t/dict-loaded-from-jld-file-very-long-time-why-22-seconds/1629 "2017-01-21T18:58:18Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![programista](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/programista/32/5372_2.png) [@programista](https://discourse.julialang.org/u/programista)\
**Post date:** [January 21, 2017, 6:58pm UTC](https://discourse.julialang.org/t/dict-loaded-from-jld-file-very-long-time-why-22-seconds/1629/1 "2017-01-21T18:58:19Z")

</div>

My Dict  
julia\> DTT\_HH  
Dict{Int64,Any} with 381701 entries:  
61670 =\> 4150042  
119601 =\> 5600255  
136672 =\> 8300228  
231648 =\> 5600221  
? =\> ?

julia\> save(“TT\_Dicts.jld”,“DTT\_HH”,DTT\_HH)

julia\> @time x=load(“TT\_Dicts.jld”,“DTT\_HH”)  
22.373567 seconds (10.89 M allocations: 476.183 MB, 9.09% gc time)  
Dict{Int64,Any} with 381701 entries:  
247825 =\> 6300114  
43031 =\> 7500068  
117566 =\> 5700126  
231648 =\> 5600221  
? =\> ?

Similar Dict is loading in 0.2 sek

julia\> D=Dict(zip([1:1:500000;],rand(500000)))  
Dict{Int64,Float64} with 500000 entries:  
247825 =\> 0.40960171167545956  
450728 =\> 0.7363745698597213  
43031 =\> 0.8869847906671853  
253410 =\> 0.03773283268230743  
? =\> ?

julia\> save(“Dict.jld”,“D”,D)

julia\> @time D1=load(“Dict.jld”,“D”)  
0.281323 seconds (563 allocations: 33.155 MB, 77.93% gc time)  
Dict{Int64,Float64} with 500000 entries:  
247825 =\> 0.40960171167545956  
450728 =\> 0.7363745698597213  
43031 =\> 0.8869847906671853  
349542 =\> 0.507127647847206  
? =\> ?

---

<div class="post-metadata">

**Author:** ![programista](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/programista/32/5372_2.png) [@programista](https://discourse.julialang.org/u/programista)\
**Post date:** [January 21, 2017, 7:15pm UTC](https://discourse.julialang.org/t/dict-loaded-from-jld-file-very-long-time-why-22-seconds/1629/2 "2017-01-21T19:15:02Z")

</div>

julia\> DTT\_HH  
Dict{Int64,Any} with 381701 entries:

what to do with Any Array ?  
Any idea ?  
Paul

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [January 21, 2017, 8:14pm UTC](https://discourse.julialang.org/t/dict-loaded-from-jld-file-very-long-time-why-22-seconds/1629/3 "2017-01-21T20:14:40Z")

</div>

Maybe. What happens if you make that same dictionary be strictly typed?

---

<div class="post-metadata">

**Author:** ![programista](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/programista/32/5372_2.png) [@programista](https://discourse.julialang.org/u/programista)\
**Post date:** [January 22, 2017, 11:06am UTC](https://discourse.julialang.org/t/dict-loaded-from-jld-file-very-long-time-why-22-seconds/1629/4 "2017-01-22T11:06:21Z")

</div>

W dniu 2017-01-21 21:18, Christopher Rackauckas pisze:

> What happens if you make that same dictionary be strictly typed?

Sorry, _“strictly typed”_ I understand , What do You mean ?  
Paul

---

<div class="post-metadata">

**Author:** ![felix](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/felix/32/604_2.png) [@felix](https://discourse.julialang.org/u/felix)\
**Post date:** [January 22, 2017, 11:21am UTC](https://discourse.julialang.org/t/dict-loaded-from-jld-file-very-long-time-why-22-seconds/1629/5 "2017-01-22T11:21:25Z")

</div>

The `Dict{K,V}` notation means its keys have type `K` and its values have type `V`. In your slow example, `Dict{Int64,Any}`, the keys are `Int64` but there is no available type information about the values (hence they are recorded as the `Any` type). It may perform faster if, instead of `Any`, the values are all of the same concrete type _and this is known to the compiler_. Notice that’s what happened in your fast-performing example `Dict{Int64,Float64}`, all the values were guaranteed to be of the same type `Float64`. This situation is called “strictly-typed”, unlike the case where values have `Any` type. Having strict type information can allow Julia to make significant optimisations.

---

<div class="post-metadata">

**Author:** ![programista](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/programista/32/5372_2.png) [@programista](https://discourse.julialang.org/u/programista)\
**Post date:** [January 22, 2017, 11:32am UTC](https://discourse.julialang.org/t/dict-loaded-from-jld-file-very-long-time-why-22-seconds/1629/6 "2017-01-22T11:32:34Z")

</div>

W dniu 2017-01-22 12:24, felix pisze:

> [felix] felix [http://discourse.julialang.org/users/felix](http://discourse.julialang.org/users/felix)  
> January 22
> 
> The |Dict{K,V}| notation means its keys have type |K| and its values  
> have type |V|. In your slow example, |Dict{Int64,Any}|, the keys are  
> |Int64| but there is no available type information about the values  
> (hence they are recorded as the |Any| type). It may perform faster if,  
> instead of |Any|, the values are all of the same concrete type /and  
> this is known to the compiler/. Notice that’s what happened in your  
> fast-performing example |Dict{Int64,Float64}|, all the values were  
> guaranteed to be of the same type |Float64|. This situation is called  
> “strictly-typed”, unlike the case where values have |Any| type. Having  
> strict type information can allow Julia to make significant optimisations.
> 
> * * *
> 
> Visit Topic  
> [http://discourse.julialang.org/t/dict-loaded-from-jld-file-very-long-time-why-22-seconds/1629/5](http://discourse.julialang.org/t/dict-loaded-from-jld-file-very-long-time-why-22-seconds/1629/5)  
> or reply to this email to respond.
> 
> * * *
> 
> ```
> In Reply To
> 
> ```
> 
> [programista] programista  
> [http://discourse.julialang.org/users/programista](http://discourse.julialang.org/users/programista)  
> January 22
> 
> W dniu 2017-01-21 21:18, Christopher Rackauckas pisze: What happens if  
> you make that same dictionary be strictly typed? Sorry, “strictly  
> typed” I understand , What do You mean ? Paul
> 
> * * *
> 
> Visit Topic  
> [http://discourse.julialang.org/t/dict-loaded-from-jld-file-very-long-time-why-22-seconds/1629/5](http://discourse.julialang.org/t/dict-loaded-from-jld-file-very-long-time-why-22-seconds/1629/5)  
> or reply to this email to respond.
> 
> To unsubscribe from these emails, click here  
> [http://discourse.julialang.org/email/unsubscribe/e6178c85c63318f02dd27acb2edeee161ce9b8cd1fbfce1151c2c2b64ccb143e](http://discourse.julialang.org/email/unsubscribe/e6178c85c63318f02dd27acb2edeee161ce9b8cd1fbfce1151c2c2b64ccb143e).

Ok,  
But What to do if I have valuse as string?

julia\> TT\_baza=readcsv(“HH\_baza/TT\_baza.txt”)  
381701x3 Array{Any,2}:  
1 100139 “T1\_0”  
1 100143 “T1\_0”  
1 100147 “T1\_0”

julia\> @time DTT=Dict(zip(1:size(TT\_baza,1),TT\_baza[:,3]))  
0.765525 seconds (2.82 M allocations: 93.261 MB, 9.13% gc time)  
Dict{Int64,Any} with 381701 entries:  
247825 =\> “T1\_0”  
43031 =\> “T2\_0”  
349542 =\> “T1\_0”

Paul

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [January 22, 2017, 3:03pm UTC](https://discourse.julialang.org/t/dict-loaded-from-jld-file-very-long-time-why-22-seconds/1629/7 "2017-01-22T15:03:07Z")

</div>

That array is still typed with `Any`

---

<div class="post-metadata">

**Author:** ![holger](https://avatars.discourse-cdn.com/v4/letter/h/ed8c4c/32.png) [@holger](https://discourse.julialang.org/u/holger)\
**Post date:** [August 24, 2017, 9:54pm UTC](https://discourse.julialang.org/t/dict-loaded-from-jld-file-very-long-time-why-22-seconds/1629/8 "2017-08-24T21:54:58Z")

</div>

I ran into a similar problem with JLD, it took over 20 minutes to load my data. I just wanted to leave this here if anybody runs into something similar, with [JLD2](https://discourse.julialang.org/t/ann-jld2-jld-in-pure-julia/4753) it only took 50 seconds to load the same data.

```julia
using JLD2
@time load("outfiles/aggregated_results.jld2")
 49.773390 seconds (9.72 M allocations: 2.651 GiB, 7.02% gc time)
Dict{String,Any} with 7 entries:

```