# JLD takes too long reading names from a file

**URL:** <https://discourse.julialang.org/t/jld-takes-too-long-reading-names-from-a-file/2154>\
**Category:** Data\
**Tags:** jld\
**Created:** [February 17, 2017, 2:40pm UTC](https://discourse.julialang.org/t/jld-takes-too-long-reading-names-from-a-file/2154 "2017-02-17T14:40:24Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![LewisHein](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lewishein/32/5690_2.png) [@LewisHein](https://discourse.julialang.org/u/LewisHein)\
**Post date:** [February 17, 2017, 2:40pm UTC](https://discourse.julialang.org/t/jld-takes-too-long-reading-names-from-a-file/2154/1 "2017-02-17T14:40:24Z")

</div>

JLD seems to take an awfully long time just reading the names present in a JLD file – not reading values, just names.

```
julia> outfile = jldopen("/tmp/blah.jl", "w")
Julia data file version 0.1.1: /tmp/blah.jl

julia> for i in 1:10000
   write(outfile, "$(rand())", rand())
   end
julia> names(outfile) #compile (though it doesn't seem to make any difference)
julia> @time names(outfile)
44.540928 seconds (40.15 k allocations: 1.688 MB)

```

When the file contains ~75,000 entries. this names() call takes several hours.

On the contrary,

```
  time h5ls /tmp/blah.jl

```

Takes only

```
 real	0m0.284s
 user	0m0.236s
 sys	0m0.030s

```

What is going on? How do I fix this? (I’m willing to mess around with the internals of JLD if necessary)

---

<div class="post-metadata">

**Author:** ![vchuravy](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/vchuravy/32/8_2.png) [@vchuravy](https://discourse.julialang.org/u/vchuravy)\
**Post date:** [April 11, 2017, 9:12am UTC](https://discourse.julialang.org/t/jld-takes-too-long-reading-names-from-a-file/2154/2 "2017-04-11T09:12:23Z")

</div>

> [@LewisHein](#):
>
> @time names(outfile)

A quick investigation with `@profile` and `ProfileView.view(C=true)` shows that most of the time is spend within `HDF5` especially the [`H5Gget_objname_by_idx`](https://support.hdfgroup.org/HDF5/doc/RM/RM_H5G.html#Group-GetObjNameByIdx) function (see [https://github.com/JuliaIO/HDF5.jl/blob/0366bb050d8ded8dff2d8f148818151610bbb75b/src/HDF5.jl#L987](https://github.com/JuliaIO/HDF5.jl/blob/0366bb050d8ded8dff2d8f148818151610bbb75b/src/HDF5.jl#L987) where the call originates).

Since profiling seems to indicate the time is not spend within Julia `h5ls` must be using a different API to access the information or `HDF5.jl` is using the API the wrong way.

---

<div class="post-metadata">

**Author:** ![kristoffer.carlsson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kristoffer.carlsson/32/22_2.png) [@kristoffer.carlsson](https://discourse.julialang.org/u/kristoffer.carlsson)\
**Post date:** [April 11, 2017, 9:28am UTC](https://discourse.julialang.org/t/jld-takes-too-long-reading-names-from-a-file/2154/3 "2017-04-11T09:28:19Z")

</div>

FWIW:

```julia
using JLD2

Base.names(jld::JLD2.JLDFile) = keys(jld.datasets)

outfile = jldopen("/tmp/blah.jl", "w");
for i in 1:10000
    write(outfile, "$(rand())", rand())
end
close(outfile)

```

```julia
julia> @time infile = jldopen("/tmp/blah.jl", "r");
  0.000445 seconds (141 allocations: 11.354 KiB)

julia> @time names(infile);
  0.008764 seconds (3.38 k allocations: 169.777 KiB)

```

[https://github.com/simonster/JLD2.jl](https://github.com/simonster/JLD2.jl)