JLD seems to take an awfully long time just reading the names present in a JLD file – not reading values, just names.
julia> outfile = jldopen("/tmp/blah.jl", "w") Julia data file version 0.1.1: /tmp/blah.jl julia> for i in 1:10000 write(outfile, "$(rand())", rand()) end julia> names(outfile) #compile (though it doesn't seem to make any difference) julia> @time names(outfile) 44.540928 seconds (40.15 k allocations: 1.688 MB)
When the file contains ~75,000 entries. this names() call takes several hours.
On the contrary,
time h5ls /tmp/blah.jl
real 0m0.284s user 0m0.236s sys 0m0.030s
What is going on? How do I fix this? (I’m willing to mess around with the internals of JLD if necessary)