Hey, I am trying to run something quite large in Julia, and I need to store a large number of Arrays (for constructing sparse matrices). Below is a simplified version of what I need to do, where it creates a random array, and saves it to a file. The dimensions are approximately what I need to use.
(Running Julia 1.1.1 on Ubuntu 18.04)
for j in 1:1200 file = h5open("example$(j).h5","w") write(file, "X", rand(1,700000)) close(file) GC.gc() end
I don’t see why it is eating up memory when running. My machine has 16GB of RAM, will run out of memory before the above will complete. I have tried lots of ways to get around it, I was originally using JLD, which didn’t have this problem when writing the files, but encountered the same issue when reading the files.
Thanks in advance,