Performance of Memory Mapped Arrays (vs. JLD2)

I think mmap needs to read the data from disk first, cf https://stackoverflow.com/questions/31014515/write-only-mapping-a-o-wronly-opened-file-supposed-to-work.

Otoh, lustre is ZFS based, right? Sparse files / RLE / LZ4 should make reading/storing a compressible file extremely fast, and there is nothing more compressible than a bunch of zeros (which is your initial state). Or do you need to talk to some other device doing the storage that maybe fails to do compression on some path? Or is compression disabled for some reason?

Try this with zeros. If this takes an appreciable amount of time / disk IO / network IO, then complain to your storage people. If this is fast, then remember that it is only fast for write-once, i.e. you need to delete your file and create a new one if you want to regenerate your matrix.