RAM needed to initialise large matrices

I don’t think this is true? Calling zeros(UInt8, 1000_000_000) seems to first invoke the undef constructor to get v = Vector{UInt8}(undef, (1000_000_000,)) which doesn’t have to allocate physical memory when it’s done via mmap. But this is followed by fill!(v, zero(UInt8)) which would touch every byte in the array, causing many OS page faults which in turn allocate physical memory pages.

For another interesting thread on page faults, see here: Julia slower in Windows - #16 by c42f

(Edit: As a side note - it seems technically possible to do lazy zero fill in (a) the case that a large array happens to be allocated with mmap and MAP_ANONYMOUS somewhere deep in the allocator and (b) zero(eltype(a)) happens to be a bits type with a bit pattern of zeros. I think this would only help in specialized circumstances, though.)

6 Likes