Does rand(Float64, 1000) returns an array of consecutive physical addresses?

I’m new to julia. I want to know how to apply for an array with contiguous physical addresses.

Yes, it returns a Vector{Float64}, which is contiguous in memory.

Incredible. What if applying for 320G memory with the rand(Float64, 40*2^30)? I’m curious about what C interface is used at the bottom layer to ensure that the physical addresses are contiguous.

If you don’t have that much memory available you will get an OutOfMemoryError:

julia> rand(Float64, 40*2^30)
ERROR: OutOfMemoryError()

malloc I believe.

Thanks for your help!

Note that while currently this uses malloc in the future it is completely possible that the mechanism could change to another allocator (eg gemalloc).

Technically, the operating system kernel allocates for you a consecutive sequence of virtual addresses, but these may only correspond to consecutive “physical” addresses within a given VM page. Not that it makes any practical difference — accessing consecutive virtual addresses is fast.

Probably TMI, but there is plenty of information online about virtual address spaces if you are curious.

4 Likes

Funny video! That is to say, the virtual address of the user mode is continuous, but for the rand function of the order of 320G space, there is no guarantee that the physical address is also continuous. Is my understanding correct?

When you allocate an array with more than 4KiB of space (the page size, I think), the physical addresses might not be contiguous. (Again, this is a low-level operating-system/hardware thing — not specific to Julia!)

(On top of that, in some cases the operating system might “swap” memory out to disk, so it might not reside in RAM at all. You can also request this for memory-mapped files using the Mmap standard library.)

3 Likes