How to stream a large random matrix to a file?

General Usage
,
4

tldr: call open(… “w”) as in the example provided by @stevengj here.

Thanks to @oheil for spotting additional bugs.

The following works:

using Test,Mmap

@testset "stream an m x n matrix to a file and read via Mmap" begin
    m=10
    n=10^5
    fn="/tmp/mmap.bin"
    s = open(fn, "w")
    a1 = rand(m)
    for i in 1:n
        i == 1 ? write(s,a1) : write(s, rand(m))
    end
    close(s)
    @test isfile(fn)

    s = open("/tmp/mmap.bin")
    A = Mmap.mmap(s, Matrix{Float64}, (m,n))
    @test A[:,1] == a1
end

Test Summary:                                      | Pass  Total
stream an m x n matrix to a file and read via Mmap |    2      2
Test.DefaultTestSet("stream an m x n matrix to a file and read via Mmap", Any[], 2, false)
1 Like