Struggling to use Mmap with ZipArchives

General Usage
,
1

I’m struggling to get ZipArchives.jl and Mmap.jl to work together in the way I want - basically just following the docs.

MWE looks like this:

using ZipArchives
using Mmap

io=open("simpletest.zip")
mm=Mmap.mmap(io)
q = ZipArchives.ZipReader(mm)
readme_n_lines = zip_openentry(q, "xl/worksheets/sheet1.xml") do z
    countlines(z)
end
close(io)
#GC.gc()
ZipArchives.ZipWriter("simpletest.zip") do w 
    zip_newfile(w, "test/test2.txt")
    write(w, "I am data inside test2.txt in the zip file")
end

Which produces the following:

ERROR: LoadError: SystemError: opening file "simpletest.zip": Invalid argument
Stacktrace:
  [1] systemerror(p::String, errno::Int32; extrainfo::Nothing)
    @ Base .\error.jl:176
  [2] systemerror
    @ .\error.jl:176
  [3] systemerror
    @ .\error.jl:175 [inlined]
  [4] open(fname::String; lock::Bool, read::Nothing, write::Bool, create::Nothing, truncate::Nothing, append::Nothing)
    @ Base .\iostream.jl:295
  [5] open
    @ .\iostream.jl:277 [inlined]
  [6] #ZipWriter#17
    @ C:\Users\tim\.julia\packages\ZipArchives\5fdTS\src\writer.jl:65 [inlined]
  [7] ZipWriter(f::Function, filename::String)
    @ ZipArchives C:\Users\tim\.julia\packages\ZipArchives\5fdTS\src\writer.jl:64
  [8] top-level scope
    @ c:\Users\tim\OneDrive\Documents\Julia\ZipArchives\TestMmap.jl:13
  [9] include(fname::String)
    @ Main .\sysimg.jl:38
 [10] run(debug_session::VSCodeDebugger.DebugAdapter.DebugSession, error_handler::VSCodeDebugger.var"#3#4"{String})
    @ VSCodeDebugger.DebugAdapter c:\Users\tim\.vscode\extensions\julialang.language-julia-1.144.2\scripts\packages\DebugAdapter\src\packagedef.jl:123
 [11] startdebugger()
    @ VSCodeDebugger c:\Users\tim\.vscode\extensions\julialang.language-julia-1.144.2\scripts\packages\VSCodeDebugger\src\VSCodeDebugger.jl:47
 [12] top-level scope
    @ c:\Users\tim\.vscode\extensions\julialang.language-julia-1.144.2\scripts\debugger\run_debugger.jl:12
 [13] include(mod::Module, _path::String)
    @ Base .\Base.jl:557
 [14] exec_options(opts::Base.JLOptions)
    @ Base .\client.jl:323
 [15] _start()
    @ Base .\client.jl:531
in expression starting at c:\Users\tim\OneDrive\Documents\Julia\ZipArchives\TestMmap.jl:13

I found this old thread, so I tried swapping the close for a GC.gc() or doing both (close first) or doing neither. Every case resulted in the same error.

In this case, the zip file is a renamed Excel file but, to be honest, I don’t think the file content is causing this issue.

Am I doing something daft, or is there something awry in ZipArchives.jl?

Edit: to add:

pkg> st
Status `C:\Users\tim\OneDrive\Documents\Julia\ZipArchives\Project.toml`
  [72c71f33] XML v0.3.5
  [49080126] ZipArchives v2.4.1
  [a63ad114] Mmap v1.11.0

julia> versioninfo()
Julia Version 1.11.5
Commit 760b2e5b73 (2025-04-14 06:53 UTC)
Build Info:
  Official https://julialang.org/ release
Platform Info:
  OS: Windows (x86_64-w64-mingw32)
  CPU: 24 × AMD Ryzen 9 9900X 12-Core Processor
  WORD_SIZE: 64
  LLVM: libLLVM-16.0.6 (ORCJIT, generic)
Threads: 8 default, 0 interactive, 4 GC (on 24 virtual cores)
Environment:
  JULIA_EDITOR = code
  JULIA_VSCODE_REPL = 1
  JULIA_NUM_THREADS = 8
2

Your code as written (with a random xlsx file) runs with no errors for me.

julia> versioninfo()
Julia Version 1.11.5
Commit 760b2e5b739 (2025-04-14 06:53 UTC)
Build Info:
  Official https://julialang.org/ release
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: 32 × AMD Ryzen 9 9950X 16-Core Processor
  WORD_SIZE: 64
  LLVM: libLLVM-16.0.6 (ORCJIT, generic)
Threads: 32 default, 1 interactive, 16 GC (on 32 virtual cores)
Environment:
  JULIA_NUM_THREADS = auto,auto
  JULIA_REVISE_INCLUDE = 1

(jl_sgw3Xt) pkg> st
Status `/tmp/jl_sgw3Xt/Project.toml`
  [49080126] ZipArchives v2.4.1

From the stack trace, it looks like Mmap has nothing to do with the error, the error happens in a call to ZipWriter which is not using the mapped io. I wonder if the Windows version does not allow writing to an existing file without specifying some additional info? The ZipArchives documentation is not as clear about that as I would like.

3

Well, if I take the mmap out of the code, like this:

using ZipArchives
using Mmap

#io=open("simpletest.zip")
#mm=Mmap.mmap(io)
#q = ZipArchives.ZipReader(mm)
q = ZipArchives.ZipReader(read("simpleTest.zip"))
readme_n_lines = zip_openentry(q, "xl/worksheets/sheet1.xml") do z
    countlines(z)
end
#close(io)
#GC.gc()
ZipArchives.ZipWriter("simpleTest.zip") do w 
    zip_newfile(w, "test/test2.txt")
    write(w, "I am data inside test2.txt in the zip file")
end

I don’t see the error and I do get a revised file, so there does seem to be some dependency on mmap.

4

Perhaps the mmap is still holding some resource? Try putting each block in a function to see if that causes the mmap to be released.

function read()
    open("simpletest.zip") do io
        mm=Mmap.mmap(io)
        q = ZipArchives.ZipReader(mm)
        zip_openentry(q, "xl/worksheets/sheet1.xml") do z
            countlines(z)
        end
    end
end

function write()
    ZipArchives.ZipWriter("simpletest.zip") do w 
        zip_newfile(w, "test/test2.txt")
        write(w, "I am data inside test2.txt in the zip file")
    end
end

readme_n_lines = read()
write()
5

In my real life use case, the ZipReader and ZipWriter calls are distantly separated in different sets of nested calls. Trying the same in my MWE yields the same error as in the OP.

This example points the finger even more clearly at mmap, I think, although it doesn’t help me with a solution:

using ZipArchives
using Mmap

function reading()
    io=open("simpleTest.zip")
    mm=Mmap.mmap(io)
    q = ZipArchives.ZipReader(mm)
    #q = ZipArchives.ZipReader(read("simpleTest.zip"))
    readme_n_lines = zip_openentry(q, "xl/worksheets/sheet1.xml") do z
        countlines(z)
    end
    close(io)
    GC.gc()
end
#function writing()
#    ZipArchives.ZipWriter("simpleTest.zip") do w 
#        zip_newfile(w, "test/test2.txt")
#        write(w, "I am data inside test2.txt in the zip file")
#    end
#end

reading()
isfile("simpleTest.zip") && rm("simpleTest.zip")
#writing()

This generates the following:

ERROR: LoadError: IOError: unlink("simpleTest.zip"): permission denied (EACCES)
Stacktrace:
  [1] uv_error
    @ .\libuv.jl:106 [inlined]
  [2] unlink(p::String)
    @ Base.Filesystem .\file.jl:1105
  [3] rm(path::String; force::Bool, recursive::Bool)
    @ Base.Filesystem .\file.jl:283
  [4] rm(path::String)
    @ Base.Filesystem .\file.jl:273
  [5] top-level scope
    @ c:\Users\tim\OneDrive\Documents\Julia\ZipArchives\TestMmap.jl:24
  [6] include(fname::String)
    @ Main .\sysimg.jl:38
  [7] run(debug_session::VSCodeDebugger.DebugAdapter.DebugSession, error_handler::VSCodeDebugger.var"#3#4"{String})
    @ VSCodeDebugger.DebugAdapter c:\Users\tim\.vscode\extensions\julialang.language-julia-1.144.2\scripts\packages\DebugAdapter\src\packagedef.jl:123
  [8] startdebugger()
    @ VSCodeDebugger c:\Users\tim\.vscode\extensions\julialang.language-julia-1.144.2\scripts\packages\VSCodeDebugger\src\VSCodeDebugger.jl:47
  [9] top-level scope
    @ c:\Users\tim\.vscode\extensions\julialang.language-julia-1.144.2\scripts\debugger\run_debugger.jl:12
 [10] include(mod::Module, _path::String)
    @ Base .\Base.jl:557
 [11] exec_options(opts::Base.JLOptions)
    @ Base .\client.jl:323
 [12] _start()
    @ Base .\client.jl:531
in expression starting at c:\Users\tim\OneDrive\Documents\Julia\ZipArchives\TestMmap.jl:24

EDIT: If I take out all reference to ZipReader and simply mmap the file and then close it again, the file is successfully removed. This suggests it is, in fact, the interaction between ZipReader and mmap that is the problem

6

That version also runs without error for me. This seems like a windows-specific problem and is likely worth reporting to ZipArchives.

7

Issue opened here.

8

ZipReader acts like view here because there is no special mmap logic in ZipArchives.jl.

For example:

using Mmap

fname = tempname()
write(fname, "bar")

function reading()
    io = open(fname)
    mm = Mmap.mmap(io)
    q = view(mm, 1:2)
    close(io)
    GC.gc()
end

reading()

isfile(fname) && rm(fname)

Errors with:

ERROR: LoadError: IOError: unlink("C:\\Users\\nzimm\\AppData\\Local\\Temp\\jl_g1GveD8R03"): permission denied (EACCES)
Stacktrace:
 [1] uv_error
   @ .\libuv.jl:106 [inlined]
 [2] unlink(p::String)
 [1] uv_error
   @ .\libuv.jl:106 [inlined]
 [2] unlink(p::String)
   @ .\libuv.jl:106 [inlined]
 [2] unlink(p::String)
 [2] unlink(p::String)
   @ Base.Filesystem .\file.jl:1105
 [3] rm(path::String; force::Bool, recursive::Bool)
   @ Base.Filesystem .\file.jl:283
 [4] rm(path::String)
   @ Base.Filesystem .\file.jl:273
 [5] top-level scope
   @ C:\Users\nzimm\github\ZipArchives.jl\mmap.jl:16
in expression starting at C:\Users\nzimm\github\ZipArchives.jl\mmap.jl:16
9

You can get it to work if you move the GC.gc().

using Mmap

fname = tempname()
write(fname, "bar")

function reading()
    io=open(fname)
    mm=Mmap.mmap(io)
    q = view(mm, 1:2)
    close(io)
end

reading()
GC.gc()
GC.gc()
GC.gc()
GC.gc()

isfile(fname) && rm(fname)
10

Mmap can be very tricky, especially when the file is not read only.
ZipReader currently works with any AbstractArray but it could be updated to also support a nicer way to read from files: possibly using DiskArrays.jl