# \[ANN\] CodecInflate64.jl - Testing help needed from Windows ZIP file users

**URL:** https://discourse.julialang.org/t/ann-codecinflate64-jl-testing-help-needed-from-windows-zip-file-users/117680
**Category:** Package Announcements
**Tags:** package, announcement, zip
**Created:** [July 31, 2024, 2:40pm UTC](https://discourse.julialang.org/t/ann-codecinflate64-jl-testing-help-needed-from-windows-zip-file-users/117680 "2024-07-31T14:40:06Z")
**Posts on this page:** 5
**Page:** 1

<div class="post-metadata">

### Author: ![nhz2](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nhz2/32/44428_2.png) [@nhz2](https://discourse.julialang.org/u/nhz2)
#### Post date: [July 31, 2024, 2:40pm UTC](https://discourse.julialang.org/t/ann-codecinflate64-jl-testing-help-needed-from-windows-zip-file-users/117680/1 "2024-07-31T14:40:06Z")

</div>

[CodecInflate64.jl](https://github.com/nhz2/CodecInflate64.jl) is a work in progress Julia implementation of deflate64 decompression.

Deflate64 is an incompatible variant of deflate that Windows File Explorer sometimes uses when making large ZIP files.

If you have any large ZIP files created on Windows, please help me test this library by running the following script and reporting any errors here or on GitHub.

```sh
julia testzip.jl myzipfile.zip

```

```julia
# This script tests if CodecInflate64 can be used to correctly read a zip file.
# Run it with for example: `julia testzip.jl myzipfile.zip`
import Pkg
Pkg.activate(;temp=true)
Pkg.add([
    "CodecInflate64",
    "ZipArchives",
    "CRC32",
    "InputBuffers"
])

using ZipArchives, CRC32, InputBuffers, CodecInflate64, Mmap

function checkcrc32_zipfile(zipfile::String; bufsize=2^14)
    data = mmap(open(zipfile; read=true))
    r = ZipReader(data)
    for i in 1:zip_nentries(r)
        method = ZipArchives.zip_compression_method(r, i)
        a = ZipArchives.zip_entry_data_offset(r,i)
        s = zip_compressed_size(r,i)
        c = data[begin+a:begin+a+s-1]
        u = if method == 9
            Deflate64DecompressorStream(InputBuffer(c); bufsize)
        elseif method == 8
            DeflateDecompressorStream(InputBuffer(c); bufsize)
        elseif method == 0
            InputBuffer(c)
        else
            error("unknown method in $(repr(zipfile)) entry: $(i) name: $(repr(zip_name(r,i)))")
        end
        if crc32(u) != zip_stored_crc32(r, i)
            error("crc32 wrong for $(repr(zipfile)) entry: $(i) name: $(repr(zip_name(r,i)))")
        end
    end
    @info "$(zip_nentries(r)) entries in $(repr(zipfile)) successfully checked"
end

checkcrc32_zipfile(ARGS[1])

```

Any ideas for more automated compatibility testing would also be appreciated, but Microsoft seems to try and prevent this [Why is Windows Compressed Folders (Zip folders) support stuck at the turn of the century? - The Old New Thing](https://devblogs.microsoft.com/oldnewthing/20180515-00/?p=98755)

[testzip.jl](https://discourse.julialang.org/uploads/short-url/k2zyyCyUDyIRVO0L1Ei7JdmupsT.jl) (1.3 KB)

---

<div class="post-metadata">

### Author: ![nhz2](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nhz2/32/44428_2.png) [@nhz2](https://discourse.julialang.org/u/nhz2)
#### Post date: [July 31, 2024, 3:37pm UTC](https://discourse.julialang.org/t/ann-codecinflate64-jl-testing-help-needed-from-windows-zip-file-users/117680/2 "2024-07-31T15:37:06Z")

</div>

@TimG Does the `testzip.jl` script work on the ZIP files you were having issues with?

---

<div class="post-metadata">

### Author: ![TimG](https://avatars.discourse-cdn.com/v4/letter/t/82dd89/32.png) [@TimG](https://discourse.julialang.org/u/TimG)
#### Post date: [August 1, 2024, 8:11am UTC](https://discourse.julialang.org/t/ann-codecinflate64-jl-testing-help-needed-from-windows-zip-file-users/117680/3 "2024-08-01T08:11:00Z")

</div>

@nhz2 Thank you for taking this initiative.

I’ve tried running `testzip.jl` on one of my zip files and got this result:

```julia
ERROR: LoadError: SystemError: opening file "GCS-00000140": No such file or directory
Stacktrace:
 [1] systemerror(p::String, errno::Int32; extrainfo::Nothing)
   @ Base .\error.jl:176
 [2] systemerror
   @ .\error.jl:175 [inlined]
 [3] open(fname::String; lock::Bool, read::Bool, write::Nothing, create::Nothing, truncate::Nothing, append::Nothing)
   @ Base .\iostream.jl:293
 [4] open
   @ .\iostream.jl:275 [inlined]
 [5] checkcrc32_zipfile(zipfile::String; bufsize::Int64)
   @ Main C:\Users\TGebbels\...\Documents\Julia Experimenting\testzip.jl:16
 [6] checkcrc32_zipfile(zipfile::String)
   @ Main C:\Users\TGebbels\...\Documents\Julia Experimenting\testzip.jl:15
 [7] top-level scope
   @ C:\Users\TGebbels\...\Documents\Julia Experimenting\testzip.jl:39
in expression starting at C:\Users\TGebbels\...\Documents\Julia Experimenting\testzip.jl:39

```

The files in my zip file are typically like this:

```julia
GCS-00000139 - YorkshireSculpturePark.xlsx
GCS-00000140 - YoungPeoplesSportPanel.xlsx
GCS-00000141 - Image1.jpg
GCS-00000141 - YoungpeoplegettheirskatesonStreetSkateTo.xlsx

```

and I wonder on the face of it if you do not allow for spaces in filenames?

---

<div class="post-metadata">

### Author: ![nhz2](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nhz2/32/44428_2.png) [@nhz2](https://discourse.julialang.org/u/nhz2)
#### Post date: [August 1, 2024, 11:39am UTC](https://discourse.julialang.org/t/ann-codecinflate64-jl-testing-help-needed-from-windows-zip-file-users/117680/4 "2024-08-01T11:39:32Z")

</div>

Spaces in filenames are fine, just put them in quotes when running the script:  
`julia testzip.jl "my zip file with spaces.zip"`

Alternatively, you can run the `checkcrc32_zipfile` function directly in the REPL to avoid CLI issues:

```julia
using ZipArchives, CRC32, InputBuffers, CodecInflate64, Mmap

function checkcrc32_zipfile(zipfile::String; bufsize=2^14)
    data = mmap(open(zipfile; read=true))
    r = ZipReader(data)
    for i in 1:zip_nentries(r)
        method = ZipArchives.zip_compression_method(r, i)
        a = ZipArchives.zip_entry_data_offset(r,i)
        s = zip_compressed_size(r,i)
        c = data[begin+a:begin+a+s-1]
        u = if method == 9
            Deflate64DecompressorStream(InputBuffer(c); bufsize)
        elseif method == 8
            DeflateDecompressorStream(InputBuffer(c); bufsize)
        elseif method == 0
            InputBuffer(c)
        else
            error("unknown method: $(method) in $(repr(zipfile)) entry: $(i) name: $(repr(zip_name(r,i)))")
        end
        if crc32(u) != zip_stored_crc32(r, i)
            error("crc32 wrong for $(repr(zipfile)) entry: $(i) name: $(repr(zip_name(r,i)))")
        end
    end
    @info "$(zip_nentries(r)) entries in $(repr(zipfile)) successfully checked"
end

checkcrc32_zipfile("my zip file with spaces")

```

---

<div class="post-metadata">

### Author: ![TimG](https://avatars.discourse-cdn.com/v4/letter/t/82dd89/32.png) [@TimG](https://discourse.julialang.org/u/TimG)
#### Post date: [August 1, 2024, 11:46am UTC](https://discourse.julialang.org/t/ann-codecinflate64-jl-testing-help-needed-from-windows-zip-file-users/117680/5 "2024-08-01T11:46:15Z")

</div>

Sorry for not understanding.

Tried again:

```julia
[ Info: 243 entries in "GCS-00000140 - YoungPeoplesSportPanel.zip" successfully checked

```
