Hi all,
I’d like to introduce my new packages: TranscodingStreams.jl and its friends.
TranscodingStreams.jl is a package to make it simpler and more consistent to read/write encoded data stream. For example, if you use Libz.jl (I am a maintainer of it) TranscodingStreams.jl and CodecZlib.jl will be a drop-in replacement of it. Moreover, CodecBzip2.jl, CodecXz.jl, and CodecZstd.jl support interfaces of TranscodingStremas.jl and hence you can use these all lossless compression algorithms without knowing the details.
Here is an example of reading a gzip-compressed file:
using CodecZlib
stream = GzipDecompressionStream(open("data.txt.gz"))
for line in eachline(stream)
# do something...
end
close(stream)
If you want to read a bzip2-, xz-, or zstd-compressrd file instead of gzip in the example above, you can replace GzipDecompressionStream with Bzip2DecompressionStream, XzDecompressionStream, or ZstdDecompressionStream, respectively. These are all aliases to TranscodingStream{C,S} where C<:Codec and S<:IO, which is a subtype of IO and supports a range of I/O operations. More detailed explanation and examples can be found at the docs of TranscodingStreams.jl.
These packages are still quite young and haven’t support Windows yet. I really appreciate pull requests to support Windows platforms if you have access to Windows. I hope to get feedbacks from users.
Thank you.