# Partially decompressing Bzip2 files

**URL:** https://discourse.julialang.org/t/partially-decompressing-bzip2-files/96429
**Category:** General Usage
**Tags:** question, io, data-compression
**Created:** [March 22, 2023, 7:04am UTC](https://discourse.julialang.org/t/partially-decompressing-bzip2-files/96429 "2023-03-22T07:04:14Z")
**Posts on this page:** 3
**Page:** 1

<div class="post-metadata">

### Author: ![tristian](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tristian/32/6860_2.png) [@tristian](https://discourse.julialang.org/u/tristian)
#### Post date: [March 22, 2023, 7:04am UTC](https://discourse.julialang.org/t/partially-decompressing-bzip2-files/96429/1 "2023-03-22T07:04:14Z")

</div>

As the title of the question states, I have a few `*.csv.bzip2` compressed files that are as big as 13GB when decompressed. The files have a structured format and are essentially CSV files.

I’d like to read the headers of the files to generate a SQL schema so that I can import them into a RDBMS for more efficient querying and exploration.

I’ve looked at `CodecBzip2` package and it hangs whenever I attempt to decode a partial bytes vector.

Some sample code (omitting proper handling for brevity):  
Setup:

```julia
] add CodecBzip2, TranscodingStreams, CSV

```

Script:

```julia
using TranscodingStreams, CodecBzip2

f = open("/path/to/csv.bzip2", "r")
zipdata = read(f, 2048)

# The following line hangs
d = transcode(Bzip2Decompressor, zipdata)

println(String(d))

```

My understanding is that the algorithm’s compressed blocks should be independently decompressable; I’m most likely not grabbing “valid” data blocks for decompression by just reading the first `N` bytes of the file.

Are there any packages that would offer partial decompression? has anyone done something similar?

Many thanks in advance!

---

<div class="post-metadata">

### Author: ![fredrikekre](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/fredrikekre/32/1688_2.png) [@fredrikekre](https://discourse.julialang.org/u/fredrikekre)
#### Post date: [March 22, 2023, 7:48am UTC](https://discourse.julialang.org/t/partially-decompressing-bzip2-files/96429/2 "2023-03-22T07:48:09Z")

</div>

You can use the stream version of the compressor, since you only want to read some bytes.

```julia
shell> bzcat test.csv.bz2
a,b
1,"hello"

julia> using CodecBzip2

julia> x = open("test.csv.bz2") do compressed
           decompressed = Bzip2DecompressorStream(compressed)
           String(read(decompressed, 10))
       end
"a,b\n1,\"hel"

```

---

<div class="post-metadata">

### Author: ![tristian](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tristian/32/6860_2.png) [@tristian](https://discourse.julialang.org/u/tristian)
#### Post date: [March 23, 2023, 1:00am UTC](https://discourse.julialang.org/t/partially-decompressing-bzip2-files/96429/3 "2023-03-23T01:00:58Z")

</div>

Hi @fredrikekre! Thank you so much, this works great!
