# Data Import Types and Compression

**URL:** <https://discourse.julialang.org/t/data-import-types-and-compression/7907>\
**Category:** New to Julia\
**Tags:** question, data-compression\
**Created:** [December 22, 2017, 1:14am UTC](https://discourse.julialang.org/t/data-import-types-and-compression/7907 "2017-12-22T01:14:33Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![ldsands](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ldsands/32/2800_2.png) [@ldsands](https://discourse.julialang.org/u/ldsands)\
**Post date:** [December 22, 2017, 1:14am UTC](https://discourse.julialang.org/t/data-import-types-and-compression/7907/1 "2017-12-22T01:14:33Z")

</div>

Hello all,

Sorry if this is a bad questions but I use data sets from ICPSR (a social science repository) and to use an example a dataset that I recently used is about 200 mg in uncompressed csv however I’m able to get these down to about 30 mg in rds format in r and Stata dta format has a similar size. When I try to save it in jld using this code `save("dta.jld", "dta", dta, compress=true)` it is still around 200 mg. I suspect that putting a lot of the variables which Julia imports as floats into int since most variables only contain a dozen or less unique values or something else might help but I have no idea how to do that on a larger scale (this example dataset has a little over 10,000 variables)

1. What methods should I look into for this problem?
2. What is the most user friendly way of doing this?
3. What is the current state of data file formats in Julia, and future plans?

Although I’m familiar with R and Stata I’m no computer programmer but I want to learn Julia for the speed benefits while data wrangling. Getting more managable files sizes is one of those quality of life things I want to figure out.

Thank you!

---

<div class="post-metadata">

**Author:** ![Yifan\_Liu](https://avatars.discourse-cdn.com/v4/letter/y/4da419/32.png) [@Yifan\_Liu](https://discourse.julialang.org/u/Yifan_Liu)\
**Post date:** [December 22, 2017, 4:36am UTC](https://discourse.julialang.org/t/data-import-types-and-compression/7907/2 "2017-12-22T04:36:40Z")

</div>

Try the fst format for R and Julia

> **[Lightning Fast Serialization of Data Frames](http://www.fstpackage.org/)**
>
> Multithreaded serialization of compressed data frames using the
> fst format. The fst format allows for random access of stored data and
> compression with the LZ4 and ZSTD compressors created by Yann Collet. The ZSTD
> compression library is...

[https://github.com/xiaodaigh/fstformat.jl](https://github.com/xiaodaigh/fstformat.jl)