# Including CSV Files in a Package I'm Developing..?

**URL:** https://discourse.julialang.org/t/including-csv-files-in-a-package-im-developing/31532
**Category:** General Usage
**Tags:** package
**Created:** [November 26, 2019, 2:37pm UTC](https://discourse.julialang.org/t/including-csv-files-in-a-package-im-developing/31532 "2019-11-26T14:37:06Z")
**Posts on this page:** 5
**Page:** 1

<div class="post-metadata">

### Author: ![mthelm85](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mthelm85/32/224164_2.png) [@mthelm85](https://discourse.julialang.org/u/mthelm85)
#### Post date: [November 26, 2019, 2:37pm UTC](https://discourse.julialang.org/t/including-csv-files-in-a-package-im-developing/31532/1 "2019-11-26T14:37:07Z")

</div>

I’m attempting to develop my first package and I have a question about how to incorporate some necessary data into the package. Right now, I have four .csv files that include some data/lookup values that will be necessary for package users. Is it okay to do it this way or is there a better approach? For example, would it be better to simply convert them to Dicts and assign them to variables that are defined in a .jl file?

The current structure looks something like this:

```julia
src /
  data_dicts/
      csv1.csv
      csv2.csv
      csv3.csv
      csv4.csv
  MyPkg.jl

```

In this contrived example, MyPkg.jl reads from the .csv files. The files are small (6kb each) and wouldn’t be too much of a pain to convert to dicts, but it’s more convenient for me to leave them in .csv format.

---

<div class="post-metadata">

### Author: ![ExpandingMan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/expandingman/32/866_2.png) [@ExpandingMan](https://discourse.julialang.org/u/ExpandingMan)
#### Post date: [November 26, 2019, 2:45pm UTC](https://discourse.julialang.org/t/including-csv-files-in-a-package-im-developing/31532/2 "2019-11-26T14:45:23Z")

</div>

It depends on what you are doing, but it seems quite unlikely that using CSV is the best approach as reading them is rather slow (even though CSV.jl is excellent). You may want to consider a different serialization format such as [HDF5](https://github.com/JuliaIO/HDF5.jl).

If it’s a small enough amount of data, writing a script that converts the CSV’s to Julia source code can be a good approach. It has the significant advantage that presumably then you would not have to decide at what point in your program you’d have to read in the data and possibly memoize it.

I personally would put such data in a separate `data` directory rather than in `src`, since it isn’t source code unless you convert it.

If you don’t care any of that, of course there’s nothing stopping you from just using CSV’s if that’s what you want to do.

---

<div class="post-metadata">

### Author: ![mthelm85](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mthelm85/32/224164_2.png) [@mthelm85](https://discourse.julialang.org/u/mthelm85)
#### Post date: [November 26, 2019, 2:59pm UTC](https://discourse.julialang.org/t/including-csv-files-in-a-package-im-developing/31532/3 "2019-11-26T14:59:56Z")

</div>

Thanks for the info. I’m going to go the HDF5 (or some other format) route (stored in a separate data folder) for now because the amount of data is likely to grow so I like the idea of storing it in a more efficient format.

---

<div class="post-metadata">

### Author: ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)
#### Post date: [November 26, 2019, 3:12pm UTC](https://discourse.julialang.org/t/including-csv-files-in-a-package-im-developing/31532/4 "2019-11-26T15:12:08Z")

</div>

I think that saving small example datasets a directory of your choice (eg `data/`) within a package is fine. And CSV is a great choice, since for small datasets size and other considerations (like reading time, ability to mmap, etc) not very significant compared to the advantages (simplicity, transparency, lightweight dependencies).

For certain kinds of data,

[https://docs.julialang.org/en/v1/stdlib/DelimitedFiles/](https://docs.julialang.org/en/v1/stdlib/DelimitedFiles/)

may be enough, too.

You can then define a [path lookup function](https://discourse.julialang.org/t/correct-way-of-passing-paths-in-the-repl-for-a-project-containing-sub-folders/30380/2) in your package.

---

<div class="post-metadata">

### Author: ![mthelm85](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mthelm85/32/224164_2.png) [@mthelm85](https://discourse.julialang.org/u/mthelm85)
#### Post date: [November 26, 2019, 4:03pm UTC](https://discourse.julialang.org/t/including-csv-files-in-a-package-im-developing/31532/5 "2019-11-26T16:03:50Z")

</div>

Thanks for the info @Tamas_Papp!
