# CSV.jl : how to specify the columns types when the total columns number is not known?

**URL:** <https://discourse.julialang.org/t/csv-jl-how-to-specify-the-columns-types-when-the-total-columns-number-is-not-known/34753>\
**Category:** Data\
**Created:** [February 17, 2020, 10:42am UTC](https://discourse.julialang.org/t/csv-jl-how-to-specify-the-columns-types-when-the-total-columns-number-is-not-known/34753 "2020-02-17T10:42:02Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![Fred](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/fred/32/14175_2.png) [@Fred](https://discourse.julialang.org/u/Fred)\
**Post date:** [February 17, 2020, 10:42am UTC](https://discourse.julialang.org/t/csv-jl-how-to-specify-the-columns-types-when-the-total-columns-number-is-not-known/34753/1 "2020-02-17T10:42:02Z")

</div>

Hi,

I have to read large tables, I don’t know the total number of columns of the tables but I do know that the first one contains strings and all the others are Float64. I can easily specify the type of the first one like this  
`x = CSV.read("file.tsv"; delim ="\t", types=Dict(1=>String)) |> DataFrame!`  
But how I can set all other columns to Float64 to increase the parsing speed ?

Thanks in advance 😉

---

<div class="post-metadata">

**Author:** ![JeffreySarnoff](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jeffreysarnoff/32/1980_2.png) [@JeffreySarnoff](https://discourse.julialang.org/u/JeffreySarnoff)\
**Post date:** [February 17, 2020, 11:19am UTC](https://discourse.julialang.org/t/csv-jl-how-to-specify-the-columns-types-when-the-total-columns-number-is-not-known/34753/2 "2020-02-17T11:19:04Z")

</div>

`CSV.jl` will recognize those column types without help.  
try `x = CSV.read("file.tsv"; delim='\t', header=false)`  
or just `x = CSV.read("file.tsv"; header=false)` assuming no header line  
or just `x = CSV.read("file.tsv")` assuming a header line (column names)

---

<div class="post-metadata">

**Author:** ![Fred](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/fred/32/14175_2.png) [@Fred](https://discourse.julialang.org/u/Fred)\
**Post date:** [February 17, 2020, 12:35pm UTC](https://discourse.julialang.org/t/csv-jl-how-to-specify-the-columns-types-when-the-total-columns-number-is-not-known/34753/3 "2020-02-17T12:35:37Z")

</div>

@JeffreySarnoff thank you for your answer ! In fact I want to specify the columns type to increase the parsing speed 😉

---

<div class="post-metadata">

**Author:** ![JeffreySarnoff](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jeffreysarnoff/32/1980_2.png) [@JeffreySarnoff](https://discourse.julialang.org/u/JeffreySarnoff)\
**Post date:** [February 17, 2020, 2:21pm UTC](https://discourse.julialang.org/t/csv-jl-how-to-specify-the-columns-types-when-the-total-columns-number-is-not-known/34753/4 "2020-02-17T14:21:34Z")

</div>

If you know the most columns that you may have (or some number greater than that which is certain to be larger than the number of columns, `most`), you can set `types=Dict(1=>String, 2=>Float64, ... most=>Float64)` with no harm and it will work. You could read in the first row only `limit=1` and find out how many columns there are. There may well be other ways.

---

<div class="post-metadata">

**Author:** ![Fred](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/fred/32/14175_2.png) [@Fred](https://discourse.julialang.org/u/Fred)\
**Post date:** [February 17, 2020, 2:42pm UTC](https://discourse.julialang.org/t/csv-jl-how-to-specify-the-columns-types-when-the-total-columns-number-is-not-known/34753/5 "2020-02-17T14:42:27Z")

</div>

@JeffreySarnoff thank you ! The column number is highly variable unfortunately. If the possibility I search is not implemented in the package, I have indeed only the possibility to read the first line, but I think it could be slower than left the columns type blank.

---

<div class="post-metadata">

**Author:** ![JeffreySarnoff](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jeffreysarnoff/32/1980_2.png) [@JeffreySarnoff](https://discourse.julialang.org/u/JeffreySarnoff)\
**Post date:** [February 17, 2020, 3:01pm UTC](https://discourse.julialang.org/t/csv-jl-how-to-specify-the-columns-types-when-the-total-columns-number-is-not-known/34753/6 "2020-02-17T15:01:26Z")

</div>

I don’t think there is much you need to worry about with not explicitly giving the column types. That stuff gets done quite quickly.

---

<div class="post-metadata">

**Author:** ![quinnj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/quinnj/32/11_2.png) [@quinnj](https://discourse.julialang.org/u/quinnj)\
**Post date:** [February 17, 2020, 4:05pm UTC](https://discourse.julialang.org/t/csv-jl-how-to-specify-the-columns-types-when-the-total-columns-number-is-not-known/34753/7 "2020-02-17T16:05:40Z")

</div>

There is a keyword argument `type=Float64`, that allows specifying the type for all columns, but they have to be homogenous. One thing we should probably allow is doing something like `type=Float64, types=(:col1=String,)`, so you could specify the types of “all” columns, but override a single column with `types`. If you wouldn’t mind opening an issue about it, I’m planning on doing a bunch of CSV.jl work in the next week or two.

But as @JeffreySarnoff mentioned, this also shouldn’t make a drastic change in performance as the detection code is usually negligible.

---

<div class="post-metadata">

**Author:** ![JeffreySarnoff](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jeffreysarnoff/32/1980_2.png) [@JeffreySarnoff](https://discourse.julialang.org/u/JeffreySarnoff)\
**Post date:** [February 17, 2020, 7:21pm UTC](https://discourse.julialang.org/t/csv-jl-how-to-specify-the-columns-types-when-the-total-columns-number-is-not-known/34753/8 "2020-02-17T19:21:23Z")

</div>

issue raised [Allow a default col type and some cols to have specific types · Issue #575 · JuliaData/CSV.jl · GitHub](https://github.com/JuliaData/CSV.jl/issues/575)

---

<div class="post-metadata">

**Author:** ![Fred](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/fred/32/14175_2.png) [@Fred](https://discourse.julialang.org/u/Fred)\
**Post date:** [February 18, 2020, 6:27am UTC](https://discourse.julialang.org/t/csv-jl-how-to-specify-the-columns-types-when-the-total-columns-number-is-not-known/34753/9 "2020-02-18T06:27:12Z")

</div>

Thank you @quinnj, @JeffreySarnoff !