# Newbie using CSV with categorical=true

**URL:** <https://discourse.julialang.org/t/newbie-using-csv-with-categorical-true/16259>\
**Category:** Data\
**Created:** [October 12, 2018, 10:42pm UTC](https://discourse.julialang.org/t/newbie-using-csv-with-categorical-true/16259 "2018-10-12T22:42:50Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![ilanggear](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ilanggear/32/862_2.png) [@ilanggear](https://discourse.julialang.org/u/ilanggear)\
**Post date:** [October 12, 2018, 10:42pm UTC](https://discourse.julialang.org/t/newbie-using-csv-with-categorical-true/16259/1 "2018-10-12T22:42:50Z")

</div>

How do I use the categorical=true feature of CSV.validate(…)?  
I’ve tried a bunch of variations, and keep getting the error “ERROR: TypeError: in setfield!, expected Union{Missing, CategoricalString{UInt32}}, got String”.

I have two test files called “data/tmp1.csv” and “data/tmp2.csv”. The contents tmp1 are the two lines:  
0001000  
0001000

The contents tmp2 are the two lines:  
“0001000”  
“0001000”

I’ve defined the following variables for use in the CSV.validate(…) call.  
colNames=[:senID]  
colTypes=Dict(:senID=\>String)  
colTypes2=Dict(:senID=\>CategoricalString{UInt32})

I’ve tried all the following variations, plus others, and still get the error above. Can someone tell how to do this correctly?

CSV.validate(“data/tmp1.csv”,header=colNames, types=colTypes, categorical=true, allowmissing=:none, strict=true)  
CSV.validate(“data/tmp2.csv”,header=colNames, types=colTypes, categorical=true, allowmissing=:none, strict=true)  
CSV.validate(“data/tmp1.csv”,header=colNames, types=colTypes2, categorical=true, allowmissing=:none, strict=true)  
CSV.validate(“data/tmp2.csv”,header=colNames, types=colTypes2, categorical=true, allowmissing=:none, strict=true)  
CSV.validate(“data/tmp1.csv”,header=colNames, types=colTypes, categorical=true, allowmissing=:none, strict=true)  
CSV.validate(“data/tmp1.csv”,header=colNames, types=colTypes, typemap=tmpmap, categorical=true, allowmissing=:none, strict=true)

Thanks for your attention and help!

---

<div class="post-metadata">

**Author:** ![JeffreySarnoff](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jeffreysarnoff/32/1980_2.png) [@JeffreySarnoff](https://discourse.julialang.org/u/JeffreySarnoff)\
**Post date:** [October 12, 2018, 10:52pm UTC](https://discourse.julialang.org/t/newbie-using-csv-with-categorical-true/16259/2 "2018-10-12T22:52:31Z")

</div>

> [@ilanggear](#):
>
> CSV.validate

Please see the bottom half of this thread:

> <https://github.com/JuliaData/CSV.jl/issues/195>
>
> I am trying to read a file, and \`CSV.read\` keeps failing. I then have tried to u…se \`CSV.validate\` to understand why and get the exact same error, which I cannot interpret. I am hoping for help.
> 
> I am particularly perplexed because I created this file with \`CSV.write\`.
> \`\`\`julia
> julia\> CSV.validate("logs/net\_transition\_discount\_vary\_reference\_solver\_training.csv")
> ERROR: MethodError: no method matching \*(::Base.TwicePrecision{Float64}, ::Base.TwicePrecision{Float64})
> Closest candidates are:
> \*(::Any, ::Any, ::Any, ::Any...) at operators.jl:424
> \*(::Base.TwicePrecision, ::Integer) at twiceprecision.jl:470
> \*(::Base.TwicePrecision{R}, ::S\<:Number) where {S\<:Number, R} at twiceprecision.jl:488
> ...
> Stacktrace:
> \[1\] scale(::Int64, ::Int64, ::Int64, ::Int64, ::Int64) at /home/brett/.julia/v0.6/CSV/src/float.jl:71
> \[2\] parsefield(::Base.AbstractIOBuffer{Array{UInt8,1}}, ::Type{Float64}, ::CSV.Options{Void}, ::Int64, ::Int64, ::Base.RefValue{CSV.ParsingState}, ::CSV.##6#7) at /home/brett/.julia/v0.6/CSV/src/float.jl:208
> \[3\] parsefield(::Base.AbstractIOBuffer{Array{UInt8,1}}, ::Type{Union{Float64, Missings.Missing}}, ::CSV.Options{Void}, ::Int64, ::Int64, ::Base.RefValue{CSV.ParsingState}) at /home/brett/.julia/v0.6/CSV/src/parsefields.jl:130
> \[4\] validate(::CSV.Source{Base.AbstractIOBuffer{Array{UInt8,1}},Void}) at /home/brett/.julia/v0.6/CSV/src/validate.jl:32
> \[5\] #validate#56(::Bool, ::Dict{Int64,Function}, ::Array{Any,1}, ::Function, ::String, ::Type{T} where T) at /home/brett/.julia/v0.6/CSV/src/validate.jl:48
> \[6\] validate(::String) at /home/brett/.julia/v0.6/CSV/src/validate.jl:48
> \`\`\`

Ask for more help if that does not get your stuff working.

---

<div class="post-metadata">

**Author:** ![ilanggear](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ilanggear/32/862_2.png) [@ilanggear](https://discourse.julialang.org/u/ilanggear)\
**Post date:** [October 12, 2018, 11:58pm UTC](https://discourse.julialang.org/t/newbie-using-csv-with-categorical-true/16259/3 "2018-10-12T23:58:51Z")

</div>

I tried “add CSV#master” from pkg mode as suggested near the end of that thread, and that step succeeded, but I still get the same error.

---

<div class="post-metadata">

**Author:** ![JeffreySarnoff](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jeffreysarnoff/32/1980_2.png) [@JeffreySarnoff](https://discourse.julialang.org/u/JeffreySarnoff)\
**Post date:** [October 13, 2018, 12:42am UTC](https://discourse.julialang.org/t/newbie-using-csv-with-categorical-true/16259/4 "2018-10-13T00:42:03Z")

</div>

What version of Julia are you using, and is it on Linux OSX or Win?

```julia
versioninfo()

```

gives that and more – would you post what it reports?

---

<div class="post-metadata">

**Author:** ![ilanggear](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ilanggear/32/862_2.png) [@ilanggear](https://discourse.julialang.org/u/ilanggear)\
**Post date:** [October 13, 2018, 12:59am UTC](https://discourse.julialang.org/t/newbie-using-csv-with-categorical-true/16259/5 "2018-10-13T00:59:39Z")

</div>

I’m on a Macbook Pro, OSX 10.13.6 (17G65).

Julia Version 1.0.1

Commit 0d713926f8 (2018-09-29 19:05 UTC)

Platform Info:

OS: macOS (x86\_64-apple-darwin14.5.0)

CPU: Intel(R) Core™ i5-5257U CPU @ 2.70GHz

WORD\_SIZE: 64

LIBM: libopenlibm

LLVM: libLLVM-6.0.0 (ORCJIT, broadwell)

Environment:

JULIA\_EDITOR = atom -a

JULIA\_NUM\_THREADS = 2

---

<div class="post-metadata">

**Author:** ![y4lu](https://avatars.discourse-cdn.com/v4/letter/y/47e85d/32.png) [@y4lu](https://discourse.julialang.org/u/y4lu)\
**Post date:** [October 13, 2018, 2:00am UTC](https://discourse.julialang.org/t/newbie-using-csv-with-categorical-true/16259/6 "2018-10-13T02:00:41Z")

</div>

It might need something like `colTypes=Dict(:senID=>Union{String, Missing})`, but you could also try giving the file a header line and using less params, ie `CSV.validate(“data/tmp2.csv”, categorical=true)`

If all else fails, you might be able to call `CategoricalStrings` on a readdlm output

```julia
using DelimitedFiles
testfile = IOBuffer("0001000\n0001000");
readdlm(testfile, ',', String, header=false)
> "0001000"
  "0001000"
```

---

<div class="post-metadata">

**Author:** ![nalimilan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nalimilan/32/147_2.png) [@nalimilan](https://discourse.julialang.org/u/nalimilan)\
**Post date:** [October 13, 2018, 9:09am UTC](https://discourse.julialang.org/t/newbie-using-csv-with-categorical-true/16259/7 "2018-10-13T09:09:44Z")

</div>

That’s just a bug, I’ve filed [an issue](https://github.com/JuliaData/CSV.jl/issues/323).

As a workaround, note that `categorical=true` shouldn’t make any difference for validation, so you can just drop that argument. It’s mostly useful for `CSV.read`/`CSV.File`, and there it works AFAICT.

---

<div class="post-metadata">

**Author:** ![ilanggear](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ilanggear/32/862_2.png) [@ilanggear](https://discourse.julialang.org/u/ilanggear)\
**Post date:** [October 13, 2018, 3:31pm UTC](https://discourse.julialang.org/t/newbie-using-csv-with-categorical-true/16259/8 "2018-10-13T15:31:51Z")

</div>

Thanks!