# Whitespace as missingstring in CSV.File()

**URL:** <https://discourse.julialang.org/t/whitespace-as-missingstring-in-csv-file/47245>\
**Category:** Data\
**Tags:** csv\
**Created:** [September 25, 2020, 7:27am UTC](https://discourse.julialang.org/t/whitespace-as-missingstring-in-csv-file/47245 "2020-09-25T07:27:08Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![Jakob](https://avatars.discourse-cdn.com/v4/letter/j/71c47a/32.png) [@Jakob](https://discourse.julialang.org/u/Jakob)\
**Post date:** [September 25, 2020, 7:27am UTC](https://discourse.julialang.org/t/whitespace-as-missingstring-in-csv-file/47245/1 "2020-09-25T07:27:08Z")

</div>

I have a CSV file which has all of `""` (empty string), `" "` (1x whitespace), and " " (2x whitespace) as missing values (horrible, I know). I tried tackling it with `CSV.File(file, missingstrings = ["", " ", " "])` but apparently whitespace characters are not allowed as I get the error _sentinel value isn’t allowed to start with wh1 or wh2 characters_.

Is there a workaround for that or do I just go over the file afterwards and replace the additional missings?

---

<div class="post-metadata">

**Author:** ![pdeffebach](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pdeffebach/32/10320_2.png) [@pdeffebach](https://discourse.julialang.org/u/pdeffebach)\
**Post date:** [September 25, 2020, 12:54pm UTC](https://discourse.julialang.org/t/whitespace-as-missingstring-in-csv-file/47245/2 "2020-09-25T12:54:58Z")

</div>

cc @quinnj this seems like a bug in CSV

---

<div class="post-metadata">

**Author:** ![quinnj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/quinnj/32/11_2.png) [@quinnj](https://discourse.julialang.org/u/quinnj)\
**Post date:** [September 25, 2020, 1:15pm UTC](https://discourse.julialang.org/t/whitespace-as-missingstring-in-csv-file/47245/3 "2020-09-25T13:15:44Z")

</div>

Ah, the issue here is the 2x space missingstring; by default, CSV.jl uses a single space to remove leading/trailing whitespace around numbers when parsing, and if a single space is used as `missingstring`, then it doesn’t strip the leading/trailing whitespace. But in the case of 2x space, it’s not switching to _not_ stripping whitespace, but also realizes there will be a conflict (i.e. it will try to strip whitespace, but hte sentinel value is also whitespace). Let me think of a better way we can handle this in CSV.jl. Maybe we just need to allow manually setting the “whitespace” characters to strip, because then you coudl set them to `0x00` and not get hte conflict.

---

<div class="post-metadata">

**Author:** ![Jakob](https://avatars.discourse-cdn.com/v4/letter/j/71c47a/32.png) [@Jakob](https://discourse.julialang.org/u/Jakob)\
**Post date:** [September 25, 2020, 1:22pm UTC](https://discourse.julialang.org/t/whitespace-as-missingstring-in-csv-file/47245/4 "2020-09-25T13:22:43Z")

</div>

Great, thanks! I also opened an [issue](https://github.com/JuliaData/CSV.jl/issues/743).

---

<div class="post-metadata">

**Author:** ![HenriDeh](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/henrideh/32/8316_2.png) [@HenriDeh](https://discourse.julialang.org/u/HenriDeh)\
**Post date:** [September 25, 2020, 1:51pm UTC](https://discourse.julialang.org/t/whitespace-as-missingstring-in-csv-file/47245/5 "2020-09-25T13:51:16Z")

</div>

If you’re allowed to modify the csv, you can find and replace with Ctrl + F. Find `," ",` and replace by `,"",`. As a workaround

---

<div class="post-metadata">

**Author:** ![Jakob](https://avatars.discourse-cdn.com/v4/letter/j/71c47a/32.png) [@Jakob](https://discourse.julialang.org/u/Jakob)\
**Post date:** [September 25, 2020, 1:57pm UTC](https://discourse.julialang.org/t/whitespace-as-missingstring-in-csv-file/47245/6 "2020-09-25T13:57:23Z")

</div>

Sure, but I’d rather not edit files manually. Also they’re 8GB each so a manual find and replace is probably not the best idea 🙂

As a workaround I can for now do a `replace!(col, " " => missing)` after reading the file, but it would probably be more efficient / elegant if this could be handled in the initial parsing step.