# DataFrame can't get filter to get rid of rows where a column contains a blank " "

**URL:** <https://discourse.julialang.org/t/dataframe-cant-get-filter-to-get-rid-of-rows-where-a-column-contains-a-blank/76726>\
**Category:** New to Julia\
**Tags:** dataframes\
**Created:** [February 18, 2022, 9:47pm UTC](https://discourse.julialang.org/t/dataframe-cant-get-filter-to-get-rid-of-rows-where-a-column-contains-a-blank/76726 "2022-02-18T21:47:51Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![anon69491625](https://avatars.discourse-cdn.com/v4/letter/a/aeb1de/32.png) [@anon69491625](https://discourse.julialang.org/u/anon69491625)\
**Post date:** [February 18, 2022, 9:47pm UTC](https://discourse.julialang.org/t/dataframe-cant-get-filter-to-get-rid-of-rows-where-a-column-contains-a-blank/76726/1 "2022-02-18T21:47:52Z")

</div>

Hi there

I have a dataframe ( df\_list\_of\_clles ) that has a column ( toffer) that contains missing values and blanks. A blank in this case is " " .

I can get rid of the missing values using the below but isnothing doesn’t seem to work. What am I doing wrong please?

```julia
filter(:toffer => x -> !(ismissing(x) || isnothing(x)) , df_list_of_clles )

```

---

<div class="post-metadata">

**Author:** ![pdeffebach](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pdeffebach/32/10320_2.png) [@pdeffebach](https://discourse.julialang.org/u/pdeffebach)\
**Post date:** [February 18, 2022, 9:59pm UTC](https://discourse.julialang.org/t/dataframe-cant-get-filter-to-get-rid-of-rows-where-a-column-contains-a-blank/76726/2 "2022-02-18T21:59:27Z")

</div>

Maybe you want to add `== " "` instead?

Or better yet, replace `" " ` with `missing` before you get to this step in the data cleaning.

Julia is not like Stata where empty strings are equivalent to `missing`.

---

<div class="post-metadata">

**Author:** ![anon69491625](https://avatars.discourse-cdn.com/v4/letter/a/aeb1de/32.png) [@anon69491625](https://discourse.julialang.org/u/anon69491625)\
**Post date:** [February 18, 2022, 10:10pm UTC](https://discourse.julialang.org/t/dataframe-cant-get-filter-to-get-rid-of-rows-where-a-column-contains-a-blank/76726/3 "2022-02-18T22:10:12Z")

</div>

excellent idea to replace " " with missing but what’s the best way to do that please? this is my first time with DataFrames.jl.

---

<div class="post-metadata">

**Author:** ![gustaphe](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gustaphe/32/18174_2.png) [@gustaphe](https://discourse.julialang.org/u/gustaphe)\
**Post date:** [February 18, 2022, 10:11pm UTC](https://discourse.julialang.org/t/dataframe-cant-get-filter-to-get-rid-of-rows-where-a-column-contains-a-blank/76726/4 "2022-02-18T22:11:09Z")

</div>

`isnothing(x)` tests if `x` is `nothing`. Since `" "` is not `nothing` (or even a `Nothing`), `isnothing(" ")` is `false`.

---

<div class="post-metadata">

**Author:** ![nilshg](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nilshg/32/2283_2.png) [@nilshg](https://discourse.julialang.org/u/nilshg)\
**Post date:** [February 18, 2022, 10:13pm UTC](https://discourse.julialang.org/t/dataframe-cant-get-filter-to-get-rid-of-rows-where-a-column-contains-a-blank/76726/5 "2022-02-18T22:13:07Z")

</div>

DataFrames is not different from base Julia, a column in a DataFrame is just a plain old Vector:

```julia
julia> x = ["a", missing, " "]
3-element Vector{Union{Missing, String}}:
 "a"
 missing
 " "

julia> replace(x, " " => missing)
3-element Vector{Union{Missing, String}}:
 "a"
 missing
 missing

```

would still be the same if `x` was a column in a DataFrame - it would just be `df.x`

---

<div class="post-metadata">

**Author:** ![anon69491625](https://avatars.discourse-cdn.com/v4/letter/a/aeb1de/32.png) [@anon69491625](https://discourse.julialang.org/u/anon69491625)\
**Post date:** [February 18, 2022, 10:26pm UTC](https://discourse.julialang.org/t/dataframe-cant-get-filter-to-get-rid-of-rows-where-a-column-contains-a-blank/76726/6 "2022-02-18T22:26:18Z")

</div>

> [@nilshg](#):
>
> `replace(x, " " => missing)`

Perfect! I almost had it but dropped the ball before the try-line 🙂 I tried this but it didn’t seem to work.

```julia
replace!(df_list_of_clles.toffer, " " => missing ) # specific column 

```

You solution ( again saving the day) worked.  
thanks

---

<div class="post-metadata">

**Author:** ![anon69491625](https://avatars.discourse-cdn.com/v4/letter/a/aeb1de/32.png) [@anon69491625](https://discourse.julialang.org/u/anon69491625)\
**Post date:** [February 18, 2022, 10:28pm UTC](https://discourse.julialang.org/t/dataframe-cant-get-filter-to-get-rid-of-rows-where-a-column-contains-a-blank/76726/7 "2022-02-18T22:28:40Z")

</div>

thank you for filling in another gap in my julia knowledge. I’ve added the

```julia
isnothing(" ")

```

to my list of things to try before bugging you good people. Right now I’m suffering from overload and NOT enjoying fighting with vscode.

---

<div class="post-metadata">

**Author:** ![nilshg](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nilshg/32/2283_2.png) [@nilshg](https://discourse.julialang.org/u/nilshg)\
**Post date:** [February 19, 2022, 8:37am UTC](https://discourse.julialang.org/t/dataframe-cant-get-filter-to-get-rid-of-rows-where-a-column-contains-a-blank/76726/8 "2022-02-19T08:37:04Z")

</div>

Glad it worked, but I think I should clarify so as not to mislead you: `replace!` is generally preferred as it will replace elements within a vector in place, while `replace` allocates a new vector:

```julia
julia> x = rand(["a", " ", missing], 100_000);

julia> @btime replace($x, " " => missing) setup=(x=$x);
  1.124 ms (2 allocations: 781.30 KiB)

julia> @btime replace!($x, " " => missing) setup=(x=$x);
  652.087 μs (0 allocations: 0 bytes)

```

Again as a DataFrame is just a collection of plain Vectors so:

```julia
julia> df = DataFrame(x = 1:3, toffer = ["a", missing, " "])
3×2 DataFrame
 Row │ x toffer  
     │ Int64 String? 
─────┼────────────────
   1 │ 1 a
   2 │ 2 missing 
   3 │ 3

julia> replace!(df.toffer, " " => missing); df
3×2 DataFrame
 Row │ x toffer  
     │ Int64 String? 
─────┼────────────────
   1 │ 1 a
   2 │ 2 missing 
   3 │ 3 missing 

```

If that didn’t work for you it’s likely that your `toffer` column wasn’t a `Union{Missing, T}` but just a `Vector{T}` that can’t hold `missing`s.

More broadly you should ask yourself how you end up with `" "` in your Data - if you’re reading from csv or some other delimited text format (or similarly formatted data from an `IOBuffer`) you can use `CSV.read(data, DataFrame; missingstring = " ")` to read the data in correctly in the first place.

---

<div class="post-metadata">

**Author:** ![anon69491625](https://avatars.discourse-cdn.com/v4/letter/a/aeb1de/32.png) [@anon69491625](https://discourse.julialang.org/u/anon69491625)\
**Post date:** [February 19, 2022, 1:31pm UTC](https://discourse.julialang.org/t/dataframe-cant-get-filter-to-get-rid-of-rows-where-a-column-contains-a-blank/76726/9 "2022-02-19T13:31:24Z")

</div>

good morning @nilshg

an excellent surprise gift of more tools in my toolbox. Thank you so much for going the extra mile, AGAIN!

```julia
CSV.read(data, DataFrame; missingstring = " ") 

```

Is something I missed in the CSV documentation so you have prompted me to go back and revisit that. I had to rush things. Thanks again for all your help on this.