# Filter dataframe with regular expression

**URL:** <https://discourse.julialang.org/t/filter-dataframe-with-regular-expression/76888>\
**Category:** New to Julia\
**Tags:** regex, dataframes\
**Created:** [February 22, 2022, 8:13am UTC](https://discourse.julialang.org/t/filter-dataframe-with-regular-expression/76888 "2022-02-22T08:13:33Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![Mastomaki](https://avatars.discourse-cdn.com/v4/letter/m/779978/32.png) [@Mastomaki](https://discourse.julialang.org/u/Mastomaki)\
**Post date:** [February 22, 2022, 8:13am UTC](https://discourse.julialang.org/t/filter-dataframe-with-regular-expression/76888/1 "2022-02-22T08:13:33Z")

</div>

Hello,

I am trying to filter a dataframe based on values in certain column. This works fine when I use `subset()` with ByRow(in([“mytext”])). But I get problems with regular expressions:

```julia
df = subset(df, :mycol => x -> ByRow(occursin(r"mytext", x) ))

ERROR: MethodError: no method matching occursin(::Regex, ::Array{String31,1})
Closest candidates are:
  occursin(::Regex, ::SubString; offset) at regex.jl:176
  occursin(::Regex, ::AbstractString; offset) at regex.jl:171

```

---

<div class="post-metadata">

**Author:** ![bkamins](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bkamins/32/208538_2.png) [@bkamins](https://discourse.julialang.org/u/bkamins)\
**Post date:** [February 22, 2022, 8:34am UTC](https://discourse.julialang.org/t/filter-dataframe-with-regular-expression/76888/2 "2022-02-22T08:34:48Z")

</div>

> [@Mastomaki](#):
>
> ```julia
> df = subset(df, :mycol => x -> ByRow(occursin(r"mytext", x) ))
> 
> ```

```julia
subset(df, :mycol => ByRow(x -> occursin(r"mytext", x) ))

```

should work, or using DataFramesMeta.jl:

```julia
@rsubset(df, occursin(r"mytext", :mycol))

```

---

<div class="post-metadata">

**Author:** ![George\_Githinji](https://avatars.discourse-cdn.com/v4/letter/g/e68b1a/32.png) [@George\_Githinji](https://discourse.julialang.org/u/George_Githinji)\
**Post date:** [March 1, 2022, 9:38am UTC](https://discourse.julialang.org/t/filter-dataframe-with-regular-expression/76888/3 "2022-03-01T09:38:22Z")

</div>

> [@bkamins](#):
>
> `subset(df, :mycol => ByRow(x -> occursin(r"mytext", x) ))`

Tried something similar through with a different dataframe and i get an error.

```julia
ERROR: LoadError: MethodError: no method matching occursin(::Regex, ::Missing)
Closest candidates are:
  occursin(::Regex, ::SubString; offset) at /Applications/Julia-1.7.app/Contents/Resources/julia/share/julia/base/regex.jl:269
  occursin(::Regex, ::AbstractString; offset) at /Applications/Julia-1.7.app/Contents/Resources/julia/share/julia/base/regex.jl:264
  occursin(::Any) at /Applications/Julia-1.7.app/Contents/Resources/julia/share/julia/base/strings/search.jl:636

```

Wondering what i am missing

---

<div class="post-metadata">

**Author:** ![nilshg](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nilshg/32/2283_2.png) [@nilshg](https://discourse.julialang.org/u/nilshg)\
**Post date:** [March 1, 2022, 9:53am UTC](https://discourse.julialang.org/t/filter-dataframe-with-regular-expression/76888/4 "2022-03-01T09:53:16Z")

</div>

You are (quite literally) _missing_ some data (sorry couldn’t resist).

```julia
julia> occursin(r"abc", "abcde")
true

julia> occursin(r"abc", missing)
ERROR: MethodError: no method matching occursin(::Regex, ::Missing)

```

---

<div class="post-metadata">

**Author:** ![George\_Githinji](https://avatars.discourse-cdn.com/v4/letter/g/e68b1a/32.png) [@George\_Githinji](https://discourse.julialang.org/u/George_Githinji)\
**Post date:** [March 1, 2022, 10:27am UTC](https://discourse.julialang.org/t/filter-dataframe-with-regular-expression/76888/5 "2022-03-01T10:27:46Z")

</div>

Thanks! I have a large dataset of \>2M records and I was of the idea that the target filter column was complete but yes it had a number of missing values! i have dropped the missing values and the subset works. PS: newish to Julia and DataFrames and migrating from R 🙂

---

<div class="post-metadata">

**Author:** ![nilshg](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nilshg/32/2283_2.png) [@nilshg](https://discourse.julialang.org/u/nilshg)\
**Post date:** [March 1, 2022, 10:49am UTC](https://discourse.julialang.org/t/filter-dataframe-with-regular-expression/76888/6 "2022-03-01T10:49:56Z")

</div>

As a hint, most environments (REPL, VSCode, IJulia, Pluto…) will print the column type at the top of the table, underneath the column names. This will generally alert you to the presence of missing values either with a question mark or an explicit `Union{Float64, missing}` as column type:

```julia
julia> DataFrame(a = rand(5), b = rand([missing; 1:5], 5))
5×2 DataFrame
 Row │ a b       
     │ Float64 Int64?  
─────┼────────────────────
   1 │ 0.403111 3
   2 │ 0.726926 missing 
   3 │ 0.891562 5
   4 │ 0.0909533 5
   5 │ 0.820027 4

```

Notice the question mark in `Int64?` for the second column. This doesn’t necessarily mean that there are missing values, as the type won’t change after you’ve filtered missings out, but it can at least prompt you to check!

---

<div class="post-metadata">

**Author:** ![bkamins](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bkamins/32/208538_2.png) [@bkamins](https://discourse.julialang.org/u/bkamins)\
**Post date:** [March 1, 2022, 11:06am UTC](https://discourse.julialang.org/t/filter-dataframe-with-regular-expression/76888/7 "2022-03-01T11:06:54Z")

</div>

> [@bkamins](#):
>
> subset(df, :mycol =\> ByRow(x → occursin(r"mytext", x) ))

Use:

```julia
subset(df, :mycol => ByRow(x -> passmissing(occursin)(r"mytext", x)), skipmissing=true)

```

to make the operation return `missing` when the input has `missing`. Then by `skipmissing=true` you will skip them.

---

<div class="post-metadata">

**Author:** ![pdeffebach](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pdeffebach/32/10320_2.png) [@pdeffebach](https://discourse.julialang.org/u/pdeffebach)\
**Post date:** [March 1, 2022, 3:26pm UTC](https://discourse.julialang.org/t/filter-dataframe-with-regular-expression/76888/8 "2022-03-01T15:26:19Z")

</div>

You can also use the `@passmissing` flag in a DataFramesMeta.jl transformation

```julia
@rsubset @passmissing occursin(r"mytext", :x)

```

---

<div class="post-metadata">

**Author:** ![zapiano](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/zapiano/32/48063_2.png) [@zapiano](https://discourse.julialang.org/u/zapiano)\
**Post date:** [February 20, 2025, 2:44am UTC](https://discourse.julialang.org/t/filter-dataframe-with-regular-expression/76888/9 "2025-02-20T02:44:05Z")

</div>

I’m not sure if I misunderstood the question, but I was trying to do a similar thing and I solved it by doing:

```julia
df[occursin.(r"mytext", df.col_name), :]

```

Not sure if the solution with `subset` performs better or if they are equivalent, but I tried the `subset` and it didn’t work for me…