# Dropping rows from DataFrame

**URL:** <https://discourse.julialang.org/t/dropping-rows-from-dataframe/49469>\
**Category:** New to Julia\
**Created:** [November 2, 2020, 5:18pm UTC](https://discourse.julialang.org/t/dropping-rows-from-dataframe/49469 "2020-11-02T17:18:09Z")\
**Posts on this page:** 13\
**Page:** 1

<div class="post-metadata">

**Author:** ![alunap](https://avatars.discourse-cdn.com/v4/letter/a/58956e/32.png) [@alunap](https://discourse.julialang.org/u/alunap)\
**Post date:** [November 2, 2020, 5:18pm UTC](https://discourse.julialang.org/t/dropping-rows-from-dataframe/49469/1 "2020-11-02T17:18:09Z")

</div>

I have a DataFrame read in from a json file, and the first step it to select just a few columns and then strip out any rows where one of the columns (a string) starts with ‘\<’. I have tried a couple of things, ultimately ending with

function filter\_captures(df)  
@linq df |\>  
select(:user, :term, :policy, :time) |\>  
where(:term .startswith(r"\[1\].\*))  
end

the where term I can’t work out. I see I can broadcast over :term column, but I want to broadcast where the first character is NOT ‘\<’ (or alternatively where it IS a ‘\*’ or a letter). I’m unclear how broadcast is supposed to work with a function like startswith. Any ideas?

thanks in advance

* * *

1. ^\<

---

<div class="post-metadata">

**Author:** ![mike\_k](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mike_k/32/211864_2.png) [@mike\_k](https://discourse.julialang.org/u/mike_k)\
**Post date:** [November 2, 2020, 5:42pm UTC](https://discourse.julialang.org/t/dropping-rows-from-dataframe/49469/2 "2020-11-02T17:42:33Z")

</div>

Hi, a while ago I posted a solved quite similar question [here](https://discourse.julialang.org/t/dataframes-how-to-conditionally-remove-rows-based-on-data-values/47538). Maybe this helps.

---

<div class="post-metadata">

**Author:** ![alunap](https://avatars.discourse-cdn.com/v4/letter/a/58956e/32.png) [@alunap](https://discourse.julialang.org/u/alunap)\
**Post date:** [November 2, 2020, 5:54pm UTC](https://discourse.julialang.org/t/dropping-rows-from-dataframe/49469/3 "2020-11-02T17:54:30Z")

</div>

Thanks for the reply Mike. I did look at that, but it doesn’t deal with the essential issue, which is how you broadcast a function that takes arguments (including the content of the item). I have filled in the startswith() function now, but startswith takes two arguments, I don’t understand what the first one should be, or whether you can broadcast such a function at all.

---

<div class="post-metadata">

**Author:** ![mike\_k](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mike_k/32/211864_2.png) [@mike\_k](https://discourse.julialang.org/u/mike_k)\
**Post date:** [November 2, 2020, 6:12pm UTC](https://discourse.julialang.org/t/dropping-rows-from-dataframe/49469/4 "2020-11-02T18:12:12Z")

</div>

I am not sure if I really understand what you mean with “broadcast”. But the problem of your initial question (as I understood it) should work with:

```julia
using DataFrames

function filter_captures!(df)
    filter!(row -> !startswith(row.A, "<"), df)
end

df = DataFrame(A = ["<X", "*X", "Y", "<Y"], B = ["C", "C", "C", "D"])
print(df, "\n\n")

filter_captures!(df)
print(df)

```

gives

```julia
4×2 DataFrame
│ Row │ A │ B │
│ │ String │ String │
├─────┼────────┼────────┤
│ 1 │ <X │ C │
│ 2 │ *X │ C │
│ 3 │ Y │ C │
│ 4 │ <Y │ D │

2×2 DataFrame
│ Row │ A │ B │
│ │ String │ String │
├─────┼────────┼────────┤
│ 1 │ *X │ C │
│ 2 │ Y │ C │

```

---

<div class="post-metadata">

**Author:** ![nilshg](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nilshg/32/2283_2.png) [@nilshg](https://discourse.julialang.org/u/nilshg)\
**Post date:** [November 2, 2020, 6:15pm UTC](https://discourse.julialang.org/t/dropping-rows-from-dataframe/49469/5 "2020-11-02T18:15:32Z")

</div>

Not sure about `@linq`, but you can just use normal syntax:

```julia
julia> using DataFrames

julia> df = DataFrame(term = rand([string.('a':'z'); "<"], 100), b = rand(100));

julia> df[startswith.(df.term, "<"), :]
2×2 DataFrame
│ Row │ term │ b │
│ │ String │ Float64 │
├─────┼────────┼──────────┤
│ 1 │ < │ 0.918707 │
│ 2 │ < │ 0.942731 │

```

and negate with `.!`

---

<div class="post-metadata">

**Author:** ![alunap](https://avatars.discourse-cdn.com/v4/letter/a/58956e/32.png) [@alunap](https://discourse.julialang.org/u/alunap)\
**Post date:** [November 2, 2020, 6:17pm UTC](https://discourse.julialang.org/t/dropping-rows-from-dataframe/49469/6 "2020-11-02T18:17:30Z")

</div>

That did it! I went down the @linq route, and the examples showed that the where clause could broadcast .\< over a column, but it got me nowhere. I found the function would compile if I changed to

where(:term, :startswith.(:term, r"\[1\].\*"))

But that fails on run with an indecipherable error about ‘&’. But your simpler solution works. Thanks very much!

* * *

1. \<

---

<div class="post-metadata">

**Author:** ![alunap](https://avatars.discourse-cdn.com/v4/letter/a/58956e/32.png) [@alunap](https://discourse.julialang.org/u/alunap)\
**Post date:** [November 2, 2020, 6:19pm UTC](https://discourse.julialang.org/t/dropping-rows-from-dataframe/49469/7 "2020-11-02T18:19:58Z")

</div>

got it, at least in part. My original problem was looking at the example code for DataFrames which broadcasts using .\<, which I thought meant I should use .startswith(), when in fact it should be startswith.(). What you have there makes sense, I was clearly making it too complicated. Thanks very much.

---

<div class="post-metadata">

**Author:** ![bkamins](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bkamins/32/208538_2.png) [@bkamins](https://discourse.julialang.org/u/bkamins)\
**Post date:** [November 2, 2020, 6:26pm UTC](https://discourse.julialang.org/t/dropping-rows-from-dataframe/49469/8 "2020-11-02T18:26:29Z")

</div>

you can also write:

```julia
filter(:term => startswith("<"), df)

```

---

<div class="post-metadata">

**Author:** ![alunap](https://avatars.discourse-cdn.com/v4/letter/a/58956e/32.png) [@alunap](https://discourse.julialang.org/u/alunap)\
**Post date:** [November 2, 2020, 6:28pm UTC](https://discourse.julialang.org/t/dropping-rows-from-dataframe/49469/9 "2020-11-02T18:28:38Z")

</div>

Shorter and shorter. Now I need to go away and work out exactly how that works. Thanks!

---

<div class="post-metadata">

**Author:** ![bkamins](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bkamins/32/208538_2.png) [@bkamins](https://discourse.julialang.org/u/bkamins)\
**Post date:** [November 2, 2020, 6:30pm UTC](https://discourse.julialang.org/t/dropping-rows-from-dataframe/49469/10 "2020-11-02T18:30:01Z")

</div>

> **[How is =\> used in DataFrames.jl?](https://bkamins.github.io/julialang/2020/07/17/pair.html)**
>
> Introduction

In 0.22 that will be soon released small changes will be introduced, but in this case nothing changes.

---

<div class="post-metadata">

**Author:** ![pdeffebach](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pdeffebach/32/10320_2.png) [@pdeffebach](https://discourse.julialang.org/u/pdeffebach)\
**Post date:** [November 2, 2020, 6:34pm UTC](https://discourse.julialang.org/t/dropping-rows-from-dataframe/49469/11 "2020-11-02T18:34:23Z")

</div>

To answer your original question with DataFramesMeta

```julia
julia> df = DataFrame(a = ["xy", "xz", "yz"])
3×1 DataFrame
│ Row │ a │
│ │ String │
├─────┼────────┤
│ 1 │ xy │
│ 2 │ xz │
│ 3 │ yz │

julia> @where(df, startswith.(:a, r"x") .== false)
1×1 DataFrame
│ Row │ a │
│ │ String │
├─────┼────────┤
│ 1 │ yz │

```

---

<div class="post-metadata">

**Author:** ![alunap](https://avatars.discourse-cdn.com/v4/letter/a/58956e/32.png) [@alunap](https://discourse.julialang.org/u/alunap)\
**Post date:** [November 2, 2020, 6:39pm UTC](https://discourse.julialang.org/t/dropping-rows-from-dataframe/49469/12 "2020-11-02T18:39:31Z")

</div>

got it, so within @linq it becomes

@linq df |\>  
select(:user, :term, :policy, :time) |\>  
where(startswith.(:term, r"\<") .== false)

correct? So now I have several approaches, very useful.

---

<div class="post-metadata">

**Author:** ![pdeffebach](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pdeffebach/32/10320_2.png) [@pdeffebach](https://discourse.julialang.org/u/pdeffebach)\
**Post date:** [November 2, 2020, 6:40pm UTC](https://discourse.julialang.org/t/dropping-rows-from-dataframe/49469/13 "2020-11-02T18:40:42Z")

</div>

yeah that sounds right