# Deleting rows from a dataframe containing stock symbols not found in another dataframe

**URL:** <https://discourse.julialang.org/t/deleting-rows-from-a-dataframe-containing-stock-symbols-not-found-in-another-dataframe/86882>\
**Category:** New to Julia\
**Created:** [September 7, 2022, 12:34pm UTC](https://discourse.julialang.org/t/deleting-rows-from-a-dataframe-containing-stock-symbols-not-found-in-another-dataframe/86882 "2022-09-07T12:34:12Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![Nash](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nash/32/14482_2.png) [@Nash](https://discourse.julialang.org/u/Nash)\
**Post date:** [September 7, 2022, 12:34pm UTC](https://discourse.julialang.org/t/deleting-rows-from-a-dataframe-containing-stock-symbols-not-found-in-another-dataframe/86882/1 "2022-09-07T12:34:12Z")

</div>

I have two dataframes, df\_A, and df\_B, each containing stock data (of different kinds) and each containing a column of stock symbols. However, df\_A is longer than df\_B, because it contains more stock symbols than df\_B.

How do I delete the rows in df\_A that contain stock symbols not found in df\_B?

---

<div class="post-metadata">

**Author:** ![junder873](https://avatars.discourse-cdn.com/v4/letter/j/e95f7d/32.png) [@junder873](https://discourse.julialang.org/u/junder873)\
**Post date:** [September 7, 2022, 12:42pm UTC](https://discourse.julialang.org/t/deleting-rows-from-a-dataframe-containing-stock-symbols-not-found-in-another-dataframe/86882/2 "2022-09-07T12:42:06Z")

</div>

Probably the easiest way is using `innerjoin`, e.g.:

```julia
df_a = DataFrame(ticker=[:a, :b, :c], ret=randn(3))

df_b = DataFrame(ticker=[:a, :b], price=[2, 3])

innerjoin(
    df_a,
    df_b,
    on=:ticker
)
# 2×3 DataFrame
# Row │ ticker ret price 
# │ Symbol Float64 Int64 
# ─────┼────────────────────────────
# 1 │ a -0.0907065 2 
# 2 │ b 0.00463602 3

```

---

<div class="post-metadata">

**Author:** ![Nash](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nash/32/14482_2.png) [@Nash](https://discourse.julialang.org/u/Nash)\
**Post date:** [September 7, 2022, 12:46pm UTC](https://discourse.julialang.org/t/deleting-rows-from-a-dataframe-containing-stock-symbols-not-found-in-another-dataframe/86882/3 "2022-09-07T12:46:29Z")

</div>

> [@junder873](#):
>
> ```julia
> innerjoin(
> df_a,
> df_b,
> on=:ticker
> )
> 
> ```

The problem I have is that the dataframes are very large and my system runs out of memory during an inner join. I was hoping to delete rows that are not necessary, and then proceed to a step where I join the two dataframes.

---

<div class="post-metadata">

**Author:** ![junder873](https://avatars.discourse-cdn.com/v4/letter/j/e95f7d/32.png) [@junder873](https://discourse.julialang.org/u/junder873)\
**Post date:** [September 7, 2022, 12:58pm UTC](https://discourse.julialang.org/t/deleting-rows-from-a-dataframe-containing-stock-symbols-not-found-in-another-dataframe/86882/4 "2022-09-07T12:58:48Z")

</div>

Are there repeated stock symbols in df\_b? If so, the result of an innerjoin like this can be bigger than the original (which is consistent with your out of memory error). You can check this by adding a `validate=(false, true)` to the innerjoin. One way I have gotten around this is to create a DataFrame based on df\_b that is just the unique stock symbols before the join:

```julia
df_c = unique(df_b[:, [:ticker]])

innerjoin(
    df_a,
    df_c,
    on=:ticker,
    validate=(false, true)
)

```

---

<div class="post-metadata">

**Author:** ![digital\_carver](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/digital_carver/32/33818_2.png) [@digital\_carver](https://discourse.julialang.org/u/digital_carver)\
**Post date:** [September 7, 2022, 5:26pm UTC](https://discourse.julialang.org/t/deleting-rows-from-a-dataframe-containing-stock-symbols-not-found-in-another-dataframe/86882/5 "2022-09-07T17:26:43Z")

</div>

```julia
subset!(df_A, 
        :stocksyms => s -> (!isnothing).(indexin(s, df_B.stocksyms)))

```

`indexin` returns `nothing` in places where its first argument has an element that isn’t in the second argument, so this filters the dataframe based on that.

---

<div class="post-metadata">

**Author:** ![jules](https://avatars.discourse-cdn.com/v4/letter/j/41988e/32.png) [@jules](https://discourse.julialang.org/u/jules)\
**Post date:** [September 7, 2022, 5:52pm UTC](https://discourse.julialang.org/t/deleting-rows-from-a-dataframe-containing-stock-symbols-not-found-in-another-dataframe/86882/6 "2022-09-07T17:52:58Z")

</div>

Or just make a set of the symbols separately and then filter based on that.  
Something like (untested):

```julia
syms = Set(df_B.stocksyms)
subset!(df_A, :stocksyms => ss -> [s in syms for s in ss])

```