# Indices of a sub-dataframe

**URL:** <https://discourse.julialang.org/t/indices-of-a-sub-dataframe/96650>\
**Category:** Data\
**Tags:** dataframes\
**Created:** [March 27, 2023, 9:21am UTC](https://discourse.julialang.org/t/indices-of-a-sub-dataframe/96650 "2023-03-27T09:21:55Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![johnbb](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/johnbb/32/34233_2.png) [@johnbb](https://discourse.julialang.org/u/johnbb)\
**Post date:** [March 27, 2023, 9:21am UTC](https://discourse.julialang.org/t/indices-of-a-sub-dataframe/96650/1 "2023-03-27T09:21:55Z")

</div>

Assume I have two dataframes A and B where A is a subset of B. How can I get the rowindices of A?

---

<div class="post-metadata">

**Author:** ![bkamins](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bkamins/32/208538_2.png) [@bkamins](https://discourse.julialang.org/u/bkamins)\
**Post date:** [March 27, 2023, 9:47am UTC](https://discourse.julialang.org/t/indices-of-a-sub-dataframe/96650/2 "2023-03-27T09:47:36Z")

</div>

do you mean that `B` is a `DataFrame` and `A` is a `SubDataFrame` whose `parent` is `B` and you want to know which rows of `B` constitute `A`? If yes then this is done using the `parentindices` function.

---

<div class="post-metadata">

**Author:** ![johnbb](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/johnbb/32/34233_2.png) [@johnbb](https://discourse.julialang.org/u/johnbb)\
**Post date:** [March 27, 2023, 10:29am UTC](https://discourse.julialang.org/t/indices-of-a-sub-dataframe/96650/3 "2023-03-27T10:29:12Z")

</div>

My actual problem is that I want the `df` indices of the following

```julia
combine(groupby(df, :group), :myvar => u -> sort(u, rev=true)[1:3])

```

Is there an easy way?

---

<div class="post-metadata">

**Author:** ![rocco\_sprmnt21](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rocco_sprmnt21/32/20127_2.png) [@rocco\_sprmnt21](https://discourse.julialang.org/u/rocco_sprmnt21)\
**Post date:** [March 27, 2023, 11:34am UTC](https://discourse.julialang.org/t/indices-of-a-sub-dataframe/96650/4 "2023-03-27T11:34:33Z")

</div>

try

```julia
indexin(eachrow(A),eachrow(B))

```

---

<div class="post-metadata">

**Author:** ![johnbb](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/johnbb/32/34233_2.png) [@johnbb](https://discourse.julialang.org/u/johnbb)\
**Post date:** [March 27, 2023, 11:45am UTC](https://discourse.julialang.org/t/indices-of-a-sub-dataframe/96650/5 "2023-03-27T11:45:46Z")

</div>

Thanks, that seems to work. Only had to rename the column name created by `combine` in the example above.

---

<div class="post-metadata">

**Author:** ![rocco\_sprmnt21](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rocco_sprmnt21/32/20127_2.png) [@rocco\_sprmnt21](https://discourse.julialang.org/u/rocco_sprmnt21)\
**Post date:** [March 27, 2023, 12:32pm UTC](https://discourse.julialang.org/t/indices-of-a-sub-dataframe/96650/6 "2023-03-27T12:32:34Z")

</div>

> [@johnbb](#):
>
> My actual problem is that I want the `df` indices of the following
> 
> ```julia
> combine(groupby(df, :group), :myvar => u -> sort(u, rev=true)[1:3])
> 
> ```
> 
> Is there an easy way?

I had only read the first message and replied to that.  
I don’t know what to say about this clarification, as it is not clear to me what you are trying to do and what you would like to achieve.  
If you post a minimal example complete with data to think about, you will surely receive more precise and complete answers.

---

<div class="post-metadata">

**Author:** ![johnbb](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/johnbb/32/34233_2.png) [@johnbb](https://discourse.julialang.org/u/johnbb)\
**Post date:** [March 27, 2023, 12:44pm UTC](https://discourse.julialang.org/t/indices-of-a-sub-dataframe/96650/7 "2023-03-27T12:44:45Z")

</div>

Ok. Below is an example including a solution where the aim is to find the indices of the three largest values in each group

```julia
B = DataFrame(group = rand(1:4,40), values = rand(40))
A = combine(groupby(B, :group), :values => u -> sort(u, rev=true)[1:3])
rename!(A, :values_function => :values)
indexin(eachrow(A), eachrow(B))

```

---

<div class="post-metadata">

**Author:** ![bkamins](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bkamins/32/208538_2.png) [@bkamins](https://discourse.julialang.org/u/bkamins)\
**Post date:** [March 27, 2023, 12:45pm UTC](https://discourse.julialang.org/t/indices-of-a-sub-dataframe/96650/8 "2023-03-27T12:45:47Z")

</div>

There several ways to do it, but the simplest is to add `df.id = axes(df, 1)` column and then refer to it in the operations you perform.

---

<div class="post-metadata">

**Author:** ![rocco\_sprmnt21](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rocco_sprmnt21/32/20127_2.png) [@rocco\_sprmnt21](https://discourse.julialang.org/u/rocco_sprmnt21)\
**Post date:** [March 27, 2023, 1:02pm UTC](https://discourse.julialang.org/t/indices-of-a-sub-dataframe/96650/9 "2023-03-27T13:02:13Z")

</div>

perhaps not the simplest, …

```julia
grps=groupby(B,:group)

[partialsort(tuple.(parentindices(g)[1], g.values),1:3,by=last) for g in grps]

```

or

```julia
bestningroup(n)=g->partialsort(tuple.(parentindices(g)[1], g.values),1:n,by=last)
`combine(bestningroup(3), grps)`
julia> combine(bestningroup(3), grps)
12×2 DataFrame
 Row │ group x1
     │ Int64 Tuple…
─────┼────────────────────────
   1 │ 1 (12, 0.100642)
   2 │ 1 (19, 0.125845)
   3 │ 1 (1, 0.253362)
   4 │ 2 (22, 0.0399776)
   5 │ 2 (39, 0.102041)
   6 │ 2 (21, 0.194081)
   7 │ 3 (40, 0.0169355)
   8 │ 3 (32, 0.157626)
   9 │ 3 (31, 0.187634)
  10 │ 4 (5, 0.111986)
  11 │ 4 (26, 0.123399)
  12 │ 4 (27, 0.258254)

```

and for the largest n

```julia
julia> bestningroup(n, col)=g->partialsort(tuple.(parentindices(g)[1], g[:,col]),1:n,by=last,lt=!isless)
bestningroup (generic function with 2 methods)

julia> combine(bestningroup(3,:values), grps)
12×2 DataFrame
 Row │ group x1
     │ Int64 Tuple…
─────┼───────────────────────
   1 │ 1 (3, 0.79187)
   2 │ 1 (9, 0.724257)
   3 │ 1 (33, 0.692813)
   4 │ 2 (36, 0.990465)
   5 │ 2 (30, 0.818662)
   6 │ 2 (6, 0.804834)
   7 │ 3 (13, 0.760777)
   8 │ 3 (10, 0.461795)
   9 │ 3 (38, 0.431897)
  10 │ 4 (15, 0.9416)
  11 │ 4 (24, 0.731547)
  12 │ 4 (20, 0.634004)

```

---

<div class="post-metadata">

**Author:** ![tbeason](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tbeason/32/15898_2.png) [@tbeason](https://discourse.julialang.org/u/tbeason)\
**Post date:** [March 27, 2023, 1:44pm UTC](https://discourse.julialang.org/t/indices-of-a-sub-dataframe/96650/10 "2023-03-27T13:44:53Z")

</div>

@bkamins Sort of related to your response: have you ever considered adding a `rownum` shortcut function sort of like `nrow` to do this sort of thing in the “minilanguage”?

---

<div class="post-metadata">

**Author:** ![Dan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dan/32/42581_2.png) [@Dan](https://discourse.julialang.org/u/Dan)\
**Post date:** [March 27, 2023, 2:22pm UTC](https://discourse.julialang.org/t/indices-of-a-sub-dataframe/96650/11 "2023-03-27T14:22:40Z")

</div>

Inspired by @rocco_sprmnt21 answers:

```julia
combine(groupby(B, :group), 
  :values => u -> parentindices(u)[1][sortperm(u, rev=true)[1:3]])

```

P.S. `partialsortperm` is faster than `sortperm`, when time critical and large groups it should be preferred.

---

<div class="post-metadata">

**Author:** ![rocco\_sprmnt21](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rocco_sprmnt21/32/20127_2.png) [@rocco\_sprmnt21](https://discourse.julialang.org/u/rocco_sprmnt21)\
**Post date:** [March 27, 2023, 3:47pm UTC](https://discourse.julialang.org/t/indices-of-a-sub-dataframe/96650/12 "2023-03-27T15:47:01Z")

</div>

> [@Dan](#):
>
> `... :values => u -> parentindices(u) ...`

I wonder how this expression works.  
I understood that in the col=\>fun syntax the function fun is passed only the vector of the values of the col column. How does (would) in this case to go back to the parentindices having only values?

Does this mean that in some cases, in addition to the values, some other information is propagated towards fun?  
I ask because in some cases it would have been convenient to have the name of the column in addition to the values.

---

<div class="post-metadata">

**Author:** ![pdeffebach](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pdeffebach/32/10320_2.png) [@pdeffebach](https://discourse.julialang.org/u/pdeffebach)\
**Post date:** [March 27, 2023, 3:54pm UTC](https://discourse.julialang.org/t/indices-of-a-sub-dataframe/96650/13 "2023-03-27T15:54:18Z")

</div>

I don’t think this helps answer the question. The answer is no, the function acts on a `view` of the vectors and `view`s, have `parentindices` defined. It has nothing to do with data frames.

```julia
julia> x = [1, 2, 3];

julia> y = view(x, 1:2);

julia> parentindices(y)
(1:2,)

```

---

<div class="post-metadata">

**Author:** ![rocco\_sprmnt21](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rocco_sprmnt21/32/20127_2.png) [@rocco\_sprmnt21](https://discourse.julialang.org/u/rocco_sprmnt21)\
**Post date:** [March 27, 2023, 3:56pm UTC](https://discourse.julialang.org/t/indices-of-a-sub-dataframe/96650/14 "2023-03-27T15:56:09Z")

</div>

Sure clarifies for me that parentindices is not a specific function of subgroups, as I thought.  
Thank you.

PS  
I still have to figure out what exactly is being passed to fun in this case

---

<div class="post-metadata">

**Author:** ![pdeffebach](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pdeffebach/32/10320_2.png) [@pdeffebach](https://discourse.julialang.org/u/pdeffebach)\
**Post date:** [March 27, 2023, 4:05pm UTC](https://discourse.julialang.org/t/indices-of-a-sub-dataframe/96650/15 "2023-03-27T16:05:16Z")

</div>

It’s a `view` of the underlying column in the data frame, a `SubDataFrame` is basically a `DataFrame` of `view`s.

---

<div class="post-metadata">

**Author:** ![rocco\_sprmnt21](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rocco_sprmnt21/32/20127_2.png) [@rocco\_sprmnt21](https://discourse.julialang.org/u/rocco_sprmnt21)\
**Post date:** [March 27, 2023, 4:10pm UTC](https://discourse.julialang.org/t/indices-of-a-sub-dataframe/96650/16 "2023-03-27T16:10:33Z")

</div>

and the view of a vector is not a simple collection of values but a complex structure with lots of information, right?  
Could it be something like a pointer to the start of the parent array and the view offsets!?!

---

<div class="post-metadata">

**Author:** ![pdeffebach](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pdeffebach/32/10320_2.png) [@pdeffebach](https://discourse.julialang.org/u/pdeffebach)\
**Post date:** [March 27, 2023, 4:17pm UTC](https://discourse.julialang.org/t/indices-of-a-sub-dataframe/96650/17 "2023-03-27T16:17:05Z")

</div>

[Here](https://docs.julialang.org/en/v1/base/arrays/#Views-(SubArrays-and-other-view-types)) is the documentation for `view`s.

---

<div class="post-metadata">

**Author:** ![rocco\_sprmnt21](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rocco_sprmnt21/32/20127_2.png) [@rocco\_sprmnt21](https://discourse.julialang.org/u/rocco_sprmnt21)\
**Post date:** [March 27, 2023, 4:22pm UTC](https://discourse.julialang.org/t/indices-of-a-sub-dataframe/96650/18 "2023-03-27T16:22:26Z")

</div>

thanks (I need to read more).  
But the fact that sometimes double TAB after the . provides some answers helps my laziness and I read the manuals little

```julia
julia> x = [3,4,5, 2, 3];

julia> y = view(x, 1:2)
2-element view(::Vector{Int64}, 1:2) with eltype Int64:
 3
 4

julia> y. #pressing TAB twice
indices offset1 parent stride1
julia> y.parent
5-element Vector{Int64}:
 3
 4
 5
 2
 3

julia> y.indices
(1:2,)

```

I tried it to see what happens 😄

```julia
julia> y.indices = (1:3,)
ERROR: setfield!: immutable struct of type SubArray cannot be changed
Stacktrace:
 [1] setproperty!(x::SubArray{Int64, 1, Vector{Int64}, Tuple{UnitRange{Int64}}, true}, f::Symbol, v::Tuple{UnitRange{Int64}})

```

---

<div class="post-metadata">

**Author:** ![bkamins](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bkamins/32/208538_2.png) [@bkamins](https://discourse.julialang.org/u/bkamins)\
**Post date:** [March 27, 2023, 9:20pm UTC](https://discourse.julialang.org/t/indices-of-a-sub-dataframe/96650/19 "2023-03-27T21:20:38Z")

</div>

> [@tbeason](#):
>
> adding a `rownum` shortcut function sort of like `nrow`

I understand you want the row number in the parent data frame? If you would find it useful we could consider adding it. It was not asked for before.

@pdeffebach - a small note.  
Note that `parentindices` will not always return exactly what I assume was originally expected:

```julia
julia> using DataFrames

julia> df = DataFrame(id=[1,2,3,3,4,4])
6×1 DataFrame
 Row │ id
     │ Int64
─────┼───────
   1 │ 1
   2 │ 2
   3 │ 3
   4 │ 3
   5 │ 4
   6 │ 4

julia> gdf = groupby(view(df, 3:6, :), :id)
GroupedDataFrame with 2 groups based on key: id
First Group (2 rows): id = 3
 Row │ id
     │ Int64
─────┼───────
   1 │ 3
   2 │ 3
⋮
Last Group (2 rows): id = 4
 Row │ id
     │ Int64
─────┼───────
   1 │ 4
   2 │ 4

julia> combine(gdf, :id => v -> parentindices(v)[1], sdf -> parentindices(sdf)[1])
4×3 DataFrame
 Row │ id id_function x1
     │ Int64 Int64 Int64
─────┼───────────────────────────
   1 │ 3 3 3
   2 │ 3 4 4
   3 │ 4 5 5
   4 │ 4 6 6

```

---

<div class="post-metadata">

**Author:** ![tbeason](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tbeason/32/15898_2.png) [@tbeason](https://discourse.julialang.org/u/tbeason)\
**Post date:** [March 28, 2023, 12:56pm UTC](https://discourse.julialang.org/t/indices-of-a-sub-dataframe/96650/20 "2023-03-28T12:56:33Z")

</div>

Yea I use it all the time! I typically pass a random column to `x -> 1:length(x)`. Your `axes(x,1)` solution could be a bit better, but I think this is probably something that enough people use that it would make sense to have a performant shortcut.

[Next page](https://discourse.julialang.org/t/indices-of-a-sub-dataframe/96650.md?page=2)