# Inconsistent results when using split-apply-combine for DataFrames

**URL:** <https://discourse.julialang.org/t/inconsistent-results-when-using-split-apply-combine-for-dataframes/27705>\
**Category:** New to Julia\
**Created:** [August 19, 2019, 8:11am UTC](https://discourse.julialang.org/t/inconsistent-results-when-using-split-apply-combine-for-dataframes/27705 "2019-08-19T08:11:47Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![rocflight83](https://avatars.discourse-cdn.com/v4/letter/r/3ec8ea/32.png) [@rocflight83](https://discourse.julialang.org/u/rocflight83)\
**Post date:** [August 19, 2019, 8:11am UTC](https://discourse.julialang.org/t/inconsistent-results-when-using-split-apply-combine-for-dataframes/27705/1 "2019-08-19T08:11:47Z")

</div>

I’m looking to add a column to my DataFrame combo2 that represents the ranking of DK\_points by each Pos (position) group with the following code.

```julia
sort!(combo2, [:Pos, :DK_points], rev=[true, true])
dk_rank = []

for pos in groupby(combo2, :Pos)
    append!(dk_rank, 1:size(pos, 1))
end
insertcols!(combo2, 6, DK_rank=dk_rank)

```

However, one particular value for Pos, “QB”, gives unusual results with the ranking starting at 14 rather than the desired 1.

```julia
by(combo2, :Pos, y->DataFrame(minRank = minimum(y[:,:DK_rank])))

```

```julia
	Pos	minRank
String	Int64
1	QB	14
2	RB	1
3	WR	1
4	TE	1
5	DST	1

```

Very confused why this would happen.

Also, wondering if there are better ways to accomplish adding a ranking for multiple columns in a data set by each group of another particular column.

---

<div class="post-metadata">

**Author:** ![bkamins](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bkamins/32/208538_2.png) [@bkamins](https://discourse.julialang.org/u/bkamins)\
**Post date:** [August 19, 2019, 9:14am UTC](https://discourse.julialang.org/t/inconsistent-results-when-using-split-apply-combine-for-dataframes/27705/2 "2019-08-19T09:14:16Z")

</div>

First, how you can do it more easily is:

```julia
by(combo2, :Pos, y -> (DK_rank=axes(y, 1)))

```

if your data frame is sorted  
or

```julia
by(combo2, :Pos, y -> (DK_rank=sortperm(y, :DK_points, rev=true)))

```

if it is not sorted yet.

Now the problem you encounter is most likely related to sorting (but I am not sure). Can you please give the result of the following operation (or share your source data frame):

```julia
by(combo2, :Pos, y->DataFrame(len=nrow(y), rankrange = extrema(y[:,:DK_rank], isok=issorted(y[:,:DK_rank]))))

```

---

<div class="post-metadata">

**Author:** ![bkamins](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bkamins/32/208538_2.png) [@bkamins](https://discourse.julialang.org/u/bkamins)\
**Post date:** [August 19, 2019, 9:53am UTC](https://discourse.julialang.org/t/inconsistent-results-when-using-split-apply-combine-for-dataframes/27705/3 "2019-08-19T09:53:09Z")

</div>

Additionally - I have just checked your code on some random data and I could not reproduce the problem.

---

<div class="post-metadata">

**Author:** ![rocflight83](https://avatars.discourse-cdn.com/v4/letter/r/3ec8ea/32.png) [@rocflight83](https://discourse.julialang.org/u/rocflight83)\
**Post date:** [August 19, 2019, 9:54am UTC](https://discourse.julialang.org/t/inconsistent-results-when-using-split-apply-combine-for-dataframes/27705/4 "2019-08-19T09:54:28Z")

</div>

```julia
	Pos	len	rankrange	isok
String	Int64	Tuple…	Bool
1	QB	39	(14, 52)	true
2	RB	53	(1, 56)	false
3	WR	56	(1, 39)	false
4	TE	52	(1, 53)	false
5	DST	33	(1, 33)	true

```

Appears that the append order is out of whack. But not really sure why.

---

<div class="post-metadata">

**Author:** ![bkamins](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bkamins/32/208538_2.png) [@bkamins](https://discourse.julialang.org/u/bkamins)\
**Post date:** [August 19, 2019, 10:00am UTC](https://discourse.julialang.org/t/inconsistent-results-when-using-split-apply-combine-for-dataframes/27705/5 "2019-08-19T10:00:12Z")

</div>

can you send me the data privately if you cannot share them openly? (as I have said - I have checked your codes on random data and they are OK). Also please confirm if the two methods I suggested work correctly on your data.

---

<div class="post-metadata">

**Author:** ![rocflight83](https://avatars.discourse-cdn.com/v4/letter/r/3ec8ea/32.png) [@rocflight83](https://discourse.julialang.org/u/rocflight83)\
**Post date:** [August 19, 2019, 10:26am UTC](https://discourse.julialang.org/t/inconsistent-results-when-using-split-apply-combine-for-dataframes/27705/6 "2019-08-19T10:26:37Z")

</div>

If I check for “sortedness” just on DK\_points, seems sorting indeed didn’t work correctly.

```julia
sort!(combo2, [:Pos, :DK_points], rev=[true, true])
by(combo2, :Pos, y->DataFrame(len=nrow(y), isok=issorted(y[:,:DK_points])))

```

```julia
Pos	len	isok
String	Int64	Bool
1	QB	39	false
2	RB	53	false
3	WR	56	false
4	TE	52	false
5	DST	33	false

```

---

<div class="post-metadata">

**Author:** ![rocflight83](https://avatars.discourse-cdn.com/v4/letter/r/3ec8ea/32.png) [@rocflight83](https://discourse.julialang.org/u/rocflight83)\
**Post date:** [August 19, 2019, 10:39am UTC](https://discourse.julialang.org/t/inconsistent-results-when-using-split-apply-combine-for-dataframes/27705/7 "2019-08-19T10:39:44Z")

</div>

With both of the methods you provided, if I insert the rank column back to the dataframe using

```julia
m = by(combo2, :Pos, y -> (DK_rank=sortperm(y, :DK_points, rev=true)))[:, :x1]
insertcols!(combo2, 6, DK_rank=m)

```

I get the same result as before

---

<div class="post-metadata">

**Author:** ![bkamins](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bkamins/32/208538_2.png) [@bkamins](https://discourse.julialang.org/u/bkamins)\
**Post date:** [August 19, 2019, 1:00pm UTC](https://discourse.julialang.org/t/inconsistent-results-when-using-split-apply-combine-for-dataframes/27705/8 "2019-08-19T13:00:56Z")

</div>

> [@rocflight83](#):
>
> ```julia
> Pos minRank
> String Int64
> 1 QB 14
> 2 RB 1
> 3 WR 1
> 4 TE 1
> 5 DST 1
> 
> ```

Please make sure that your `combo2` data frame is sorted. In your post `:Pos` column does not seem to be sorted in reverse order. The order should be like:

```julia
5×2 DataFrame
│ Row │ Pos │ minRank │
│ │ String │ Int64 │
├─────┼────────┼─────────┤
│ 1 │ WR │ 1 │
│ 2 │ TE │ 1 │
│ 3 │ RB │ 1 │
│ 4 │ QB │ 1 │
│ 5 │ DST │ 1 │

```