# Groupby for regular array
**URL:** https://discourse.julialang.org/t/groupby-for-regular-array/9877
**Category:** General Usage
**Created:** [March 22, 2018, 5:26am UTC](https://discourse.julialang.org/t/groupby-for-regular-array/9877 "2018-03-22T05:26:48Z")
**Posts on this page:** 11
**Page:** 1
### Author:  [@tk3369](https://discourse.julialang.org/u/tk3369)
#### Post date: [March 22, 2018, 5:26am UTC](https://discourse.julialang.org/t/groupby-for-regular-array/9877/1 "2018-03-22T05:26:48Z")
Hi there.
Does anyone know a function or a package that can do the same DataFrame `groupby` function on a regular array? e.g.
```julia
julia> A = [1,2,3,4,1,2,4,4]
8-element Array{Int64,1}:
1
2
3
4
1
2
4
4
```
Then, `sortperm` gets me half way:
```julia
julia> sortperm(A)
8-element Array{Int64,1}:
1
5
2
6
3
4
7
8
```
…as I want to get something like:
```julia
[1,5], [2,6], [3], [4,7,8]
```
---
### Author:  [@tk3369](https://discourse.julialang.org/u/tk3369)
#### Post date: [March 22, 2018, 6:04am UTC](https://discourse.julialang.org/t/groupby-for-regular-array/9877/2 "2018-03-22T06:04:33Z")
Never mind. There is already a good post.
> [@Allocation - groupby](https://discourse.julialang.org/t/allocation-groupby/7505):
>
> I’m attempting to create a function groupby(f, itr) that returns a dictionary whose keys are the results of applying f to the elements of itr, and the values are some collection of the elements themselves. Similar to Mathematica’s [GroupBy](http://reference.wolfram.com/language/ref/GroupBy.html). Example, groupby(iseven, 1:5) --\> Dict(true=\>(2, 4), false =\>(1, 3, 5)). The type of the values doesn’t need to be a tuple if there’s a better alternative. This is partly because I need it, and partly because I’m learning Julia. So, both answers that explain …
---
### Author:  [@davidanthoff](https://discourse.julialang.org/u/davidanthoff)
#### Post date: [March 22, 2018, 5:02pm UTC](https://discourse.julialang.org/t/groupby-for-regular-array/9877/3 "2018-03-22T17:02:48Z")
[Query.jl](https://github.com/davidanthoff/Query.jl) works on regular arrays, so you can just use its [@groupby](http://www.david-anthoff.com/Query.jl/stable/experimental.html#The-@groupby-command-1) command.
---
### Author:  [@tk3369](https://discourse.julialang.org/u/tk3369)
#### Post date: [March 23, 2018, 4:36am UTC](https://discourse.julialang.org/t/groupby-for-regular-array/9877/4 "2018-03-23T04:36:31Z")
Interesting. I’ve tried but unsure how to get the indices rather than values.
```julia
julia> A = [1,2,3,4,1,2,4,4]
8-element Array{Int64,1}:
1
2
3
4
1
2
4
4
julia> A |> @groupby(_)
?-element query result
[1, 1]
[2, 2]
[3]
[4, 4, 4]
```
---
### Author:  [@tk3369](https://discourse.julialang.org/u/tk3369)
#### Post date: [March 23, 2018, 4:52am UTC](https://discourse.julialang.org/t/groupby-for-regular-array/9877/5 "2018-03-23T04:52:31Z")
I got it!
```julia
julia> 1:length(A) |> @groupby(A[_]) |> collect
4-element Array{QueryOperators.Grouping{Any,Int64},1}:
[1, 5]
[2, 6]
[3]
[4, 7, 8]
```
---
### Author:  [@tk3369](https://discourse.julialang.org/u/tk3369)
#### Post date: [March 23, 2018, 7:44am UTC](https://discourse.julialang.org/t/groupby-for-regular-array/9877/6 "2018-03-23T07:44:07Z")
I was curious about performance. Since it’s not a tough problem, I’ve created a custom groupby function and compared it with Query.jl. Just sharing results:
[https://gist.github.com/tk3369/f979a4292fd696bee753f37cae93b45c](https://gist.github.com/tk3369/f979a4292fd696bee753f37cae93b45c)
---
### Author:  [@oheil](https://discourse.julialang.org/u/oheil)
#### Post date: [March 23, 2018, 12:41pm UTC](https://discourse.julialang.org/t/groupby-for-regular-array/9877/7 "2018-03-23T12:41:32Z")
function sampleRanks of package NormalizeQuantiles.jl maybe also a solution. Just for your information.
> **[GitHub - oheil/NormalizeQuantiles.jl: NormalizeQuantiles.jl implements...](https://github.com/oheil/NormalizeQuantiles.jl)**
>
> NormalizeQuantiles.jl implements quantile normalization - GitHub - oheil/NormalizeQuantiles.jl: NormalizeQuantiles.jl implements quantile normalization
---
### Author:  [@oheil](https://discourse.julialang.org/u/oheil)
#### Post date: [March 23, 2018, 12:41pm UTC](https://discourse.julialang.org/t/groupby-for-regular-array/9877/8 "2018-03-23T12:41:35Z")
```julia
julia> Pkg.add("NormalizeQuantiles")
julia> using NormalizeQuantiles
julia> A = [1,2,3,4,1,2,4,4];
julia> (r,m) = sampleRanks(A,resultMatrix=true);
julia> m
Dict{Int64,Array{Int64,N} where N} with 4 entries:
4 => [4, 7, 8]
2 => [2, 6]
3 => [3]
1 => [1, 5]
```
---
### Author:  [@oheil](https://discourse.julialang.org/u/oheil)
#### Post date: [March 23, 2018, 12:47pm UTC](https://discourse.julialang.org/t/groupby-for-regular-array/9877/9 "2018-03-23T12:47:08Z")
julia\> @time 1:length(A) |\> @groupby(A[\_]) |\> collect
0.027596 seconds (9.28 k allocations: 522.957 KiB)
julia\> @time A |\> @groupby(\_)
0.004368 seconds (1.95 k allocations: 118.646 KiB)
julia\> @time (r,m) = sampleRanks(A,resultMatrix=true)
0.000128 seconds (234 allocations: 11.938 KiB)
---
### Author:  [@tk3369](https://discourse.julialang.org/u/tk3369)
#### Post date: [March 25, 2018, 7:19am UTC](https://discourse.julialang.org/t/groupby-for-regular-array/9877/10 "2018-03-25T07:19:11Z")
Need to use BenchmarkTools for proper benchmarking.
Tried NormalizedQuantiles. It returns a `Dict`, which is nice, but it’s much slower than others and it does not scale with larger arrays. See [updated gist](https://gist.github.com/tk3369/f979a4292fd696bee753f37cae93b45c) for benchmark details.
| Method | Function | 8 elements | 1,000 elements | 10,000 elements |
| --- | --- | --- | --- | --- |
| Custom function | woz | 1 μs | 20 μs | 174 μs |
| Query | foo | 4 μs | 49 μs | 425 μs |
| NormalizedQuantiles | bar | 47 μs | 10,658 μs | 133,827 μs |
---
### Author:  [@oheil](https://discourse.julialang.org/u/oheil)
#### Post date: [March 25, 2018, 10:35am UTC](https://discourse.julialang.org/t/groupby-for-regular-array/9877/11 "2018-03-25T10:35:28Z")
Your custom function is quite fast. I think the main performance loss is because I have to expect dirty data, e.g.
A = [1,2,NaN,3,4,1,“5,0”,5.0,2,4,4]
where anything which is not of type “a number in general” is NA (not available, missing, …, like NA in R)
However, while comparing with Query.jl and your function I found a bug in my code (if the last value in the array is NA or all values are NA), which is now resolved.
Next step is to analyse your code in deep so maybe I can improve my code.
@groupby seems to be most versatile as it also groups e.g. strings: A = [“a”,“a”,“b”,“a”,“b”,“c”] or any other mix of types.